Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update 0.8.4 #8514

Merged
merged 135 commits into from
Feb 20, 2024
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
135 commits
Select commit Hold shift + click to select a range
cb92050
[refactor] fixing issues in `syft/user`
khoaguin Feb 6, 2024
297e0b3
[refactor] mypy passes for files in `service/user`
khoaguin Feb 7, 2024
9f1e129
fix: change k8s port to 5800
tcp Feb 8, 2024
69ae274
[helm] fix container port
yashgorana Feb 8, 2024
e934baa
[ci] untag old betas
yashgorana Feb 9, 2024
9e8ed3c
[ci] move cleanup to azure tasks
yashgorana Feb 9, 2024
89261b3
update README with updated release version info
shubham3121 Feb 9, 2024
39e7f2a
Merge branch 'dev' into fix-mypy-issues-service-user
khoaguin Feb 12, 2024
932aa99
Merge branch 'dev' into yash/fix-cd-syft-dev
yashgorana Feb 12, 2024
bdaec6e
[ci] remove az cli login
yashgorana Feb 12, 2024
291e680
Merge branch 'dev' into fix/k8s-port
yashgorana Feb 12, 2024
5cee9bb
Merge branch 'dev' into fix-mypy-issues-service-user
kiendang Feb 12, 2024
543af6a
[cd] fix node 16 deprecation
yashgorana Feb 12, 2024
a3e26fe
[refactor] change ServiceRole.roles_for_level to classmethod
khoaguin Feb 13, 2024
74f761f
Switch type order
kiendang Feb 13, 2024
60da3d2
Merge branch 'dev' into fix-mypy-issues-service-user
kiendang Feb 13, 2024
0243c23
fix: patch coredns so k3d registry with custom port is accessible ins…
tcp Feb 13, 2024
dff79c1
Merge branch 'dev' into fix/k8s-port
tcp Feb 13, 2024
8f8199c
Merge branch 'dev' into fix-mypy-issues-service-user
kiendang Feb 13, 2024
2e50b74
test: patch ci
tcp Feb 13, 2024
69b9bf3
test: patch integration tests for k8s
tcp Feb 13, 2024
cf3ec3e
fix: doc sed
tcp Feb 13, 2024
51633f3
test: k8s patch coredns after test cluster creation
tcp Feb 13, 2024
aa3e45f
Remove unnecessary quotes around type
kiendang Feb 14, 2024
8fc658d
Merge branch 'dev' into fix-mypy-issues-service-user
kiendang Feb 14, 2024
970125c
Fix regex file matcher
kiendang Feb 14, 2024
65f83a8
trying macos arm64 runners
rasswanth-s Feb 14, 2024
7f808b7
commented out Mac Arm python version
rasswanth-s Feb 14, 2024
326b6fe
test hagrid launch
rasswanth-s Feb 14, 2024
952c083
try automating hagrid tests
rasswanth-s Feb 14, 2024
5bf9e56
include api/0.8 notebooks paths in workflow
rasswanth-s Feb 14, 2024
fde6bcb
modified to include absolute paths
rasswanth-s Feb 14, 2024
0da797f
modified python command
rasswanth-s Feb 14, 2024
26db521
modified port
rasswanth-s Feb 14, 2024
8ea74a2
Merge branch 'dev' into fix/k8s-port
yashgorana Feb 14, 2024
59d7c19
Added syft installation with data science packages
rasswanth-s Feb 14, 2024
93ae47a
modified orchestra deployment to k8s
rasswanth-s Feb 14, 2024
ea49b11
Merge branch 'dev' into fix-mypy-issues-service-user
kiendang Feb 14, 2024
c2436aa
Merge branch 'dev' into yash/fix-cd-syft-dev
rasswanth-s Feb 15, 2024
e15c8e4
Merge pull request #8471 from OpenMined/yash/fix-cd-syft-dev
rasswanth-s Feb 15, 2024
be62fe4
Merge branch 'dev' into fix-mypy-issues-service-user
khoaguin Feb 15, 2024
b0c3640
Merge pull request #8460 from khoaguin/fix-mypy-issues-service-user
kiendang Feb 15, 2024
003f32f
Merge branch 'dev' into fix/k8s-port
yashgorana Feb 15, 2024
e2b042f
Merge pull request #8469 from OpenMined/fix/k8s-port
yashgorana Feb 15, 2024
26df8f5
[syft]bump version
alfred-openmined-bot Feb 15, 2024
4d262b1
Added e2e.test.notebook to tox
rasswanth-s Feb 15, 2024
ac971fc
added syft version to tox e2e.test.notebook
rasswanth-s Feb 15, 2024
499633e
added syft version to tox e2e.test.notebook without printing version …
rasswanth-s Feb 15, 2024
4d2f317
lint
rasswanth-s Feb 15, 2024
864496c
fix e2e.test.notebook
rasswanth-s Feb 15, 2024
6543512
modified tox test
rasswanth-s Feb 15, 2024
64ca18f
fix minor spelling
rasswanth-s Feb 15, 2024
d70b2bf
skip collection
rasswanth-s Feb 15, 2024
588ecc7
Merge branch 'dev' into rasswanth/optimize-post-release-tests
rasswanth-s Feb 15, 2024
39f5abf
Co-Authored by: @khoaguin
rasswanth-s Feb 15, 2024
5cae6ee
Merge pull request #8493 from OpenMined/fix-protocol-bug
rasswanth-s Feb 15, 2024
ef72e35
[syft]bump version
alfred-openmined-bot Feb 15, 2024
f3da2da
Merge branch 'dev' into rasswanth/optimize-post-release-tests
rasswanth-s Feb 15, 2024
aee0a8c
removed redundant install steps
rasswanth-s Feb 15, 2024
b025faa
re-enabled syft install tests
rasswanth-s Feb 15, 2024
cb5cca8
Added post release tests to cd-syft release pipeline
rasswanth-s Feb 15, 2024
3e45a21
removed unsued comments
rasswanth-s Feb 15, 2024
e34f8c4
fix the version branch to dev
rasswanth-s Feb 15, 2024
8e6b98e
Merge pull request #8486 from OpenMined/rasswanth/optimize-post-relea…
rasswanth-s Feb 15, 2024
726814f
[syft]bump version
alfred-openmined-bot Feb 15, 2024
43cb69c
added syft version to github.event.inputs for dependent workflow
rasswanth-s Feb 15, 2024
5dfad0b
Merge pull request #8494 from OpenMined/rasswanth/fix-syft-version-cd
rasswanth-s Feb 15, 2024
f43e5df
shift to using workflow_call
rasswanth-s Feb 15, 2024
2f829cc
Merge pull request #8495 from OpenMined/rasswanth/fix-syft-version-cd
rasswanth-s Feb 15, 2024
54f5937
Bump pre-commit/action from 3.0.0 to 3.0.1
dependabot[bot] Feb 15, 2024
014501f
[syft]bump version
alfred-openmined-bot Feb 15, 2024
d358628
removed hardcoded syft versions
rasswanth-s Feb 15, 2024
ad50311
Merge branch 'dev' into rasswanth/fix-notebook-base-versions
rasswanth-s Feb 15, 2024
f6664a2
Merge pull request #8496 from OpenMined/rasswanth/fix-notebook-base-v…
rasswanth-s Feb 15, 2024
ac7fabe
[syft]bump version
alfred-openmined-bot Feb 15, 2024
0c858c6
Merge pull request #8477 from OpenMined/dependabot/github_actions/pre…
rasswanth-s Feb 16, 2024
7eab201
Bump pnpm/action-setup from 2 to 3
dependabot[bot] Feb 16, 2024
e510f7f
Merge pull request #8476 from OpenMined/dependabot/github_actions/pnp…
rasswanth-s Feb 16, 2024
98436da
modified syft node uid to syft node location
rasswanth-s Feb 16, 2024
9dad273
added k8s deployment to notebooks
rasswanth-s Feb 16, 2024
7a667f8
Merge pull request #8498 from OpenMined/rasswanth/fix-node-uid
rasswanth-s Feb 16, 2024
cee737b
[syft]bump version
alfred-openmined-bot Feb 16, 2024
7568cb2
re-enabled helm tests with notebook automation
rasswanth-s Feb 16, 2024
2d2adae
Merge branch 'dev' into rasswanth/automated-helm-test
rasswanth-s Feb 16, 2024
8976cd2
generalized 11-container-images notebook
rasswanth-s Feb 16, 2024
d9bb406
fix duplicate tox passenv
rasswanth-s Feb 16, 2024
8fa6a38
shift external registry to k3d registry for testing
rasswanth-s Feb 16, 2024
5cec78c
passed external registry context to registries
rasswanth-s Feb 16, 2024
4014c66
removed password printing in external registries
rasswanth-s Feb 16, 2024
61a5f3b
moved reg username and password to request.approve()
rasswanth-s Feb 16, 2024
aa786a7
fix helm error due to coredns
rasswanth-s Feb 17, 2024
0ce7284
shifted k3d to bash execution
rasswanth-s Feb 17, 2024
463b8e0
modify k3d installation method
rasswanth-s Feb 17, 2024
e8715c5
added storage cleanup and
rasswanth-s Feb 17, 2024
c58c7a1
shifted location of storage cleanup in github actions
rasswanth-s Feb 17, 2024
d121bab
added e2e notebook tests github workflow
rasswanth-s Feb 17, 2024
4128899
Merge pull request #8499 from OpenMined/rasswanth/automated-helm-test
rasswanth-s Feb 17, 2024
f80994e
Merge pull request #8507 from OpenMined/rasswanth/e2e-notebooks-yaml
rasswanth-s Feb 17, 2024
9184f56
[syft]bump version
alfred-openmined-bot Feb 17, 2024
90f4b2a
Added stub scripts to bumping versions
rasswanth-s Feb 18, 2024
8ccb9c3
Finished script for bumping versions
rasswanth-s Feb 19, 2024
15f7d1c
[syft]bump version
alfred-openmined-bot Feb 19, 2024
0ffcf6c
separated files by projects:
rasswanth-s Feb 19, 2024
4abede7
fix stable ersion to beta bump
rasswanth-s Feb 19, 2024
0b59c73
added a wait during syft release
rasswanth-s Feb 19, 2024
0ea5ab7
Merge branch 'dev' into rasswanth/release-script
rasswanth-s Feb 19, 2024
8fffebe
Merge pull request #8508 from OpenMined/rasswanth/release-script
rasswanth-s Feb 19, 2024
24e053d
added assert statements to check worker connection status
rasswanth-s Feb 19, 2024
c8072d9
selected only hagrid tests for debugging
rasswanth-s Feb 19, 2024
cf77b05
update ci for debugging
koenvanderveen Feb 19, 2024
e0bc904
Revert "added assert statements to check worker connection status"
rasswanth-s Feb 19, 2024
70d67f0
Revert "update ci for debugging"
rasswanth-s Feb 19, 2024
9c5f018
adding sleep to wait for worker pool creation
rasswanth-s Feb 19, 2024
64b897f
added log collector
rasswanth-s Feb 19, 2024
7fbf2a0
patch worker count to be 0 if unspecified
koenvanderveen Feb 19, 2024
044340b
Merge pull request #8510 from OpenMined/patch-worker-count
rasswanth-s Feb 20, 2024
35635c9
added tmate session for debugging
rasswanth-s Feb 20, 2024
c4a4ad9
add tunshell debugging
rasswanth-s Feb 20, 2024
773793c
[syft]bump version
alfred-openmined-bot Feb 20, 2024
24dda18
Merge branch 'dev' into rasswanth/debug-notebooks
rasswanth-s Feb 20, 2024
499869d
revert notebook changes
rasswanth-s Feb 20, 2024
9d3a80c
removed tunshell debugging and uncommented out the install checks sec…
rasswanth-s Feb 20, 2024
6375a1f
Merge pull request #8509 from OpenMined/rasswanth/debug-notebooks
rasswanth-s Feb 20, 2024
1eb34a7
Merge branch 'dev' into update-readme
rasswanth-s Feb 20, 2024
df38f1c
updated readme with new notebooks
rasswanth-s Feb 20, 2024
d4322de
update version to 0.8.4
rasswanth-s Feb 20, 2024
d00b801
disabled baseimage testing temporarily until release
rasswanth-s Feb 20, 2024
f4852ba
Merge pull request #8473 from OpenMined/update-readme
rasswanth-s Feb 20, 2024
cb8fc05
Revert "disabled baseimage testing temporarily until release"
rasswanth-s Feb 20, 2024
0668f33
changed version to syft beta versions manually
rasswanth-s Feb 20, 2024
5b846b0
Revert "Revert "disabled baseimage testing temporarily until release""
rasswanth-s Feb 20, 2024
aa033dc
Added missed packages during bump version
rasswanth-s Feb 20, 2024
18a7677
Merge pull request #8512 from OpenMined/update-cd-syft
rasswanth-s Feb 20, 2024
c090678
disabled assertion checks temporarily
rasswanth-s Feb 20, 2024
6d3ce95
Merge pull request #8511 from OpenMined/update-versions
rasswanth-s Feb 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ repos:
- id: mypy
name: "mypy: syft"
always_run: true
files: "^packages/syft/src/syft/serde|^packages/syft/src/syft/util/env.py|^packages/syft/src/syft/util/logger.py|^packages/syft/src/syft/util/markdown.py|^packages/syft/src/syft/util/notebook_ui/notebook_addons.py|^packages/syft/src/syft/service/warnings.py|^packages/syft/src/syft/service/dataset|^packages/syft/src/syft/service/worker"
files: "^packages/syft/src/syft/serde|^packages/syft/src/syft/util/env.py|^packages/syft/src/syft/util/logger.py|^packages/syft/src/syft/util/markdown.py|^packages/syft/src/syft/util/notebook_ui/notebook_addons.py|^packages/syft/src/syft/service/warnings.py|^packages/syft/src/syft/service/dataset|^packages/syft/src/syft/service/worker|^packages/syft/src/syft/service/user"
#files: "^packages/syft/src/syft/serde"
args: [
"--follow-imports=skip",
Expand Down
2 changes: 1 addition & 1 deletion packages/syft/src/syft/service/user/roles.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,5 @@ class Roles(Enum):
UPLOADER = "Uploader"
EDITOR = "Editor"

def __str__(self):
def __str__(self) -> str:
return f"{self.value}"
32 changes: 16 additions & 16 deletions packages/syft/src/syft/service/user/user.py
Original file line number Diff line number Diff line change
Expand Up @@ -172,12 +172,12 @@ class UserCreateV1(UserUpdateV1):

email: EmailStr
name: str
role: Optional[ServiceRole] = None
role: Optional[ServiceRole] = None # type: ignore[assignment]
password: str
password_verify: Optional[str] = None
password_verify: Optional[str] = None # type: ignore[assignment]
verify_key: Optional[SyftVerifyKey]
institution: Optional[str]
website: Optional[str]
institution: Optional[str] # type: ignore[assignment]
website: Optional[str] # type: ignore[assignment]
created_by: Optional[SyftSigningKey]


Expand All @@ -188,12 +188,12 @@ class UserCreate(UserUpdate):

email: EmailStr
name: str
role: Optional[ServiceRole] = None # make sure role cant be set without uid
role: Optional[ServiceRole] = None # type: ignore[assignment]
password: str
password_verify: Optional[str] = None
password_verify: Optional[str] = None # type: ignore[assignment]
verify_key: Optional[SyftVerifyKey]
institution: Optional[str]
website: Optional[str]
institution: Optional[str] # type: ignore[assignment]
website: Optional[str] # type: ignore[assignment]
created_by: Optional[SyftSigningKey]
mock_execution_permission: bool = False

Expand Down Expand Up @@ -399,40 +399,40 @@ def user_to_user_verify() -> List[Callable]:


@migrate(UserV1, User)
def upgrade_user_v1_to_v2():
def upgrade_user_v1_to_v2() -> List[Callable]:
return [make_set_default(key="mock_execution_permission", value=False)]


@migrate(User, UserV1)
def downgrade_user_v2_to_v1():
def downgrade_user_v2_to_v1() -> List[Callable]:
return [drop(["mock_execution_permission"])]


@migrate(UserUpdateV1, UserUpdate)
def upgrade_user_update_v1_to_v2():
def upgrade_user_update_v1_to_v2() -> List[Callable]:
return [make_set_default(key="mock_execution_permission", value=False)]


@migrate(UserUpdate, UserUpdateV1)
def downgrade_user_update_v2_to_v1():
def downgrade_user_update_v2_to_v1() -> List[Callable]:
return [drop(["mock_execution_permission"])]


@migrate(UserCreateV1, UserCreate)
def upgrade_user_create_v1_to_v2():
def upgrade_user_create_v1_to_v2() -> List[Callable]:
return [make_set_default(key="mock_execution_permission", value=False)]


@migrate(UserCreate, UserCreateV1)
def downgrade_user_create_v2_to_v1():
def downgrade_user_create_v2_to_v1() -> List[Callable]:
return [drop(["mock_execution_permission"])]


@migrate(UserViewV1, UserView)
def upgrade_user_view_v1_to_v2():
def upgrade_user_view_v1_to_v2() -> List[Callable]:
return [make_set_default(key="mock_execution_permission", value=False)]


@migrate(UserView, UserViewV1)
def downgrade_user_view_v2_to_v1():
def downgrade_user_view_v2_to_v1() -> List[Callable]:
return [drop(["mock_execution_permission"])]
19 changes: 11 additions & 8 deletions packages/syft/src/syft/service/user/user_roles.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# stdlib
from enum import Enum
from typing import Any
from typing import Dict
from typing import List
from typing import Tuple
from typing import Union
Expand Down Expand Up @@ -42,13 +43,13 @@ def roles_descending(cls) -> List[Tuple[int, Self]]:
tuples.append((x.value, x))
return sorted(tuples, reverse=True)

@staticmethod
def roles_for_level(level: Union[int, Self]) -> List[Self]:
@classmethod
def roles_for_level(cls, level: Union[int, Self]) -> List[Self]:
if isinstance(level, ServiceRole):
level = level.value
roles = []
level_float = float(level)
service_roles = ServiceRole.roles_descending()
service_roles = cls.roles_descending()
for role in service_roles:
role_num = role[0]
if role_num == 0:
Expand All @@ -59,7 +60,7 @@ def roles_for_level(level: Union[int, Self]) -> List[Self]:
level_float = level_float % role_num
return roles

def capabilities(self):
def capabilities(self) -> List[ServiceRoleCapability]:
return ROLE_TO_CAPABILITIES[self]

def __add__(self, other: Any) -> int:
Expand Down Expand Up @@ -90,19 +91,21 @@ def __lt__(self, other: Self) -> bool:
+ ServiceRole.ADMIN
)

DATA_SCIENTIST_ROLE_LEVEL = ServiceRole.roles_for_level(
DATA_SCIENTIST_ROLE_LEVEL: List[ServiceRole] = ServiceRole.roles_for_level(
ServiceRole.DATA_SCIENTIST + ServiceRole.DATA_OWNER + ServiceRole.ADMIN
)

ONLY_DATA_SCIENTIST_ROLE_LEVEL = ServiceRole.roles_for_level(ServiceRole.DATA_SCIENTIST)
ONLY_DATA_SCIENTIST_ROLE_LEVEL: List[ServiceRole] = ServiceRole.roles_for_level(
ServiceRole.DATA_SCIENTIST
)

DATA_OWNER_ROLE_LEVEL = ServiceRole.roles_for_level(
DATA_OWNER_ROLE_LEVEL: List[ServiceRole] = ServiceRole.roles_for_level(
ServiceRole.DATA_OWNER + ServiceRole.ADMIN
)

ADMIN_ROLE_LEVEL = ServiceRole.roles_for_level(ServiceRole.ADMIN)

ROLE_TO_CAPABILITIES = {
ROLE_TO_CAPABILITIES: Dict[ServiceRole, List[ServiceRoleCapability]] = {
ServiceRole.NONE: [],
ServiceRole.GUEST: [
ServiceRoleCapability.CAN_MAKE_DATA_REQUESTS,
Expand Down
16 changes: 10 additions & 6 deletions packages/syft/src/syft/service/user/user_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -122,8 +122,10 @@ def get_all(
for i in range(0, len(results), page_size)
]
# Return the proper slice using chunk_index
results = results[page_index]
results = UserViewPage(users=results, total=total)
if page_index is not None:
results = results[page_index]
results = UserViewPage(users=results, total=total)

return results

# 🟡 TODO: No user exists will happen when result.ok() is empty list
Expand Down Expand Up @@ -180,9 +182,9 @@ def search(
]

# Return the proper slice using page_index
results = results[page_index]

results = UserViewPage(users=results, total=total)
if page_index is not None:
results = results[page_index]
results = UserViewPage(users=results, total=total)

return results

Expand Down Expand Up @@ -317,7 +319,9 @@ def update(

return user.to(UserView)

def get_target_object(self, credentials: SyftVerifyKey, uid: UID):
def get_target_object(
self, credentials: SyftVerifyKey, uid: UID
) -> Union[User, SyftError]:
user_result = self.stash.get_by_uid(credentials=credentials, uid=uid)
if user_result.is_err():
return SyftError(message=str(user_result.err()))
Expand Down
8 changes: 4 additions & 4 deletions packages/syft/src/syft/service/user/user_stash.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,10 +60,10 @@ def set(
ignore_duplicates=ignore_duplicates,
)

def admin_verify_key(self):
def admin_verify_key(self) -> Result[Optional[SyftVerifyKey], str]:
return Ok(self.partition.root_verify_key)

def admin_user(self):
def admin_user(self) -> Result[Optional[User], str]:
return self.get_by_role(
credentials=self.admin_verify_key().ok(), role=ServiceRole.ADMIN
)
Expand Down Expand Up @@ -110,7 +110,7 @@ def get_by_verify_key(
return self.query_one(credentials=credentials, qks=qks)

def delete_by_uid(
self, credentials: SyftVerifyKey, uid: UID, has_permission=False
self, credentials: SyftVerifyKey, uid: UID, has_permission: bool = False
) -> Result[SyftSuccess, str]:
qk = UIDPartitionKey.with_obj(uid)
result = super().delete(
Expand All @@ -121,7 +121,7 @@ def delete_by_uid(
return result

def update(
self, credentials: SyftVerifyKey, user: User, has_permission=False
self, credentials: SyftVerifyKey, user: User, has_permission: bool = False
) -> Result[User, str]:
res = self.check_type(user, self.object_type)
# we dont use and_then logic here as it is hard because of the order of the arguments
Expand Down
Loading