Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Email Notifications #8455

Merged
merged 115 commits into from
Mar 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
115 commits
Select commit Hold shift + click to select a range
da65690
Create Notifier Service skeleton
jcardonnet Feb 5, 2024
5cd6fd9
Merge branch 'dev' into notifier
IonesioJunior Feb 6, 2024
5960001
Merge branch 'dev' into notifier
IonesioJunior Feb 13, 2024
2bace8b
Enable notifier Service
IonesioJunior Feb 14, 2024
c37262e
Implement static init_notifier method and add it during node start wo…
IonesioJunior Feb 14, 2024
2d23fc5
Remove auto creation from notifier stash .get()
IonesioJunior Feb 14, 2024
436ad84
Lint
IonesioJunior Feb 14, 2024
890e362
ADD notifier service admin view
IonesioJunior Feb 14, 2024
97a7272
Create notifier_service.turn_on method
IonesioJunior Feb 14, 2024
1e4dcb3
Update in Notifier base classes
IonesioJunior Feb 14, 2024
ca22509
Update/Fix NotifierService.turn_on
IonesioJunior Feb 14, 2024
0facf9c
ADD checkings for username/password or token during SMTPClient initia…
IonesioJunior Feb 14, 2024
fab6362
Turn off notifier service
IonesioJunior Feb 14, 2024
d267a47
ADD check for update results in turn_on service
IonesioJunior Feb 14, 2024
69ef589
ADD checks for notifier.turn_off update result
IonesioJunior Feb 14, 2024
8046670
Merge pull request #8490 from OpenMined/feature/turn_off_notifier
IonesioJunior Feb 15, 2024
4019396
Merge branch 'enable_notifier_service' into feature/notifier_settings
IonesioJunior Feb 15, 2024
54dba1d
Merge branch 'enable_notifier_service' into add_init_notifier
IonesioJunior Feb 15, 2024
4e8f7ce
Merge branch 'enable_notifier_service' into feature/notifier_turn_on
IonesioJunior Feb 15, 2024
ec9c18b
Merge pull request #8489 from OpenMined/feature/notifier_turn_on
jcardonnet Feb 15, 2024
4e1a9ef
Merge pull request #8487 from OpenMined/add_init_notifier
jcardonnet Feb 15, 2024
f4de4a0
Merge pull request #8488 from OpenMined/feature/notifier_settings
jcardonnet Feb 15, 2024
287bab3
Merge pull request #8485 from OpenMined/enable_notifier_service
jcardonnet Feb 15, 2024
5018a3b
Add EMAIL_TOKEN environment variable in hagrid pipeline
IonesioJunior Feb 15, 2024
2543cd1
Merge branch 'dev' into notifier
IonesioJunior Feb 15, 2024
4fdf94a
Merge branch 'notifier' into feature/hagrid_email_token_flag
IonesioJunior Feb 15, 2024
e326b0c
ADD EMAIL_TOKEN env var in backend statefulset template
IonesioJunior Feb 16, 2024
3641d37
Merge branch 'dev' into notifier
IonesioJunior Feb 16, 2024
3acc9a5
Write the dispatch_notification internal function
IonesioJunior Feb 16, 2024
61498c4
Fix the SMTPClient to authenticate and work properly with access_tokens
IonesioJunior Feb 16, 2024
36e8386
Fix EmailNotifier / Remove unused BaseEmailNotifier
IonesioJunior Feb 16, 2024
f692255
Create a notifier type enum
IonesioJunior Feb 16, 2024
a4f93d2
ADD notifier_types attribute in Notification Object and update its ve…
IonesioJunior Feb 16, 2024
2ad6249
ADD the notifier dispatch whenever a new notification is sent by the …
IonesioJunior Feb 16, 2024
994c253
ADD EMAIL.NOTIFIER as a option to notification for request.submit ser…
IonesioJunior Feb 16, 2024
56c89cb
Run linting
IonesioJunior Feb 16, 2024
f74f3e3
Merge branch 'notifier' into feature/hagrid_email_token_flag
IonesioJunior Feb 16, 2024
733274e
FIX CreateNotification migrate paramters
IonesioJunior Feb 16, 2024
904aafa
ADD return in case dispatch_notifications aren't done successfully
IonesioJunior Feb 16, 2024
a59dfa6
Lint
IonesioJunior Feb 16, 2024
881d68e
Merge pull request #8497 from OpenMined/feature/hagrid_email_token_flag
IonesioJunior Feb 16, 2024
8007878
ADD settings enable_notification/disable_notifications api
IonesioJunior Feb 16, 2024
308726e
Merge pull request #8504 from OpenMined/feature/integration_with_post…
jcardonnet Feb 19, 2024
825f685
Merge pull request #8506 from OpenMined/feature/enable_notifications_…
jcardonnet Feb 19, 2024
4a90632
Merge branch 'dev' into notifier
IonesioJunior Feb 21, 2024
af47fcd
ADD checking for invalid smtp credentials
IonesioJunior Feb 21, 2024
7a88d8f
Put activate/deactivate notifier aliases in notification_service
IonesioJunior Feb 21, 2024
3eb2dbb
Fix typo
IonesioJunior Feb 21, 2024
e92609d
Merge pull request #8520 from OpenMined/hotfix/check_invalid_email_token
jcardonnet Feb 21, 2024
e26b63f
Replace email_token -> smtp_username, smtp_password
IonesioJunior Feb 21, 2024
da96c79
Merge branch 'notifier' into feature/add_notifier_aliases
IonesioJunior Feb 21, 2024
62287f6
Replace email token with username/password.
jcardonnet Feb 22, 2024
7cdfd33
Merge branch 'notifier' into hotfix/rename_email_credentials
jcardonnet Feb 22, 2024
5e5243f
Fix merge conflict
jcardonnet Feb 22, 2024
68253d4
Small fixes
IonesioJunior Feb 22, 2024
daf48b0
Run Lint
IonesioJunior Feb 22, 2024
86f5692
Fix lint issue
IonesioJunior Feb 22, 2024
c247468
Merge pull request #8522 from OpenMined/hotfix/rename_email_credentials
IonesioJunior Feb 22, 2024
4b85a4a
Merge branch 'notifier' into feature/add_notifier_aliases
IonesioJunior Feb 22, 2024
9ac01b5
- ADD support to email templates
IonesioJunior Feb 22, 2024
bcc1a4f
Replace Suspicious Activity -> OnBoard email
IonesioJunior Feb 22, 2024
a5eefb1
Fix lint
IonesioJunior Feb 22, 2024
17dbb15
Add email notification setting to User
jcardonnet Feb 23, 2024
2afdac0
Fix bug in migrator function
jcardonnet Feb 23, 2024
99fcfa7
ADD serde/storage rules in old user version
IonesioJunior Feb 23, 2024
f348928
Refactor user enabled_notifications parameter to provide support for …
IonesioJunior Feb 23, 2024
dd57951
Fix linting
IonesioJunior Feb 23, 2024
32a8d93
Merge pull request #8524 from OpenMined/feature/add_email_templates
IonesioJunior Feb 23, 2024
15d62ff
ADD sender_email parameter and add helm configs
IonesioJunior Feb 24, 2024
6f0d664
Partial implementation of enable/disable User Notifications
jcardonnet Feb 26, 2024
2389770
Merge branch 'notifier' into feature/user_notifications
jcardonnet Feb 26, 2024
c723714
Merge pull request #8527 from OpenMined/feature/add_sender_parameter
jcardonnet Feb 26, 2024
11c470e
Merge branch 'notifier' into feature/user_notifications
jcardonnet Feb 26, 2024
286a328
Merge branch 'feature/user_notifications' of https://github.com/OpenM…
jcardonnet Feb 26, 2024
8997a75
Implement enable/disable User notifications.
jcardonnet Feb 27, 2024
8fd5724
Remove TODO comment to avoid mypy lint issues
IonesioJunior Feb 27, 2024
033a0af
Merge pull request #8525 from OpenMined/feature/user_notifications
IonesioJunior Feb 27, 2024
760c5be
Merge branch 'notifier' into feature/add_notifier_aliases
IonesioJunior Feb 27, 2024
a7c7622
Merge pull request #8521 from OpenMined/feature/add_notifier_aliases
jcardonnet Feb 27, 2024
da02a4f
Merge branch 'dev' into notifier
IonesioJunior Feb 28, 2024
5d75431
Fix enable/disable user notifications
IonesioJunior Feb 29, 2024
f6b6621
Remove public notifier public API services
IonesioJunior Feb 29, 2024
1dc56e2
Fix lint issues
IonesioJunior Feb 29, 2024
3b66f68
ADD default EMAIL notifier in activate and deactivate notifiers
IonesioJunior Feb 29, 2024
51e8e78
ADD an alias to notifier.settings
IonesioJunior Feb 29, 2024
da77ad8
ADD a comment during notifier setup and initialization
IonesioJunior Feb 29, 2024
21aa492
Merge branch 'dev' into notifier
IonesioJunior Feb 29, 2024
3d519d6
Remove "Notifications are in beta" message when calling Notifications…
jcardonnet Mar 1, 2024
3ca7b5a
Merge branch 'dev' into notifier
IonesioJunior Mar 1, 2024
a2d3fbd
Merge pull request #8540 from OpenMined/remove-beta-warning
IonesioJunior Mar 1, 2024
a751e06
Merge branch 'dev' into notifier
IonesioJunior Mar 2, 2024
825e7bb
ADD Request ID to avoid stacking email messages
IonesioJunior Mar 2, 2024
dd60830
ADD email notification when a request is approved/denied
IonesioJunior Mar 2, 2024
1f63970
Run lint
IonesioJunior Mar 2, 2024
a591929
Undo api fix
IonesioJunior Mar 2, 2024
9f6be95
ADD key parameter at check_type in validate_callable_args_and_kwargs
IonesioJunior Mar 2, 2024
48bf9ba
Undo changes in api
IonesioJunior Mar 2, 2024
18e17cf
Create a new notification service: user_settings
IonesioJunior Mar 4, 2024
6ee8319
Merge branch 'dev' into notifier
IonesioJunior Mar 4, 2024
d8baca4
ADD SMTP_HOST and SMTP_PORT parameters for notifier configuration
IonesioJunior Mar 4, 2024
bdeb843
Attempt to fix check_type issue
IonesioJunior Mar 4, 2024
9d08de7
Merge branch 'dev' into notifier
IonesioJunior Mar 5, 2024
82d004b
Fix remaining mypy issues
IonesioJunior Mar 5, 2024
9070f63
Merge branch 'dev' into notifier
IonesioJunior Mar 5, 2024
961e666
Undo check_type changes
IonesioJunior Mar 5, 2024
90b2046
Fix value types in backend-statefulset template
IonesioJunior Mar 5, 2024
ba7f86d
Add int cast type in email_port
IonesioJunior Mar 5, 2024
ebc086c
Merge branch 'dev' into notifier
kiendang Mar 6, 2024
064e276
Rollback UserCreateV2/UserUpdateV2
IonesioJunior Mar 6, 2024
878f769
Fix pydantic update issues
IonesioJunior Mar 6, 2024
5728974
Merge branch 'dev' into notifier
shubham3121 Mar 6, 2024
3f60f21
Replace get_by_email to email_exists
IonesioJunior Mar 6, 2024
e728e27
Remove old Notification / UserCreate implementations
IonesioJunior Mar 6, 2024
1d568d5
Undo register changes
IonesioJunior Mar 6, 2024
888e18a
Fix UserCreate object creation by adding required fields
IonesioJunior Mar 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions packages/grid/backend/grid/core/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,11 +72,6 @@ def sentry_dsn_can_be_blank(cls, v: str) -> Optional[str]:
return None
return v

SMTP_TLS: bool = True
SMTP_PORT: Optional[int] = None
SMTP_HOST: Optional[str] = None
SMTP_USER: Optional[str] = None
SMTP_PASSWORD: Optional[str] = None
EMAILS_FROM_EMAIL: Optional[EmailStr] = None
EMAILS_FROM_NAME: Optional[str] = None

Expand Down Expand Up @@ -149,6 +144,12 @@ def get_emails_enabled(self) -> Self:
SINGLE_CONTAINER_MODE: bool = str_to_bool(os.getenv("SINGLE_CONTAINER_MODE", False))
CONSUMER_SERVICE_NAME: Optional[str] = os.getenv("CONSUMER_SERVICE_NAME")
INMEMORY_WORKERS: bool = str_to_bool(os.getenv("INMEMORY_WORKERS", True))
SMTP_USERNAME: str = os.getenv("SMTP_USERNAME", "")
EMAIL_SENDER: str = os.getenv("EMAIL_SENDER", "")
SMTP_PASSWORD: str = os.getenv("SMTP_PASSWORD", "")
SMTP_TLS: bool = True
SMTP_PORT: Optional[str] = os.getenv("SMTP_PORT", "")
SMTP_HOST: Optional[str] = os.getenv("SMTP_HOST", "")

TEST_MODE: bool = (
True if os.getenv("TEST_MODE", "false").lower() == "true" else False
Expand Down
5 changes: 5 additions & 0 deletions packages/grid/backend/grid/core/node.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,4 +94,9 @@ def seaweedfs_config() -> SeaweedFSConfig:
queue_config=queue_config,
migrate=True,
in_memory_workers=settings.INMEMORY_WORKERS,
smtp_username=settings.SMTP_USERNAME,
smtp_password=settings.SMTP_PASSWORD,
email_sender=settings.EMAIL_SENDER,
smtp_port=settings.SMTP_PORT,
smtp_host=settings.SMTP_HOST,
)
4 changes: 2 additions & 2 deletions packages/grid/default.env
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,9 @@ DEFAULT_ROOT_PASSWORD=changethis
SMTP_TLS=True
SMTP_PORT=587
SMTP_HOST=
SMTP_USER=
SMTP_USERNAME=
SMTP_PASSWORD=
[email protected]
EMAIL_SENDER=
SERVER_HOST="https://${DOMAIN}"
NETWORK_CHECK_INTERVAL=60
DOMAIN_CHECK_INTERVAL=60
Expand Down
10 changes: 10 additions & 0 deletions packages/grid/helm/syft/templates/backend-statefulset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,16 @@ spec:
value: "true"
- name: N_CONSUMERS
value: "0"
- name: SMTP_USERNAME
value: {{ .Values.node.settings.smtpUserName }}
- name: SMTP_PASSWORD
value: {{ .Values.node.settings.smtpPassword }}
- name: EMAIL_SENDER
value: {{ .Values.node.settings.smtpSender }}
- name: SMTP_PORT
value: "{{ .Values.node.settings.smtpPort }}"
- name: SMTP_HOST
value: {{ .Values.node.settings.smtpHost }}
- name: INMEMORY_WORKERS
value: "{{ .Values.node.settings.inMemoryWorkers }}"
- name: LOG_LEVEL
Expand Down
5 changes: 5 additions & 0 deletions packages/grid/helm/syft/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,11 @@ node:
versionHash: "abc"
nodeSideType: "high"
defaultRootEmail: "[email protected]"
smtpUserName: "apikey"
smtpPassword: "password"
smtpSender: "[email protected]"
smtpPort: 587
smtpHost: "smtp.sendgrid.net"
logLevel: "info"
inMemoryWorkers: false
defaultWorkerPoolCount: 1
Expand Down
51 changes: 51 additions & 0 deletions packages/hagrid/hagrid/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,41 @@ def clean(location: str) -> None:
type=str,
help="Container image tag to use",
)
@click.option(
"--smtp-username",
default=None,
required=False,
type=str,
help="Username used to auth in email server and enable notification via emails",
)
@click.option(
"--smtp-password",
default=None,
required=False,
type=str,
help="Password used to auth in email server and enable notification via emails",
)
@click.option(
"--smtp-port",
default=None,
required=False,
type=str,
help="Port used by email server to send notification via emails",
)
@click.option(
"--smtp-host",
default=None,
required=False,
type=str,
help="Address used by email server to send notification via emails",
)
@click.option(
"--smtp-sender",
default=None,
required=False,
type=str,
help="Sender email used to deliver PyGrid email notifications.",
)
@click.option(
"--build-src",
default=DEFAULT_BRANCH,
Expand Down Expand Up @@ -1309,6 +1344,12 @@ def create_launch_cmd(
else:
parsed_kwargs["node_side_type"] = NodeSideType.HIGH_SIDE.value

parsed_kwargs["smtp_username"] = kwargs["smtp_username"]
parsed_kwargs["smtp_password"] = kwargs["smtp_password"]
parsed_kwargs["smtp_port"] = kwargs["smtp_port"]
parsed_kwargs["smtp_host"] = kwargs["smtp_host"]
parsed_kwargs["smtp_sender"] = kwargs["smtp_sender"]

parsed_kwargs["enable_warnings"] = not kwargs["no_warnings"]

# choosing deployment type
Expand Down Expand Up @@ -2156,6 +2197,11 @@ def create_launch_docker_cmd(

single_container_mode = kwargs["deployment_type"] == "single_container"
in_mem_workers = kwargs.get("in_mem_workers")
smtp_username = kwargs.get("smtp_username")
smtp_sender = kwargs.get("smtp_sender")
smtp_password = kwargs.get("smtp_password")
smtp_port = kwargs.get("smtp_port")
smtp_host = kwargs.get("smtp_host")

enable_oblv = bool(kwargs["oblv"])
print(" - NAME: " + str(snake_name))
Expand Down Expand Up @@ -2220,6 +2266,11 @@ def create_launch_docker_cmd(
"NODE_SIDE_TYPE": kwargs["node_side_type"],
"SINGLE_CONTAINER_MODE": single_container_mode,
"INMEMORY_WORKERS": in_mem_workers,
"SMTP_USERNAME": smtp_username,
"SMTP_PASSWORD": smtp_password,
"EMAIL_SENDER": smtp_sender,
"SMTP_PORT": smtp_port,
"SMTP_HOST": smtp_host,
}

if "trace" in kwargs and kwargs["trace"] is True:
Expand Down
10 changes: 6 additions & 4 deletions packages/syft/src/syft/client/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -695,14 +695,16 @@ def settings(self) -> Optional[APIModule]:

@property
def notifications(self) -> Optional[APIModule]:
print(
"WARNING: Notifications is currently is in a beta state, so use carefully!"
)
print("If possible try using client.requests/client.projects")
if self.api.has_service("notifications"):
return self.api.services.notifications
return None

@property
def notifier(self) -> Optional[APIModule]:
if self.api.has_service("notifier"):
return self.api.services.notifier
return None

@property
def peers(self) -> Optional[Union[List[NodePeer], SyftError]]:
if self.api.has_service("network"):
Expand Down
18 changes: 18 additions & 0 deletions packages/syft/src/syft/node/node.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@
from ..service.metadata.node_metadata import NodeMetadataV3
from ..service.network.network_service import NetworkService
from ..service.notification.notification_service import NotificationService
from ..service.notifier.notifier_service import NotifierService
from ..service.object_search.migration_state_service import MigrateStateService
from ..service.output.output_service import OutputService
from ..service.policy.policy_service import PolicyService
Expand Down Expand Up @@ -316,6 +317,11 @@ def __init__(
dev_mode: bool = False,
migrate: bool = False,
in_memory_workers: bool = True,
smtp_username: Optional[str] = None,
smtp_password: Optional[str] = None,
email_sender: Optional[str] = None,
smtp_port: Optional[str] = None,
smtp_host: Optional[str] = None,
):
# 🟡 TODO 22: change our ENV variable format and default init args to make this
# less horrible or add some convenience functions
Expand Down Expand Up @@ -357,6 +363,7 @@ def __init__(
DataSubjectService,
NetworkService,
PolicyService,
NotifierService,
NotificationService,
DataSubjectMemberService,
ProjectService,
Expand Down Expand Up @@ -404,7 +411,17 @@ def __init__(
node=self,
)

NotifierService.init_notifier(
node=self,
email_password=smtp_password,
email_username=smtp_username,
email_sender=email_sender,
smtp_port=smtp_port,
smtp_host=smtp_host,
)

self.client_cache: dict = {}

if isinstance(node_type, str):
node_type = NodeType(node_type)
self.node_type = node_type
Expand Down Expand Up @@ -948,6 +965,7 @@ def _construct_services(self) -> None:
DataSubjectService,
NetworkService,
PolicyService,
NotifierService,
NotificationService,
DataSubjectMemberService,
ProjectService,
Expand Down
Loading
Loading