Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/add batch emails #9296

Merged
merged 36 commits into from
Sep 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
a0ba583
ADD new enums for notification frequency
IonesioJunior Sep 16, 2024
43e9a2e
Update notifier to support batch notifications
IonesioJunior Sep 16, 2024
4808ef1
Add new email_notification_dispatcher thread in server.py
IonesioJunior Sep 16, 2024
11fc225
Update protocol version
IonesioJunior Sep 16, 2024
e72d262
ADD new logic for dispatch notification
IonesioJunior Sep 16, 2024
3336134
Merge branch 'dev' into feat/add_batch_emails
IonesioJunior Sep 17, 2024
1526ac9
Remove non-default value for batched notifications
IonesioJunior Sep 17, 2024
2404437
Undo modified protocol_version.json
IonesioJunior Sep 17, 2024
ff444ed
ADD new changed protocol_version.json
IonesioJunior Sep 17, 2024
a27607b
Add email queue iteration
IonesioJunior Sep 18, 2024
8fc9d91
Update email templates
IonesioJunior Sep 18, 2024
729f27d
Update EmailNotifier to apply email.send_batches
IonesioJunior Sep 18, 2024
3e4d868
Merge branch 'dev' into feat/add_batch_emails
IonesioJunior Sep 18, 2024
c600ad3
Merge branch 'dev' into feat/add_batch_emails
IonesioJunior Sep 19, 2024
1f911af
Merge branch 'dev' into feat/add_batch_emails
IonesioJunior Sep 19, 2024
ad001a2
Fix Request Batch email template
IonesioJunior Sep 19, 2024
4b0f7d2
Merge branch 'dev' into feat/add_batch_emails
IonesioJunior Sep 20, 2024
cf6d926
Update protocol_version with new Object Versions/hashed
IonesioJunior Sep 20, 2024
d861ece
Add new settings.batch_notifications endpoint
IonesioJunior Sep 20, 2024
a7ffbdc
Add internal set_batch_notifications and aux is_time_to_dispatch method
IonesioJunior Sep 20, 2024
abd5d5d
Create new EmailFrequency Object / Adjust NotifierSettings attributes
IonesioJunior Sep 20, 2024
198731f
Add logic to check if it's time to dispatch a new batch email
IonesioJunior Sep 20, 2024
fd2446b
ADD migration functions
IonesioJunior Sep 20, 2024
e686037
Remove unused print statements
IonesioJunior Sep 20, 2024
96a5de6
Merge branch 'dev' into feat/add_batch_emails
IonesioJunior Sep 23, 2024
2f9b2a9
Remove unecessary print statement
IonesioJunior Sep 23, 2024
f88c6e6
Fix small bug when passing NOTIFICATION_FREQUENCY.INSTANT without sta…
IonesioJunior Sep 23, 2024
f112c1e
ADD new batch notification notebook test in /scenarios/bq
IonesioJunior Sep 23, 2024
db88d8c
Fix lint
IonesioJunior Sep 23, 2024
14352e0
Rename notebook test
IonesioJunior Sep 23, 2024
9146f7a
Merge branch 'dev' into feat/add_batch_emails
IonesioJunior Sep 24, 2024
2cd0cad
ADD integration test for batching notifications
IonesioJunior Sep 24, 2024
7f1ed88
Use UTC timezone as server timezone to send notifications
IonesioJunior Sep 24, 2024
cb0fef7
Remove test duplicate
IonesioJunior Sep 24, 2024
fe7f20d
Add pytest-asyncio dep in integration.k8s too
IonesioJunior Sep 24, 2024
6c93864
Merge branch 'dev' into feat/add_batch_emails
IonesioJunior Sep 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions packages/syft/src/syft/protocol/protocol_version.json
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,20 @@
"hash": "2e1365c5535fa51c22eef79f67dd6444789bc829c27881367e3050e06e2ffbfe",
"action": "remove"
}
},
"NotifierSettings": {
"3": {
"version": 3,
"hash": "226c3e0d4de4368ea9eac6689427cfc27860cf51696741b8dda14f939f3d4fbe",
"action": "add"
}
},
"EmailFrequency": {
"1": {
"version": 1,
"hash": "7659117222a461a959eac7aa1aaf280033c2ca4f1029f97e76051e0474e56759",
"action": "add"
}
}
}
}
Expand Down
38 changes: 38 additions & 0 deletions packages/syft/src/syft/server/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
from collections.abc import Callable
from datetime import MINYEAR
from datetime import datetime
from datetime import timezone
from functools import partial
import hashlib
import json
Expand All @@ -14,6 +15,7 @@
from pathlib import Path
import subprocess # nosec
import sys
import threading
from time import sleep
import traceback
from typing import Any
Expand Down Expand Up @@ -455,6 +457,42 @@ def __init__(
self.run_peer_health_checks(context=context)

ServerRegistry.set_server_for(self.id, self)
email_dispatcher = threading.Thread(target=self.email_notification_dispatcher)
email_dispatcher.daemon = True
email_dispatcher.start()

def email_notification_dispatcher(self) -> None:
lock = threading.Lock()
while True:
# Use admin context to have access to the notifier obj
context = AuthedServiceContext(
server=self,
credentials=self.verify_key,
role=ServiceRole.ADMIN,
)
# Get notitifer settings
notifier_settings = self.services.notifier.settings(
context=context
).unwrap()
lock.acquire()
# Iterate over email_types and its queues
# Ex: {'EmailRequest': {VerifyKey: [], VerifyKey: [], ...}}
for email_template, email_queue in notifier_settings.email_queue.items():
# Get the email frequency of that specific email type
email_frequency = notifier_settings.email_frequency[email_template]
for verify_key, queue in email_queue.items():
if self.services.notifier.is_time_to_dispatch(
email_frequency, datetime.now(timezone.utc)
):
notifier_settings.send_batched_notification(
context=context, notification_queue=queue
).unwrap()
notifier_settings.email_queue[email_template][verify_key] = []
self.services.notifier.stash.update(
credentials=self.verify_key, obj=notifier_settings
).unwrap()
lock.release()
sleep(15)

def set_log_level(self, log_level: int | str | None) -> None:
def determine_log_level(
Expand Down
182 changes: 180 additions & 2 deletions packages/syft/src/syft/service/notification/email_templates.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,31 @@
class EmailTemplate:
@staticmethod
def email_title(notification: "Notification", context: AuthedServiceContext) -> str:
return ""
raise NotImplementedError(
"Email Template subclasses must implement the email_title method."
)

@staticmethod
def email_body(notification: "Notification", context: AuthedServiceContext) -> str:
return ""
raise NotImplementedError(
"Email Template subclasses must implement the email_body method."
)

@staticmethod
def batched_email_title(
notifications: list["Notification"], context: AuthedServiceContext
) -> str:
raise NotImplementedError(
"Email Template subclasses must implement the batched_email_title method."
)

@staticmethod
def batched_email_body(
notifications: list["Notification"], context: AuthedServiceContext
) -> str:
raise NotImplementedError(
"Email Template subclasses must implement the batched_email_body method."
)


@serializable(canonical_name="FailedJobTemplate", version=1)
Expand Down Expand Up @@ -475,6 +495,164 @@ def email_body(notification: "Notification", context: AuthedServiceContext) -> s
"""
return f"""<html>{head} {body}</html>"""

@staticmethod
def batched_email_title(
notifications: list["Notification"], context: AuthedServiceContext
) -> str:
return "Batched Requests Notifications"

@staticmethod
def batched_email_body(
notifications: list["Notification"], context: AuthedServiceContext
) -> str:
notifications_info = ""
for i, notification in enumerate(notifications):
if i > 3:
break
notification.linked_obj = cast(LinkedObject, notification.linked_obj)
request_obj = notification.linked_obj.resolve_with_context(
context=context
).unwrap()

request_id = request_obj.id
request_name = request_obj.requesting_user_name
request_email = request_obj.requesting_user_email
request_time = request_obj.request_time
request_status = request_obj.status.name # fails in l0 check right now
request_changes = ",".join(
[change.__class__.__name__ for change in request_obj.changes]
)

notifications_info += f"""<tr>
<td>{str(request_id)[:4] + "..."}</td>
<td>{request_name}</td>
<td>{request_email}</td>
<td>{request_time}</td>
<td>{request_status}</td>
<td>{request_changes}</td>
</tr>"""

see_more_info = ""
if len(notifications) > 4:
see_more_info = f"""<p class="more-requests">{len(notifications) - 4}
more requests made during this time period.
Connect to the server to check all requests.</p>"""
head = """
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Batched Requests Notification</title>
<style>
body {
font-family: Arial, sans-serif;
background-color: #f4f4f4;
margin: 0;
padding: 0;
color: #333;
}
.container {
width: 100%;
max-width: 600px;
margin: 0 auto;
background-color: #ffffff;
padding: 20px;
border-radius: 8px;
box-sizing: border-box; /* Added to include padding in width */
box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
}
.header {
font-size: 18px;
font-weight: bold;
margin-bottom: 20px;
color: #4CAF50;
}
.content {
font-size: 14px;
line-height: 1.6;
color: #555555;
}
.content p {
margin: 10px 0;
}
.request-table {
width: 100%;
border-collapse: collapse;
margin-top: 20px;
table-layout: fixed; /* Added to fix table layout */
}
.request-table th, .request-table td {
text-align: left;
padding: 12px;
border: 1px solid #ddd;
word-wrap: break-word; /* Added to wrap long content */
}
.request-table th {
background-color: #f8f8f8;
color: #333;
font-weight: bold;
}
.request-table tr:nth-child(even) {
background-color: #f9f9f9;
}
.more-requests {
font-size: 13px;
color: #FF5722;
margin-top: 10px;
}
.footer {
margin-top: 20px;
font-size: 12px;
color: #777777;
}
.button {
background-color: #4CAF50;
color: #ffffff;
padding: 10px 20px;
text-decoration: none;
border-radius: 5px;
font-size: 14px;
display: inline-block;
margin-top: 20px;
}
</style>
</head>
"""
body = f"""
<body>
<div class="container">
<div class="header">
Batched Requests Notification
</div>
<div class="content">
<p>Hello Admin,</p>
<p>This is to inform you that a batch of requests has been processed.
Below are the details of the most recent requests:</p>
<table class="request-table">
<thead>
<tr>
<th>Request ID</th>
<th>User</th>
<th>User Email</th>
<th>Date</th>
<th>Status</th>
<th>Changes</th>
</tr>
</thead>
<tbody>
{notifications_info}
<!-- Only show the first 3 requests -->
</tbody>
</table>
{see_more_info}
</div>
<div class="footer">
<p>Thank you,</p>
</div>
</div>
</body>
"""
return f"""<html>{head} {body}</html>"""


@serializable(canonical_name="RequestUpdateEmailTemplate", version=1)
class RequestUpdateEmailTemplate(EmailTemplate):
Expand Down
Loading
Loading