Skip to content

Commit

Permalink
Merge pull request #1442 from ScilifelabDataCentre/dev
Browse files Browse the repository at this point in the history
New Release July 5th 2023
  • Loading branch information
i-oden authored Jun 28, 2023
2 parents 6a5a345 + ac51fdb commit 774b7a5
Show file tree
Hide file tree
Showing 34 changed files with 1,476 additions and 214 deletions.
4 changes: 2 additions & 2 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@
- [ ] Merged
- [ ] No blocking PRs
- PR to `master` branch
- [ ] Yes: Read [the release instructions](../doc/procedures/new_release.md)
- [ ] I have followed steps 1-7.
- [ ] Yes: Read [the release instructions](https://github.com/ScilifelabDataCentre/dds_web/blob/master/doc/procedures/new_release.md)
- [ ] I have followed steps 1-8.
- [ ] No

## Actions / Scans
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/docker-compose-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ jobs:
- name: Run tests against database container
run: docker-compose -f docker-compose.yml -f tests/docker-compose-test.yml up --build --exit-code-from backend

- uses: codecov/codecov-action@v2
- uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./coverage/report.xml
19 changes: 19 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,25 @@
Changelog
==========

.. _2.3.1:

2.3.1 - 2023-07-05
~~~~~~~~~~~~~~~~~~~

- Dependencies:
- `requests` from `2.27.1` to `2.31.0`
- `redis` from `4.4.4` to `4.5.5`
- `Flask` from `2.0.3` to `2.2.5`
- Statistics:
- Number of TBHours stored in the last month calculated and stored in DB
- Number of TBHours stored since start calculated and stored in DB
- Endpoint `Statistics` to return rows stored in the Reporting table
- Full name of Unit Admins-, Personnel and Super Admins not shown to Researchers; Only display Unit name when...
- Listing projects
- Sending invites
- Releasing projects
- Backend Flask command `lost-files` changed to group command with subcommands `ls`, `delete`, `add-missing-bucket`

.. _2.3.0:

2.3.0 - 2023-06-07
Expand Down
16 changes: 16 additions & 0 deletions SPRINTLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,4 +254,20 @@ _Nothing merged in CLI during this sprint_
- Save number of unique Project Owners ([#1421](https://github.com/ScilifelabDataCentre/dds_web/pull/1421))
- Save amount of TB's currently stored in system ([#1424](https://github.com/ScilifelabDataCentre/dds_web/pull/1424))
- Save amount of TB's uploaded since start ([#1430](https://github.com/ScilifelabDataCentre/dds_web/pull/1430))
- Save number of TBHours stored in the last month ([#1431](https://github.com/ScilifelabDataCentre/dds_web/pull/1431))
- Save number of TBHours stored in since start ([#1434](https://github.com/ScilifelabDataCentre/dds_web/pull/1434))
- New version: 2.3.0 ([#1433](https://github.com/ScilifelabDataCentre/dds_web/pull/1433))
- Dependency: Bump `requests` to 2.31.0 due to security vulnerability alert ([#1427](https://github.com/ScilifelabDataCentre/dds_web/pull/1427))
- Endpoint: Statistics; Return all rows stored in the Reporting table ([#1435](https://github.com/ScilifelabDataCentre/dds_web/pull/1435))

# 2023-06-09 - 2023-06-23

- Dependency: Bump `Flask` to 2.2.5 due to security vulnerability alert(s) ([#1425](https://github.com/ScilifelabDataCentre/dds_web/pull/1425))
- Dependency: Bump `redis-py` to 4.5.5 due to security vulnerability alert(s) ([#1437](https://github.com/ScilifelabDataCentre/dds_web/pull/1437))
- Change from personal name to unit name if / where it's displayed in emails ([#1439](https://github.com/ScilifelabDataCentre/dds_web/pull/1439))
- Refactoring: `lost_files_s3_db` flask command changed to group with subcommands ([#1438](https://github.com/ScilifelabDataCentre/dds_web/pull/1438))

# 2023-06-26 - 2023-07-14

- Change display project info depending on the user role ([#1440](https://github.com/ScilifelabDataCentre/dds_web/pull/1440))
- New version: 2.4.0 ([#1443](https://github.com/ScilifelabDataCentre/dds_web/pull/1443))
1 change: 1 addition & 0 deletions dds_web/api/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ def output_json(data, code, headers=None):
superadmin_only.ResetTwoFactor, "/user/totp/deactivate", endpoint="reset_user_hotp"
)
api.add_resource(superadmin_only.AnyProjectsBusy, "/proj/busy/any", endpoint="projects_busy_any")
api.add_resource(superadmin_only.Statistics, "/stats", endpoint="stats")

# Invoicing ############################################################################ Invoicing #
api.add_resource(user.ShowUsage, "/usage", endpoint="usage")
17 changes: 15 additions & 2 deletions dds_web/api/project.py
Original file line number Diff line number Diff line change
Expand Up @@ -488,15 +488,23 @@ def format_project_dict(self, current_user):
# Apply the filters
user_projects = models.Project.query.filter(sqlalchemy.and_(*all_filters)).all()

researcher = False
if auth.current_user().role not in ["Super Admin", "Unit Admin", "Unit Personnel"]:
researcher = True

# Get info for all projects
for p in user_projects:
project_creator = p.creator.name if p.creator else None
if researcher:
project_creator = p.responsible_unit.external_display_name

project_info = {
"Project ID": p.public_id,
"Title": p.title,
"PI": p.pi,
"Status": p.current_status,
"Last updated": p.date_updated if p.date_updated else p.date_created,
"Created by": p.creator.name if p.creator else "Former User",
"Created by": project_creator or "Former User",
}

# Get proj size and update total size
Expand Down Expand Up @@ -967,10 +975,15 @@ def get(self):
project = dds_web.utils.collect_project(project_id=project_id)
dds_web.utils.verify_project_access(project=project)

# if current user Researcher, show unit name instead of creator name
project_creator = project.creator.name if project.creator else None
if auth.current_user().role not in ["Super Admin", "Unit Admin", "Unit Personnel"]:
project_creator = project.responsible_unit.external_display_name

# Construct a dict with info items
project_info = {
"Project ID": project.public_id,
"Created by": project.creator.name if project.creator else "Former User",
"Created by": project_creator or "Former User",
"Status": project.current_status,
"Last updated": project.date_updated,
"Size": project.size,
Expand Down
34 changes: 34 additions & 0 deletions dds_web/api/superadmin_only.py
Original file line number Diff line number Diff line change
Expand Up @@ -311,3 +311,37 @@ def get(self):
return_info.update({"projects": {p.public_id: p.date_updated for p in projects_busy}})

return return_info


class Statistics(flask_restful.Resource):
"""Get rows from Reporting table."""

@auth.login_required(role=["Super Admin"])
@logging_bind_request
@handle_db_error
def get(self):
"""Collect rows from reporting table and return them."""
stat_rows: typing.List = models.Reporting.query.all()
return {
"stats": [
{
"Date": str(row.date),
"Units": row.unit_count,
"Researchers": row.researcher_count,
"Project Owners": row.project_owner_unique_count,
"Unit Personnel": row.unit_personnel_count,
"Unit Admins": row.unit_admin_count,
"Super Admins": row.superadmin_count,
"Total Users": row.total_user_count,
"Total Projects": row.total_project_count,
"Active Projects": row.active_project_count,
"Inactive Projects": row.inactive_project_count,
"Data Now (TB)": row.tb_stored_now,
"Data Uploaded (TB)": row.tb_uploaded_since_start,
"TBHours Last Month": row.tbhours,
"TBHours Total": row.tbhours_since_start,
}
for row in stat_rows
if stat_rows
]
}
22 changes: 9 additions & 13 deletions dds_web/api/user.py
Original file line number Diff line number Diff line change
Expand Up @@ -416,26 +416,24 @@ def compose_and_send_email_to_user(userobj, mail_type, link=None, project=None):
# userobj likely an invite
recipients = [userobj.email]

unit_name = None
unit_email = None
project_id = None
deadline = None

# Don't display unit admins or personnels name
if auth.current_user().role in ["Unit Admin", "Unit Personnel"]:
unit = auth.current_user().unit
unit_name = unit.external_display_name
unit_email = unit.contact_email
sender_name = auth.current_user().name
subject_subject = unit_name

displayed_sender = unit.external_display_name
# Display name if Super admin or Project owners
else:
sender_name = auth.current_user().name
subject_subject = sender_name
displayed_sender = auth.current_user().name

# Fill in email subject with sentence subject
if mail_type == "invite":
subject = f"{subject_subject} invites you to the SciLifeLab Data Delivery System"
subject = f"{displayed_sender} invites you to the SciLifeLab Data Delivery System"
elif mail_type == "project_release":
subject = f"Project made available by {subject_subject} in the SciLifeLab Data Delivery System"
subject = f"Project made available by {displayed_sender} in the SciLifeLab Data Delivery System"
project_id = project.public_id
deadline = project.current_deadline.astimezone(datetime.timezone.utc).strftime(
"%Y-%m-%d %H:%M:%S %Z"
Expand Down Expand Up @@ -464,17 +462,15 @@ def compose_and_send_email_to_user(userobj, mail_type, link=None, project=None):
msg.body = flask.render_template(
f"mail/{mail_type}.txt",
link=link,
sender_name=sender_name,
unit_name=unit_name,
displayed_sender=displayed_sender,
unit_email=unit_email,
project_id=project_id,
deadline=deadline,
)
msg.html = flask.render_template(
f"mail/{mail_type}.html",
link=link,
sender_name=sender_name,
unit_name=unit_name,
displayed_sender=displayed_sender,
unit_email=unit_email,
project_id=project_id,
deadline=deadline,
Expand Down
Loading

0 comments on commit 774b7a5

Please sign in to comment.