Fast-forward master branch to where develop branch currently is #461
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The real allowed data fields and services data does not live in this repository. All the data that this management command creates should be treated as development data. Thus renamed the command to seed_development_data and removed the --development option. The command does not get executed in container startup at all unless the SEED_DEVELOPMENT_DATA environment variable is set.
Having both --clear and --no-clear options is a bit useless. Clearing is now the default and it can be prevented with the --no-clear option. The --no-clear option is required in tests. Combined two very similar tests to parameterized tests as the only difference was whether they added a superuser or not.
Change the `generate_data_fields` function to take the specification for the allowed data fields as an argument. Use the `DATA_FIELD_VALUES` hard coded specification only in the development data seeding management command.
This utility manages models from the services app, so it should reside in the same app.
The implementation already did this, there just wasn't a test for it. Refactored the tests so that there's a helper function that does the actual testing. The test functions only have to provide the updated specification for the allowed data fields.
Previously translations were only added to a newly created AllowedDataField. Now translations are also set for a pre-existing field, effectively updating and adding new translations.
If an allowed data field no longer has a translations for some languages according to the specification, remove them from the database.
It may happen that a field gets removed from the specification, but another field replaces it. The replacing field can specify aliases, old names for fields that it replaces. Any Services that refer to the old field, will now refer to the replacing field instead.
Now the conformance to specification check can be performed at any point in the tests.
When an allowed data field that is in use, gets removed, a replacing field should be found with an alias. If no such field is found, terminate the allowed data field modification operation with an exception and rollback all changes.
The command reads the specification in JSON format from stdin and then passes the data to the allowed data fields configurator function. If nothing gets inserted into the stdin of the command, it would wait forever. Pressing Ctrl+C interrupts the command, raising a KeyboardInterrupt exception. That exception is "swallowed" and the command just exits normally in that case.
Fixes CVE-2023-32681 (which probably didn't affect this project). In version 2.26.0 requests had changed its chardet dependency to charset-normalizer. https://www.cve.org/CVERecord?id=CVE-2023-32681
open-city-profile needs to know which identity providers a service supports to know which IDP it should fetch the GDPR API tokens from. Additionally add gdpr_audience to Service because the GDPR audience is not part of the GDPR scopes in the case of Keycloak API tokens.
Also add two new settings KEYCLOAK_GDPR_CLIENT_ID, and KEYCLOAK_GDPR_CLIENT_SECRET that will be used when requesting GDPR API tokens from Keycloak.
Tunnistamo API tokens are used if a service supports both Tunnistamo, and Keycloak tokens. That is because the Tunnistamo API tokens can be fetched in one request. For services that support only Keycloak API tokens the Keycloak API tokens are requested using KeycloakTokenExchange. An API token is requested for every service separately as Keycloak doesn't support requesting multiple tokens in one request. Also add new "authorization_code_keycloak" input to download_my_profile query, DeleteMyProfileMutation, and DeleteMyServiceDataMutationInput. The authorization code is needed if there are services that accept only API tokens from Keycloak.
The tests make sure that the there are correct calls to access_token, and API token endpoints to both Tunnistamo and Keycloak in cases where there are various kinds of service connection compositions. Also, GDPR API calls are verified to use token from a suitable IDP.
Base image uses Debian bookworm which doesn't have the netcat package.
uwsgi had at least one vulnerability: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-27522
The new version removes a possibly compromised root certificate
Cryptography had 2 vulnerabilities - GHSA-jm77-qphf-c4w8 - https://www.cve.org/CVERecord?id=CVE-2023-38325
There seems to be a problem with installing PyYAML < 6.0.1 after cython released the version 3.0.0 in 2023-07-17. See issue in PyYAML repo: yaml/pyyaml#601 django-sanitized-dump needed to be updated too because the older version didn't work with PyYAML 6.0+
There were vulnerabilities in ipython and its dependencies, but I don't think we need ipython at all.
These tools are not used in the project per-se, but could be used in the developers editor or workflow. Best to leave the choice to the developer and not to the project.
linked OpenSSL version is vulnerable to several security issues. https://www.openssl.org/news/secadv/20230908.txt
|
Kudos, SonarCloud Quality Gate passed!
|
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## master #461 +/- ##
==========================================
- Coverage 91.49% 91.25% -0.25%
==========================================
Files 54 55 +1
Lines 2682 2800 +118
Branches 312 329 +17
==========================================
+ Hits 2454 2555 +101
- Misses 175 189 +14
- Partials 53 56 +3 ☔ View full report in Codecov by Sentry. |
AntiAki-M
approved these changes
Nov 23, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Preparation for a new release.