2022.2.1

[strogonoff]

New features in this release mostly concern themselves with metrics, tracking and integrations:

• Matomo tracker or tag manager are now supported via MATOMO_URL, and MATOMO_CONTAINER and/or MATOMO_SITE_ID environment variables. (Needs more integration testing.)
• Metrics for bibliographic data accesses via GUI and API are now exported in Prometheus format under /metrics path.
• The service now allows site visitors to log in via Datatracker, showing the requisite widget if DATATRACKER_CLIENT_ID and DATATRACKER_CLIENT_SECRET environment variables are provided at deploy time. (Implement support for authenticating Datatracker API keys #69)
  • OAuth2 flow is implemented, but needs further integration testing and clarification of the user info endpoint schema.
  • Callback URL is expected to be https://<hostname>/datatracker-auth/callback/
• Multiple additional API secrets can now be provisioned for management API and management GUI access via EXTRA_API_SECRETS environment variable. (Implement support for multiple API keys/tokens for use in API/GUI authentication #68)

Added containers

There are new services in the bundled Compose file configuration.

• A container that exports Celery-level metrics in Prometheus format by monitoring Redis backend.
• A pre-configured Prometheus instance monitoring three targets (web server, and two targets for async task runner).
• A Grafana instance provisioned with Prometheus as data source and a couple of dashboards for monitoring API and GUI bibliographic data access metrics.

Notable regressions

• We now explicitly run only one worker per container, both in case of Hypercorn serving web GUI and in case of Celery running async task worker. CPU_COUNT variable is no longer supported.

  If needed, web and celery containers themselves can be scaled horizontally. In such custom configuration it’s important to also ensure Prometheus is set up to discover scaled instances in order to import and aggregate bibliographic data access statistics from all of them. Bundled Compose configuration does not support that scenario at this time.

• Links to get bibliographic item data in Relaton JSON or XML are not shown to site visitors for the time being, as they were simply linking to API endpoints and API now requires header-based authentication—so those links wouldn’t work anymore.

  They can be reenabled in future if necessary, using a different mechanism that e.g. relies on Datatracker OIDC authentication.

Other changes

Various smaller fixes and improvements include:

• List of indexable datasets in OpenAPI spec for management endpoints is now generated dynamically and no longer erroneously includes doi
• Search query formats, changed some time back, are now documented in OpenAPI spec
• Search widget on landing page now features help explaining different query methods
• json_struct method has been added to web GUI search widget
• Version displayed in OpenAPI specs and built documentation now reflects current codebase version (such as 2022.2.1 for this release)
• and more.

---

This discussion was created from the release 2022.2.1.

[strogonoff]

@kwkwan @ronaldtse We no longer use CPU_COUNT. Sorry for the hassle. It can now be removed; and we use a fixed 1 worker per instance (both web and celery). Docker-compose configuration reflects that.