open-formulieren · sergei-maertens · Nov 9, 2023 · Nov 6, 2023 · Nov 6, 2023 · Nov 6, 2023
diff --git a/.bumpversion.cfg b/.bumpversion.cfg
@@ -1,7 +1,7 @@
 [bumpversion]
 commit = False
 tag = False
-current_version = 2.4.0-alpha.0
+current_version = 2.4.0
 parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(\-(?P<pre>[a-z]+)\.(?P<build>\d+))?
 serialize = 
 	{major}.{minor}.{patch}-{pre}.{build}

diff --git a/.sdk-release b/.sdk-release
@@ -1 +1 @@
-1.6.0-alpha.0
+2.0.0
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
@@ -2,48 +2,116 @@
 Changelog
 =========
 
-2.4.0 "..." (2023-??-??)
-========================
+2.4.0 "Miffy" (2023-11-09)
+==========================
 
-.. note:: These release notes are in development!
+Open Forms 2.4.0 is a feature release.
 
-**DigiD - Eherkenning Configuration**
+.. epigraph::
 
-The configuration concerning the digid and eherkenning has been updated according to the
-new version (0.9) of the django-digid-eherkenning library. The XML metadata file is now
-automatically retrieved so you have to edit the configuration via:
-**Admin** > **Configuratie** > **DigiD-configuratie**
-**Admin** > **Configuratie** > **EHerkenning/eIDAS-configuratie**
-and add the XML metadata url. The identity provider and the metadata file fields will be
-automatically populated if the url is valid.
+   **Miffy** (or "Nijntje" in Dutch) is a fictional rabbit appearing in a series of
+   picture books authored by Dick Bruna. Both are famous Utrecht citizens. You can find
+   Miffy in a number of places, such as the "Nijntje Pleintje" (Miffy Square) and a set
+   of pedestrian traffic lights in the shape of the rabbit in the city center.
 
 Upgrade procedure
 -----------------
 
-**KVK configuration**
+⚠️ Ensure you upgrade to Open Forms 2.3.0 before upgrading to the 2.4 release series.
 
-We've done some extensive code and configuration cleanups, and the KVK configuration may
-be affected by this. The service configuration for the "zoeken" and "basisprofiel" API's
-has been merged into a single service. Normally this configuration update should have
-been performed correctly automatically, but we can't rule out possible mistakes with
-more exotic configurations via API gateways.
+To keep the codebase maintainable and follow best pratices, we've done some considerable
+cleanups in the code that may require some special attention. We've collected the
+details for this release in a separate documentation page.
+
+⚠️ Please review the instructions in the documentation under **Installation** >
+**Upgrade details to Open Forms 2.4.0** before and during upgrading.
 
-⚠️ Please check and update your configuration if necessary - you can check this via:
-**Admin** > **Configuratie** > **Configuratie overzicht** and look for the KVK entries.
+Major features
+--------------
 
-If you run into any errors, then please check your certificate configuration, API key
-and validate that the API root does *not* include the ``v1/`` suffix. An example of a
-correct API root: ``https://api.kvk.nl/api/`` or ``https://api.kvk.nl/test/api/``.
+***️ (Experimental) Suwinet plugin**
 
-**Button refactor to NL DS**
+We now support retrieving data for a logged in user (with BSN) through Suwinet. This
+feature is in experimental/preview mode, so we rely on your feedback on how to further
+develop and improve this.
 
-We are committed to NL Design System and have overhauled our custom button
-implementation and styles. While we have done our best to provide an automatic
-migration of button styling configuration, there may be some changes that we cannot
-detect or handle automatically.
+**📅 Appointments**
 
-⚠️ Please review the instructions in the documentation under **Installation** >
-**Upgrade details to Open Forms 2.4.0** > **Button styling**.
+Our Qmatic appointments plugin now also supports multiple customer/multiple products
+flows, matching the JCC feature set.
+
+**🧩 More NL Design System components**
+
+We continue bridging the gap between our custom UI-components and available NL DS
+components. Our buttons and links now no longer require OF-specific tokens and we've
+removed a whole bunch of styling code that got in the way when building your own theme.
+
+More will come in the future!
+
+Detailed changes
+----------------
+
+The 2.4.0-alpha.0 changes are included as well, see the earlier changelog entry.
+
+**New features**
+
+* Form designer
+
+    * [#586] Added support for Suwinet as a prefill plugin.
+    * [#3188] Added better error feedback when adding form steps to a form with
+      duplicate keys.
+    * [#3351] The family members component can now be used to retrieve partner
+      information instead of only the children (you can select children, partners or
+      both).
+    * [#2953] Added support for durations between dates in JSON-logic.
+    * [#2952] Form steps can now initially be non-applicable and dynamically be made
+      applicable.
+
+* [#3499] Accepting/declining cookies in the notice now no longer refreshes the page.
+* [#3477] Added CSP ``form-action`` directives, generated via the DigiD/eHerkenning
+  and Ogone configuration.
+* [#3524] The behaviour when retrieving family members who don't have a BSN is now
+  consistent and well-defined.
+* [#3566] Replaced custom buttons with utrecht-button components.
+
+**Bugfixes**
+
+* [#3527] Duplicated form steps in a form are now blocked at the database level.
+* [#3448] Fixed emails not being sent with a subject line > 70 characters.
+* [#3448] Fixed a performance issue when upgrading the underlying email sending library
+  if you have many (queued) emails.
+* [#2629] Fixed array variable inputs in the form designer.
+* [#3491] Fixed slowdown in the form designer when created a new or loading an existing
+  form when many reusable form definitions exist.
+* [#3557] Fixed a bug that would not display the available document types when
+  configuring the file upload component.
+* [#3553] Fixed a crash when validating a ZWG registration backend when no default
+  ZGW API group is set.
+* [#3537] Fixed validator plugin list endpoint to properly converting camelCase params
+  into snake_case.
+* [#3467] Fixed crashes when importing/copying forms with ``null`` in the prefill
+  configuration.
+* [#3580] Fixed incorrect attributes being sent in ZWG registration backend when
+  creating the rol/betrokkene.
+
+**Project maintenance**
+
+* Upgraded various dependencies with the most recent (security) releases.
+* [#2958] Started the rework for form field-level translations, the backend can now
+  handle present and future formats.
+* [#3489] All API client usage is updated to a new library, which should lead to a
+  better developer experience and make it easier to get better performance when making
+  (multiple) API calls.
+* Bumped pip-tools for latest pip compatibility.
+* [#3531] Added a custom migration operation class for formio component transformations.
+* [#3531] The time component now stores ``minTime``/``maxTime`` in the ``validate``
+  namespace.
+* Contributed a number of library extensions back to the library itself.
+* Squashed the variables app migrations.
+* [#2958] Upgraded (experimental) new form builder to 0.8.0, which uses the new
+  translations format.
+* Fixed test suite which didn't take DST into account.
+* [#3449] Documented the (new) co-sign flow.
 
 2.3.3 (2023-10-30)
 ==================
@@ -93,7 +161,7 @@ Detailed changes
 * [#3051] You can now schedule activation/deactivation of forms.
 * [#1884] Added more fine-grained custom errors for time field components.
 * More fields irrelevant to appointment forms are now hidden in the form designer.
-* [#3456] Implemented multi-product and multi-customer appointments for QMatic.
+* [#3456] Implemented multi-product and multi-customer appointments for Qmatic.
 * [#3413] Improved UX by including direct hyperlinks to the form in co-sign emails (
   admins can disable this behaviour).
 * [#3328] Qmatic appointments plugin now support mTLS.

diff --git a/Dockerfile b/Dockerfile
@@ -96,7 +96,7 @@ COPY ./bin/celery_beat.sh /celery_beat.sh
 COPY ./bin/celery_flower.sh /celery_flower.sh
 COPY ./bin/dump_configuration.sh /dump_configuration.sh
 RUN mkdir /app/bin /app/log /app/media /app/private_media /app/certifi_ca_bundle /app/tmp
-COPY ./bin/check_celery_worker_liveness.py ./bin/check_celery_worker_liveness.py
+COPY ./bin/check_celery_worker_liveness.py ./bin/check_non_unique_steps.py ./bin/
 
 # prevent writing to the container layer, which would degrade performance.
 # This also serves as a hint for the intended volumes.

diff --git a/README.NL.rst b/README.NL.rst
@@ -2,7 +2,7 @@
 Open Formulieren
 ================
 
-:Version: 2.4.0-alpha.0
+:Version: 2.4.0
 :Source: https://github.com/open-formulieren/open-forms
 :Keywords: e-Formulieren, Common Ground, FormIO, API
 
@@ -54,6 +54,8 @@ Versie          Release date    API specificatie
 ==============  ==============  =============================
 latest          n/a             `ReDoc <https://redocly.github.io/redoc/?url=https://raw.githubusercontent.com/open-formulieren/open-forms/master/src/openapi.yaml>`_,
                                 `Swagger <https://petstore.swagger.io/?url=https://raw.githubusercontent.com/open-formulieren/open-forms/master/src/openapi.yaml>`_
+2.4.0           2023-11-09      `ReDoc <https://redocly.github.io/redoc/?url=https://raw.githubusercontent.com/open-formulieren/open-forms/2.4.0/src/openapi.yaml>`_,
+                                `Swagger <https://petstore.swagger.io/?url=https://raw.githubusercontent.com/open-formulieren/open-forms/2.4.0/src/openapi.yaml>`_
 2.3.1           2023-09-25      `ReDoc <https://redocly.github.io/redoc/?url=https://raw.githubusercontent.com/open-formulieren/open-forms/2.3.1/src/openapi.yaml>`_,
                                 `Swagger <https://petstore.swagger.io/?url=https://raw.githubusercontent.com/open-formulieren/open-forms/2.3.1/src/openapi.yaml>`_
 2.3.0           2023-08-24      `ReDoc <https://redocly.github.io/redoc/?url=https://raw.githubusercontent.com/open-formulieren/open-forms/2.3.0/src/openapi.yaml>`_,

diff --git a/README.rst b/README.rst
@@ -2,7 +2,7 @@
 Open Forms
 ==========
 
-:Version: 2.4.0-alpha.0
+:Version: 2.4.0
 :Source: https://github.com/open-formulieren/open-forms
 :Keywords: e-Formulieren, Common Ground, FormIO, API
 
@@ -54,6 +54,8 @@ Version         Release date    API specification
 ==============  ==============  =============================
 latest          n/a             `ReDoc <https://redocly.github.io/redoc/?url=https://raw.githubusercontent.com/open-formulieren/open-forms/master/src/openapi.yaml>`_,
                                 `Swagger <https://petstore.swagger.io/?url=https://raw.githubusercontent.com/open-formulieren/open-forms/master/src/openapi.yaml>`_
+2.4.0           2023-11-09      `ReDoc <https://redocly.github.io/redoc/?url=https://raw.githubusercontent.com/open-formulieren/open-forms/2.4.0/src/openapi.yaml>`_,
+                                `Swagger <https://petstore.swagger.io/?url=https://raw.githubusercontent.com/open-formulieren/open-forms/2.4.0/src/openapi.yaml>`_
 2.3.1           2023-09-25      `ReDoc <https://redocly.github.io/redoc/?url=https://raw.githubusercontent.com/open-formulieren/open-forms/2.3.1/src/openapi.yaml>`_,
                                 `Swagger <https://petstore.swagger.io/?url=https://raw.githubusercontent.com/open-formulieren/open-forms/2.3.1/src/openapi.yaml>`_
 2.3.0           2023-08-24      `ReDoc <https://redocly.github.io/redoc/?url=https://raw.githubusercontent.com/open-formulieren/open-forms/2.3.0/src/openapi.yaml>`_,

diff --git a/docs/developers/i18n.rst b/docs/developers/i18n.rst
@@ -16,7 +16,7 @@ For the backend, see the Django `i18n documentation`_.
     before re-generating.
 
 
-.. _i18n documentation: https://docs.djangoproject.com/en/2.2/topics/i18n/translation/
+.. _i18n documentation: https://docs.djangoproject.com/en/3.2/topics/i18n/translation/
 
 .. _`Forms Admin`:
 

diff --git a/docs/developers/versioning.rst b/docs/developers/versioning.rst
@@ -51,6 +51,7 @@ experimental feature changes (see :ref:`developers_versioning_api`).
    1.3.0       2.1.0               n/a
    1.4.0       2.2.0               n/a
    1.5.0       2.3.0               n/a
+   2.0.0       2.4.0               n/a
    =========== =================== ===================
 
 .. _developers_versioning_api:
@@ -98,6 +99,7 @@ backend version.
    =============== ===========
    Backend version API version
    =============== ===========
+   2.4.x           2.4.y
    2.3.x           2.3.y
    2.2.x           2.2.y
    2.1.x           2.1.y

diff --git a/docs/installation/upgrade-240.rst b/docs/installation/upgrade-240.rst
@@ -8,9 +8,80 @@ We do our best to avoid breaking changes, but at times we cannot guarantee that
 configuration will keep working flawlessly. Open Forms 2.4.0 is such a release - and
 the manual interventions with context are documented here.
 
+.. contents:: Jump to
+   :depth: 1
+   :local:
+
+
+Check forms with duplicated steps
+=================================
+
+We now block duplicated form steps at the database level - if you have forms in your
+environment with non-unique steps, the upgrade will refuse to execute (or crash if
+you disable checks).
+
+The check script is available since Open Forms 2.3.4, you can run it using:
+
+.. code-block:: bash
+
+    python /app/bin/check_non_unique_steps.py
+
+and it will report any problematic forms. These forms need to be updated manually to
+remove the duplicated steps.
+
+DigiD/eHerkenning configuration
+===============================
+
+DigiD and/or eHerkenning configuration before Open Forms 2.4.0 required manually
+uploading the XML metadata file.
+
+This file is now automatically retrieved, provided you configure the "(XML) metadata
+URL". We cannot populate this URL on existing instances, so you need to configure this
+manually by navigating to:
+
+* **Admin** > **Configuratie** > **DigiD-configuratie** and
+* **Admin** > **Configuratie** > **EHerkenning/eIDAS-configuratie**
+
+The field can be found under the "Identity provider" section.
+
+Once these URLs are configured, the metadata file field and the identity provider ID
+will be automatically populated.
+
+Additionally, because the CSP directives are generated from this configuration, we
+recommend saving the configuration once (even if you made no changes) to create the
+necessary CSP configuration records.
+
+.. note:: Any previously uploaded metadata files continue to work as expected.
+
+Ogone configuration
+===================
+
+After upgrading, you should go to the Ogone merchants configuration and save every
+merchant (even if you made no changes) to populate the related CSP configuration
+records.
+
+KVK Configuration
+=================
+
+We've done some extensive code and configuration cleanups, and the KVK configuration may
+be affected by this. The service configuration for the "zoeken" and "basisprofiel" API's
+has been merged into a single service. Normally this configuration update should have
+been performed correctly automatically, but we can't rule out possible mistakes with
+more exotic configurations (e.g. when using API gateways).
+
+⚠️ Please check and update your configuration if necessary - you can check this via:
+**Admin** > **Configuratie** > **Configuratie overzicht** and look for the KVK entries.
+
+If you run into any errors, then please check your certificate configuration, API key
+and validate that the API root does *not* include the ``v1/`` suffix. An example of a
+correct API root: ``https://api.kvk.nl/api/`` or ``https://api.kvk.nl/test/api/``.
+
 Button styling
 ==============
 
+We are committed to NL Design System and have overhauled our custom button
+implementation and styles.
+
 Our buttons used custom design tokens since they were implemented before the `NL Design
 System buttons`_ were much of a thing. *Most* of the tokens map cleanly, and if you
 have configured any overrides in the admin environment they should have been
@@ -19,7 +90,7 @@ test/acceptance environment and prepare the necessary changes.
 
 However, if you have made custom CSS changes, we cannot incorporate those and you will
 have to check your customizations. You will have to manually check if those changes
-are still relevant or not. Our advice is also to take a critical look - the transition
+are still relevant or not. Our advice is to also take a critical look - the transition
 to community components should make it much easier to change the appearance and behaviour
 using design tokens rather than CSS overrides.
 
@@ -168,3 +239,30 @@ The mapping below shows the NL Design System tokens populated from our custom to
 
 The commented out tokens are values that used to be hardcoded in our CSS, but should now
 be specified via design tokens and reflect the (default) values set in the Open Forms theme.
+
+Alert styling
+=============
+
+We've refactored our alert styling to make use of the ``utrecht-alert`` component. This
+effectively replaces the ``--of`` design tokens with the ``--utrecht`` ones. A backwards
+compatibility layer is set up which will be dropped in Open Forms 3.0, so we recommend
+updating your stylesheets.
+
+There is no automatic data migration set up (yet).
+
+Reference
+---------
+
+The mapping below shows the NL Design System tokens populated from our custom tokens:
+
+.. code-block:: scss
+
+    :root {
+        --utrecht-alert-warning-background-color: var(--of-alert-warning-bg);
+        --utrecht-alert-info-background-color: var(--of-alert-info-bg);
+        --utrecht-alert-error-background-color: var(--of-alert-error-bg);
+        --utrecht-alert-icon-error-color: var(--of-color-danger);
+        --utrecht-alert-icon-info-color: var(--of-color-info);
+        --utrecht-alert-icon-warning-color: var(--of-color-warning);
+        --utrecht-alert-icon-ok-color: var(--of-color-success);
+    }
diff --git a/package-lock.json b/package-lock.json
diff --git a/package.json b/package.json
@@ -1,6 +1,6 @@
 {
   "name": "openforms",
-  "version": "2.4.0-alpha.0",
+  "version": "2.4.0",
   "description": "Open Forms",
   "main": "src/static/openforms/js/openforms.js",
   "directories": {

diff --git a/publiccode.yaml b/publiccode.yaml
@@ -7,7 +7,7 @@ publiccodeYmlVersion: '0.2'
 name: Open Forms Builder and API
 url: 'http://github.com/open-formulieren/open-forms.git'
 softwareType: standalone/backend
-softwareVersion: 2.4.0-alpha.0
+softwareVersion: 2.4.0
 releaseDate: '2022-03-10'
 logo: 'https://github.com/open-formulieren/open-forms/blob/master/docs/logo.svg'
 platforms: