Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a registration backend that sends the form/submission variables as JSON to some API endpoint #4908

Open
8 of 11 tasks
sergei-maertens opened this issue Dec 12, 2024 · 2 comments · May be fixed by #4956
Open
8 of 11 tasks
Assignees
Labels
enhancement owner: venlo topic: registration waiting for approval An estimate was made but the stakeholder still needs to approve it.
Milestone

Comments

@sergei-maertens
Copy link
Member

sergei-maertens commented Dec 12, 2024

There is a desire for a "simple" registration backend/plugin which just dumps the submission/form variables with their values as a "flat" JSON structure and sends this data to an API endpoint for further processing. The goal is to keep this super simple and light weight, and let the post-processing be done on the receiving end (with an enterprise service bus or other technologies).

Requirements

  • A top level object with keys values and (optional?) schema is sent
  • You can specify which service to send the data to (reference to zgw_consumers.Service)
  • You can configure which endpoint to send the data to
  • For each form variable, you can include/exclude it in the data with a simple checkbox in the variables table
  • Optionally, a JSON Schema definition of the submitted data is provided in the schema key

Example

An example form with the following configuration:

  • Authentication via DigiD, capturing a BSN
  • First form step with text fields firstName and lastName
  • Second form step with a user upload attachment

Should lead to a JSON object being sent with the shape:

{
    "values":
    {
        "auth_bsn": "123456782",
        "firstName": "Donnie",
        "lastName": "Darko",
        "attachment": "<base64 encoded data>"
    },
    "schema":
    {
        "$schema": "https://json-schema.org/draft/2020-12/schema",
        "type": "object",
        "properties":
        {
            "auth_bsn":
            {
                "type": "string",
                "pattern": "^\\d{9}$"
            },
            "firstName":
            {
                "type": "string"
            },
            "lastName":
            {
                "type": "string"
            },
            "attachment":
            {
                "type": "string",
                "contentEncoding": "base64"
            }
        },
        "required":
        [
            "auth_bsn",
            "firstName",
            "lastName"
        ],
        "additionalProperties": false
    }
}

Tasks

  • Define new registration plugin - it's probably wise to gate it behind a if settings.DEBUG since I don't expect this to be done for 3.0
  • Implement plugin configuration options:
    • Service to use
    • Endpoint to send data to (relative to service API root)
    • Form variables (including static variables!) to include
  • Add docker-compose mock service (flask app?) to simulate a receiving server
  • Implement sending the data to the configured service in the defined format, taking into account the configuration options
  • Add configuration checks to configuration overview
  • Ensure that we can generate an appropriate json-schema for each form component type

Pointers

Existing code that can be useful:

  • service fetch
  • email registration plugin
  • objects API registration configuration/options
@sergei-maertens sergei-maertens added this to the Release 3.1.0 milestone Dec 12, 2024
@sergei-maertens sergei-maertens added waiting for approval An estimate was made but the stakeholder still needs to approve it. owner: venlo labels Dec 12, 2024
@viktorvanwijk viktorvanwijk moved this from Todo to In Progress in Development Dec 16, 2024
viktorvanwijk added a commit that referenced this issue Dec 18, 2024
viktorvanwijk added a commit that referenced this issue Dec 18, 2024
Plugin name did not make sense before
viktorvanwijk added a commit that referenced this issue Dec 18, 2024
API endpoint is relative, name should reflect that
viktorvanwijk added a commit that referenced this issue Dec 18, 2024
@viktorvanwijk
Copy link
Contributor

viktorvanwijk commented Dec 18, 2024

  • Optionally, a JSON Schema definition of the submitted data is provided in the schema key

Does 'optionally' mean that the configuration options should also include a checkbox (for example) with which you can select this?

@sergei-maertens
Copy link
Member Author

  • Optionally, a JSON Schema definition of the submitted data is provided in the schema key

Does 'optionally' mean that the configuration options should also include a checkbox (for example) with which you can select this?

No - it's a wish to include this, but having the actual data is the minimum requirement for this ticket. Including the JSON Schema is desired, but not crucial and should likely be the final task of this ticket to implement :)

viktorvanwijk added a commit that referenced this issue Dec 19, 2024
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
Plugin name did not make sense before
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
API endpoint is relative, name should reflect that
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
It should match the corresponding property of the serializer (which is 'service' and not 'service_select')
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
Not relevant anymore or duplicates
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
Wrong place and doesn't look great
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
…n plugin

Need to include all form variables listed in the options to a values dictionary
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
The content of the attachment is now added to the result
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
Now includes a check on the generated values dictionary for easy testing
@viktorvanwijk viktorvanwijk linked a pull request Dec 19, 2024 that will close this issue
10 tasks
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
This is a requirement of the plugin
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
The user might want to send it to just the api root of the service
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
Makes more sense to keep static variables dict close to where it's used
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
The previous size was not big enough to show all elements without scroll bars
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
viktorvanwijk added a commit that referenced this issue Dec 20, 2024
Required for simulating a receiving service
viktorvanwijk added a commit that referenced this issue Dec 20, 2024
It is sufficient to only pass the relative endpoint to the .post method
viktorvanwijk added a commit that referenced this issue Dec 20, 2024
This check if redundant, as the for loop will never be executed anyway if the query set is empty
viktorvanwijk added a commit that referenced this issue Dec 20, 2024
It is closely related to the serializer
viktorvanwijk added a commit that referenced this issue Dec 20, 2024
…pes only

Only ORC type services are relevant for this plugin
viktorvanwijk added a commit that referenced this issue Dec 20, 2024
"Any other fields that need special attention?":
Not for the moment, all the fields are JSON serializable

"Service type relevant?":
Yes, but not here. In the serializer, it should only show the services of type ORC. The other ones are not relevant for this plugin

"Does execute_unless_result_exists need to be used?":
Not for this plugin. It is normally used in plugins where a lot of API calls are made, where it prevents objects from being created multiple times on retries (when one or multiple API calls had failed).
viktorvanwijk added a commit that referenced this issue Dec 20, 2024
This will raise a KeyError if a form variable is not available in the submitted data nor in the static variables. This is better than passing is silently
viktorvanwijk added a commit that referenced this issue Dec 20, 2024
viktorvanwijk added a commit that referenced this issue Dec 23, 2024
…e service

The instructions on how to start it is now in line with the other docker compose services
viktorvanwijk added a commit that referenced this issue Dec 23, 2024
The configuration options could be saved without specifying any form variables to include. This does not make much sense. Added a minimum length of 1 for the form variables list in the serializer (Thanks to Robin for the hint)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement owner: venlo topic: registration waiting for approval An estimate was made but the stakeholder still needs to approve it.
Projects
Status: In Progress
Development

Successfully merging a pull request may close this issue.

2 participants