Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#1650 provide WoT TM validation #1936

Merged
merged 27 commits into from
Aug 26, 2024

Conversation

thjaeckle
Copy link
Member

@thjaeckle thjaeckle commented May 6, 2024

  • restructuring of "ditto-wot" module to enable re-usability of non-pekko/non-Ditto specifics
  • adding "validator" concept and first sample implementation
  • using a json-schema-validation library to create validation exception details
  • applying validator to:
    • thing attribute updates
    • feature property updates
    • thing actions (inputs and outputs)
    • feature actions (inputs and outputs)
    • thing events
    • feature events
  • each aspect of validation is fully configurable
    • whether to enable/disable
    • also dynamically, e.g. excluding aspects for certain models from validation
  • adding traces in order to determine how much this validation costs "extra" in a productive setup

Open points:

  • Provide actual validation of TMs for all hierarchy levels of changes to Things
  • Optimize conversion from Ditto-Json to Jackson
  • Provide validation for WoT actions and their responses
  • Provide validation for WoT events
  • Add support for WoT oneOf for DataSchema - validation must only match "one of" the provided schemas
  • Provide advanced configuration options to be able to "skip" validation on certain conditions, e.g.:
    • when auth-subject is "xyz"
    • when model URL matches a configured regex
  • Add (enforcement) metrics in order to be able to inspect how much time is spent for validation
  • Provide tests
  • Provide Helm configuration
  • Provide documentation

Resolves: #1650

@thjaeckle thjaeckle added this to the 3.6.0 milestone May 6, 2024
@thjaeckle thjaeckle self-assigned this May 6, 2024
@thjaeckle thjaeckle force-pushed the wot-tm-validation branch 8 times, most recently from 7230638 to 4a8e4b6 Compare May 7, 2024 14:10
@thjaeckle thjaeckle force-pushed the wot-tm-validation branch 2 times, most recently from 45ae2b5 to 49b206e Compare June 24, 2024 07:50
@thjaeckle thjaeckle added the WoT Web of Things related enhancements label Jun 26, 2024
@thjaeckle thjaeckle force-pushed the wot-tm-validation branch 3 times, most recently from c0ab94d to c05c867 Compare July 17, 2024 05:33
thjaeckle added 14 commits July 29, 2024 13:31
* restructuring of "ditto-wot" module to enable re-usability of non-pekko/non-Ditto specifics
* adding "validator" concept and first sample implementation (WIP)

Signed-off-by: Thomas Jäckle <[email protected]>
…desired properties validation

* taking care of e.g. "ditto:category" defined
* first validation error will provide the error output (which is for now simpler and results in quicker obtained validation errors)

Signed-off-by: Thomas Jäckle <[email protected]>
* validating features and properties as part of "modify thing"
* handling "ditto:category" correctly
* ensuring completeness of defined features
* validation on creation/modification
  * of thing
  * of attributes
  * on attribute
  * on features
  * on feature

Signed-off-by: Thomas Jäckle <[email protected]>
* enforce modifying thing definition + feature definition
* enhance configuration options
…their responses

* also updated some dependencies
* simplified DefaultWotThingModelValidator a little
…iding "type" (valid for e.g. "oneOf") use

* update json-schema-validator library
…ation in smaller pieces

* strict split between config reading in DefaultWotThingModelValidation and static functionality in Internal* classes
thjaeckle added 11 commits July 29, 2024 13:32
* based on either DittoHeaders or on ThingDefinition/FeatureDefinition URLs
* fixed and harmonized other traces while doing that in order to get correct parent hierarchy of spans
…ased on the WoT model

* restructured some too long methods
* fixed remaining TODOs
* also for dynamic part
* changed config structure a bit in order to better fit Helm YAML config
…e corner cases

* e.g. deeply nested updates, removing definition via PATCH
* enabled WoT based validation by default
* first choosing potentially new definition and only as fall-back the old one
@thjaeckle thjaeckle force-pushed the wot-tm-validation branch from dbc03b1 to 608894f Compare July 29, 2024 11:46
@thjaeckle thjaeckle marked this pull request as ready for review July 29, 2024 11:46
@thjaeckle
Copy link
Member Author

FMPOV ready to be reviewed :)

It is of course a massively huge PR - let's discuss how to review it most efficiently @alstanchev @kalinkostashki

@thjaeckle thjaeckle requested a review from alstanchev July 31, 2024 07:15
@thjaeckle
Copy link
Member Author

@alstanchev did you manage to have a look? would want to merge this PR soon ..

@alstanchev
Copy link
Contributor

alstanchev commented Aug 26, 2024

yes, LGTM 👍

@thjaeckle thjaeckle merged commit e4ce12e into eclipse-ditto:master Aug 26, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
WoT Web of Things related enhancements
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Add option to enforce validation based on referenced WoT model
2 participants