docs: Proposal for FederatedCatalog Distribution and TargetNodeDirectory #1718
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… into dr_proposal_federated_catalog_distribution
… into dr_proposal_federated_catalog_distribution
… into dr_proposal_federated_catalog_distribution
… into dr_proposal_federated_catalog_distribution
… into dr_proposal_federated_catalog_distribution
… into dr_proposal_federated_catalog_distribution
|
@ndr-brt this is the new PR :) |
|
This pull request is stale because it has been open for 7 days with no activity. |
|
This pull request was closed because it has been inactive for 7 days since being marked as stale. |
|
lgblaumeiser
requested changes
Jan 6, 2025
|
|
||
| Other solution for the TargetNodeDirectory was also considered | ||
| - File in a S3 bucket (or different cloud provider's solution) | ||
| - This solution was discarded due to one file for all instead of each partner having the data that respectively needs does not match the requirement and this solution would lock the usage of a proprietary tool (cloud provider) being harder to sustain in the long run. |
There was a problem hiding this comment.
I do not understand the sentence, the part "the data that respoectively needs does not match the requirement"? What does this mean
|
|
||
| ## Approach | ||
|
|
||
| For TargetNodeDirectory it will be set by a new extension responsible for exposing an API, where a member can input the DID's of the participants from which the catalogs are wanted, and then it will retrieve and store the respective Connector URL's. This new extension would get the data from the Discovery Service, and will be named `DiscoveryServiceRetrieverExtension`. This solution allows the member to choose precisely the Target Catalog Nodes that interests them, resulting in reduced network calls and latency. |
There was a problem hiding this comment.
With a provided DID, how is the member able to choose precisely the catalog nodes he is interested in? He will receive all connectors registered from the discovery service, right?
| ## Approach | ||
|
|
||
| For TargetNodeDirectory it will be set by a new extension responsible for exposing an API, where a member can input the DID's of the participants from which the catalogs are wanted, and then it will retrieve and store the respective Connector URL's. This new extension would get the data from the Discovery Service, and will be named `DiscoveryServiceRetrieverExtension`. This solution allows the member to choose precisely the Target Catalog Nodes that interests them, resulting in reduced network calls and latency. | ||
| Additionally, if a Connector URL is registered (or unregistered) in the Discovery Service, the retriever will reflect it since it requests based on BPN and the registered URL's will be returned. |
There was a problem hiding this comment.
- Here you talk about BPN although previously you talked about DIDs.
- Why will changes in the discovery service be reflected in the federated catalog? Above, it is described, that the federated catalog service will retrieve and store the connector url's, changes in the discovery will only be recognized, if the same BPN/DID will be retrieved again. It is not mentioned that the local storage is a caching mechanism.
|
|
||
| Since the Federated Catalog Cache will be a standalone runtime, the Tractus-X EDC Connector Helm charts will be updated to include the Federated Catalog Cache as a separated deployment. The update will include the creation of a specific `deployment-federatedcatalog.yaml`, similar [to this one](https://github.com/eclipse-tractusx/tractusx-edc/blob/a263bf71a110245657131509d4b37d058a1d220d/charts/tractusx-connector-azure-vault/templates/deployment-dataplane.yaml#L47) (for `ingress` and `hpa` as well), for different scenarios (InMemory, PostreSQL, etc.). This results in added configuration complexity. | ||
|
|
||
| For its TargetNodeDirectory, the extension is able to obtain the Connectors' URL's through the Discovery Service and store them. Two API's will be provided in this new extension, at least during alpha stage, one to allow the user to input a list of DID's and other for BPN's. The `DiscoveryServiceRetrieverExtension` is responsible to retrieve the data and store it (in memory or in a database). The URL's can later be retrieved and crawled by the Federated Catalog Cache. |
There was a problem hiding this comment.
Suggested change
| For its TargetNodeDirectory, the extension is able to obtain the Connectors' URL's through the Discovery Service and store them. Two API's will be provided in this new extension, at least during alpha stage, one to allow the user to input a list of DID's and other for BPN's. The `DiscoveryServiceRetrieverExtension` is responsible to retrieve the data and store it (in memory or in a database). The URL's can later be retrieved and crawled by the Federated Catalog Cache. | |
| For its TargetNodeDirectory, the extension is able to obtain the Connectors' URL's through the Discovery Service and store them. Two API's will be provided in this new extension, at least during alpha stage, one to allow the user to input a list of DID's and other for BPN's. The `DiscoveryServiceRetrieverExtension` is responsible to retrieve the data and store it (in memory or in a database). |
| For its TargetNodeDirectory, the extension is able to obtain the Connectors' URL's through the Discovery Service and store them. Two API's will be provided in this new extension, at least during alpha stage, one to allow the user to input a list of DID's and other for BPN's. The `DiscoveryServiceRetrieverExtension` is responsible to retrieve the data and store it (in memory or in a database). The URL's can later be retrieved and crawled by the Federated Catalog Cache. | ||
| By default no TargetNodes are stored, so the extension will not request data from the Discovery Service. | ||
|
|
||
| A DID added through the `DiscoveryServiceRetrieverExtension` API will be resolved with the BDRS client to obtain the BPN which will be used to query the Discovery Service. the BDRS client must be updated [since only allows to resolve a BPN to a DID and not the other way around](https://github.com/eclipse-tractusx/tractusx-edc/blob/8e1a3202be77d6374731dee5aaf6847feec8963a/spi/bdrs-client-spi/src/main/java/org/eclipse/tractusx/edc/spi/identity/mapper/BdrsClient.java). A change to resolve a BPN given the respective DID has to be done prior to the new extension. |
There was a problem hiding this comment.
Suggested change
| A DID added through the `DiscoveryServiceRetrieverExtension` API will be resolved with the BDRS client to obtain the BPN which will be used to query the Discovery Service. the BDRS client must be updated [since only allows to resolve a BPN to a DID and not the other way around](https://github.com/eclipse-tractusx/tractusx-edc/blob/8e1a3202be77d6374731dee5aaf6847feec8963a/spi/bdrs-client-spi/src/main/java/org/eclipse/tractusx/edc/spi/identity/mapper/BdrsClient.java). A change to resolve a BPN given the respective DID has to be done prior to the new extension. | |
| A DID added through the `DiscoveryServiceRetrieverExtension` API will be resolved with the BDRS client to obtain the BPN which will be used to query the Discovery Service. The BDRS client must be enhanced to support a reversed lookup from a given DID to the matching BPN. This change affects the file [BdrsClient.java](https://github.com/eclipse-tractusx/tractusx-edc/blob/8e1a3202be77d6374731dee5aaf6847feec8963a/spi/bdrs-client-spi/src/main/java/org/eclipse/tractusx/edc/spi/identity/mapper/BdrsClient.java). |
| Some limitations of this TargetNodeDirectory solution are: | ||
| - Each partner must have the DID's beforehand. If a new Partner is registered and an existing partner would want their catalog, the DID (or BPN) of the new partner must be obtained first and added through the new extension API; | ||
| - The usage of the Discovery Service requires a technical user account to access it (must be requested). After obtaining them, the credentials can be stored in the vault; | ||
| - Change in the BDRS client to allow resolve a BPN provided the DID. |
There was a problem hiding this comment.
That is also not a limitation, but a fact you already described above.
|
|
||
| Since the Federated Catalog Cache will be a standalone runtime, the Tractus-X EDC Connector Helm charts will be updated to include the Federated Catalog Cache as a separated deployment. The update will include the creation of a specific `deployment-federatedcatalog.yaml`, similar [to this one](https://github.com/eclipse-tractusx/tractusx-edc/blob/a263bf71a110245657131509d4b37d058a1d220d/charts/tractusx-connector-azure-vault/templates/deployment-dataplane.yaml#L47) (for `ingress` and `hpa` as well), for different scenarios (InMemory, PostreSQL, etc.). This results in added configuration complexity. | ||
|
|
||
| For its TargetNodeDirectory, the extension is able to obtain the Connectors' URL's through the Discovery Service and store them. Two API's will be provided in this new extension, at least during alpha stage, one to allow the user to input a list of DID's and other for BPN's. The `DiscoveryServiceRetrieverExtension` is responsible to retrieve the data and store it (in memory or in a database). The URL's can later be retrieved and crawled by the Federated Catalog Cache. |
There was a problem hiding this comment.
Is it necessary to provide two api families, one for bpn and one for did, as it can be easily decided internally, what has been provided. So one api family that allows to CRUD an identifier could be enough.
| Once a member understands that they do not need the Catalogs from a certain DID, this can be removed. | ||
| DID to be removed is sent as a path param. | ||
| ``` | ||
| [DELETE] /api/target-nodes/{did} |
There was a problem hiding this comment.
I am not sure, whether a DID is a suitable part in an url, must at least be encoded I suppose.
| #### Retrieve DID's | ||
| Get DID's (value and connectors associated with it). | ||
| ``` | ||
| [POST] /api/target-nodes/request |
There was a problem hiding this comment.
Suggested change
| [POST] /api/target-nodes/request | |
| [GET] /api/target-nodes?filter_did={did1}&filter_did={did2} |
There is no need to make use of POST for such an easy get endpoint
| ] | ||
| }, | ||
| { | ||
| "bpn": "did:web:info:api:administration:staticdata:did:BPNL000000000002", |
There was a problem hiding this comment.
This is wrong, this is a DID, not a BPN. The BPN would be BPNL000000000002. Actually, I would propose to return both, the DID and the BPN. From this example, it is unclear to me, when a did and when a bpn is returned.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
WHAT
Decision Record for new proposal for FederatedCatalog distribution (with Tractus-X) and the TargetNodeDirectory (TND).
The TND initial proposal was defined in #1556 PR and continue in the current one.
Related with #1585
References: #1555
Closes eclipse-tractusx/sig-release#736