OntoPortal federation

[syphax-bouazzouni]

Context

This topic was introduced and discussed in the OntoPortal 2023 workshop. For more details, please refer to the OntoPortal Federated services session notes

Definition of federation

Federation refers to capacity to integrate and federate multiple (not necessarily all) OntoPortals. This allows a user of a Portal A to remotely access various types of information such as ontology metadata, concepts, properties, instances, users, notes, and more, stored remotely in Portal B or even C, D, etc. The federation does not necessarily involve all the portals. Technically speaking it's possible to federate with all, but it can be customized portal per portal. We have to keep in mind that federated queries can create technical challenges (delays, connections, etc.) that could produce bad user experience.

Canonical portal

A federation system, introduces certain content management challenges, specifically regarding duplication, provenance, and ownership. A proposed solution is to establish a Canonical portal for a given resource (ontology, user, project, note, etc.), signifying that if Portal A is designated as the canonical portal of a resource, it assumes responsibility for maintenance and curation. In such cases, the content from this canonical portal takes precedence over duplicates found in other portals within the federation.

• If the canonical portal is unavailable or unable to provide a particular resource, duplicates from the current portal will be chosen as a fallback option.
• If a resource is published across multiple portals without a designated canonical portal, priority will be given to the version present in the current portal.
• Although duplicates between portals are not ideal and often confusing , it is essential to note that the federation does not require the deletion of duplicated resources. As outlined above, the system will select a resource to display based on the situation. However, links to other duplications will persist, and users will be notified of their existence, providing them the option to access them if desired (e.g. links in the summary pages).
• Federation is not a response to handle multiple versioning. A canonical portal for a given ontology shall be in charge to host and serve the latest version of the ontology. If a portal hosts multiple versions of an ontology (as primary object i.e., with different acronym IDs), those shall be seen as “separated” ontologies in the federation.

Types of federation

As we utilize the same OntoPortal technology, for storing and sharing resources, we benefit significantly from sharing a (relatively) common API and UI. This commonality greatly facilitates the potential success of the federation. To address the introduced specificity, we differentiate between Internal OntoPortal Federation and External OntoPortal Federation. This distinction is crucial in navigating the nuanced aspects of our federated approach.

In addition, federation can occur on multiple levels:

• Content (ontology) federation: An ontology published in Portal A can be discovered/found and accessed by all members of the federation. This means federating the metadata.
• Search federation: A search query can be sent to federated portals, and results are merged and ranked based on the portal emitting the query.
• Any-object federation: Federating any of the objects (in additon to ontologies) that OntoPortal manage e.g., users, agents, projects, notes, etc. For example, we might expect that a user from Portal A can log in to all portals in the federation.
• Annotator/Recommender Federation: Computed annotations/recommendations can be performed across all portals of the federation.
• Mappings federation: Computed mappings can be accessed and used across all portals of the federation.

Goal

In this initial iteration, our primary focus will be on implementing the Content and Search federation within an Internal Federation.

External federations involving other repositories/tools, such as OLS, ShowVoc, SKOSMOS, and others, require extensive discussions with these entities to establish a consensus on a common API. Note: the FAIR-IMPACT project is working on a shared API that could enabled the external federation depicted. See https://github.com/FAIR-IMPACT/MOD-API

The implementation of Users, Mappings, and Annotator/Recommender federation is planned for a subsequent iteration.

How to achieve an OntoPortal federation

Currently, an OntoPortal isntance works, as described below: we have the three main components involved, the UI displaying the content, the API handling the storage and retrieval of data, and an intermediate component bridging the connection between the UI and API over HTTP. This intermediate component translates the UI code into HTTP requests for the API, and conversely, parses the returned JSON into code that the UI can comprehend and render.

The key factor that shall enable the OntoPortal federation relatively easily is enhancing the intermediate component connecting the UI with the API. By upgrading it to handle multiple parallel calls to different APIs without altering the current deployed APIs, we can capitalize on each instance having already indexed and cached its data. This eliminates the need for a centralized index, which could be complex to maintain in the long term.

Requirements

To be continued

• The federation of metadata will only be possible with portal following the same metadata model. Currently with have two such models.
• Canonical model also brings the question of harmonizing acronym IDs so duplicates are detected.

Action items

• Feature: implement federation of collections calls ontoportal-lirmm/ontologies_api_ruby_client#17
• Feature: Implement federatation for the browse page ontoportal-lirmm/bioportal_web_ui#621
• Feature: Aggretate federated search results ontoportal-lirmm/ontologies_api_ruby_client#18
• Federate search  agroportal/project-management#593
• [ ]

[jonquet]

Complement information from the OntoPortal Workshop 2023 report:

The OntoPortal Alliance is discussing the “federation” of all/some portals (metadata, search, content, …) Do you have any suggestions on this?

• Useful to have a federated system among the portals to deal with similar content.
• The concept and class search would be nice to have with federated portals.
• Search, referencing other resources, and dealing with duplicates/overlaps.
• Keep domain-specific ontology services, so users can decide whether to stay in a specific domain or generalize.
• ​​For specific repos domain-specific services make sense, so only think of this as an add-on.
• Must find the right ontologies, which means the ability to find the right portals.
• Really in favor of federation because duplicate ontologies are a nightmare.
• Be careful with keeping each portal's identity.
• Each portal has its own editorial process and constraints.
• Metadata authoring facilitated by the federation.

[jonquet]

Additionnal requirement from report:

• Implement a view to list (with linked logos) the public OntoPortal installation in the OntoPortal Alliance. A commitment to the Alliance means a presence on the page on all the portals.
• Better coordination when a semantic resource is hosted in multiple repositories in the alliance.
• Avoid repeating the authoring of metadata from one portal to another.
• Should be easy for final users to find content on different OntoPortal installations.
• Implement a federated search of content. Grab either statistics (number of hits) or even results (to produce snippets) in the querying interface.
• Implement a cross-portal metadata search service that will facilitate the identification of the semantic resources.
• Federated Recommender. When a query is passed to the recommender, select the option to query also the other Recommenders.