Breaking change: Removal of SERVICE clause insertion in test queries

Due to bugs and the fact that it limits the possibility to query the local KG in tests, the test queries will no longer be edited by adding a SERVICE clause if there are none.

This change may break some rulesets.

v2.2 Working with script

• Reworking of the chaining of thread for the pre/main/post step of the indexing.
• Addition of multiple new scripts used for the treatment of linked data resources.

Full Changelog: v2.1...v2.2

Resilience

IndeGx now integrates the possibility to:

• separate pre, post, and main processing
• handle many different errors in the RDF format of data found in remote endpoints
• disable the query logging of the framework
• use the query logging of the framework to avoid repeating rule application in case of an execution interruption

We also offer two automatically refreshed catalogs,

• the catalog of endpoints taken from numerous sources, updated daily
• The catalog of endpoints and their statuses, refreshed hourly

This release also contains numerous fixes.

Pre-release version

This is a new reimplementation of the IndeGx application from the previous repository, as a Docker image containing an engine written in typescript instead of Java and a Corese instance.

Differences:

• The queries used by the framework have to specify what is sent to the remote endpoint with a SERVICE clause.
• All queries used in the actions must be UPDATE queries.
• Endpoints are treated in parallel.
• Instead of "one graph per indexation and catalog", we now use a structure with 4 graphs per indexation and endpoint, with an original graph, a curated graph, a summary graph, and a metadata graph.
• The Corese instance is used to send all queries, making the framework globally more resistant to bad standard support from the endpoints

New features:

• Using kgi:recommendedPagination, one can make complex queries paginate over the dataset. However, it can only work if the query does NOT insert new triples when the SERVICE clause of the query does not generate new bindings.
• Manifests can be used as actions
• A manifest can be used as post-treatment and apply rules on the descriptions generated previously
• The new post-treatment rules include the application of equivalences between properties of the most common vocabularies, the detection of common elements between endpoints
• New HibiSCus-style rules to generate a summary of the structure of a dataset (i.e. ).
• The summary is used in post-treatment to consolidate the basic statistics about properties and hostnames.