+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Access Control Lists
Access Control Lists are rooted in the /v1/acls
collection.
diff --git a/contexts/aggregations.json b/contexts/aggregations.json new file mode 100644 index 0000000000..1f8226c417 --- /dev/null +++ b/contexts/aggregations.json @@ -0,0 +1,9 @@ +{ + "@context": { + "@vocab": "https://bluebrain.github.io/nexus/vocabulary/" + }, + "@id": "https://bluebrain.github.io/nexus/contexts/aggregations.json", + "aggregations": { + "@container": "@list" + } +} \ No newline at end of file diff --git a/contexts/composite-views.json b/contexts/composite-views.json index cbf0e46e0f..4ffb0bd1a4 100644 --- a/contexts/composite-views.json +++ b/contexts/composite-views.json @@ -15,10 +15,10 @@ "@container": "@set" }, "sources": { - "@container": "@set" + "@container": "@list" }, "projections": { - "@container": "@set" + "@container": "@list" }, "_uuid": "https://bluebrain.github.io/nexus/vocabulary/uuid" }, diff --git a/contexts/elasticsearch-metadata.json b/contexts/elasticsearch-metadata.json index 7bbc67f7b9..a344a8863a 100644 --- a/contexts/elasticsearch-metadata.json +++ b/contexts/elasticsearch-metadata.json @@ -3,8 +3,7 @@ "View": "https://bluebrain.github.io/nexus/vocabulary/View", "ElasticSearchView": "https://bluebrain.github.io/nexus/vocabulary/ElasticSearchView", "AggregateElasticSearchView": "https://bluebrain.github.io/nexus/vocabulary/AggregateElasticSearchView", - "_uuid": "https://bluebrain.github.io/nexus/vocabulary/uuid", - "_indexingRev": "https://bluebrain.github.io/nexus/vocabulary/indexingRev" + "_uuid": "https://bluebrain.github.io/nexus/vocabulary/uuid" }, "@id": "https://bluebrain.github.io/nexus/contexts/elasticsearch-metadata.json" } diff --git a/contexts/elasticsearch.json b/contexts/elasticsearch.json index 8b377b72e4..8a27561e57 100644 --- a/contexts/elasticsearch.json +++ b/contexts/elasticsearch.json @@ -18,8 +18,7 @@ "viewId": { "@type": "@id" }, - "_uuid": "https://bluebrain.github.io/nexus/vocabulary/uuid", - "_indexingRev": "https://bluebrain.github.io/nexus/vocabulary/indexingRev" + "_uuid": "https://bluebrain.github.io/nexus/vocabulary/uuid" }, "https://bluebrain.github.io/nexus/contexts/pipeline.json" ], diff --git a/contexts/metadata.json b/contexts/metadata.json index e3193c6aad..f76fa81274 100644 --- a/contexts/metadata.json +++ b/contexts/metadata.json @@ -1,6 +1,10 @@ { "@context": { "_rev": "https://bluebrain.github.io/nexus/vocabulary/rev", + "_tags": { + "@id": "https://bluebrain.github.io/nexus/vocabulary/tags", + "@container": "@set" + }, "_deprecated": "https://bluebrain.github.io/nexus/vocabulary/deprecated", "_createdAt": { "@id": "https://bluebrain.github.io/nexus/vocabulary/createdAt", diff --git a/contexts/remote-contexts.json b/contexts/remote-contexts.json new file mode 100644 index 0000000000..641de69e4f --- /dev/null +++ b/contexts/remote-contexts.json @@ -0,0 +1,9 @@ +{ + "@context": { + "@vocab": "https://bluebrain.github.io/nexus/vocabulary/", + "remoteContexts": { + "@container": "@set" + } + }, + "@id": "https://bluebrain.github.io/nexus/contexts/remote-contexts.json" +} \ No newline at end of file diff --git a/contexts/resolvers.json b/contexts/resolvers.json index ed45ffd5cb..c2dd3b633a 100644 --- a/contexts/resolvers.json +++ b/contexts/resolvers.json @@ -13,6 +13,12 @@ "identities": { "@container": "@set" }, + "history": { + "@container": "@list" + }, + "rejections": { + "@container": "@list" + }, "subject": "https://bluebrain.github.io/nexus/vocabulary/subject", "_resolverId": { "@id": "https://bluebrain.github.io/nexus/vocabulary/resolverId", diff --git a/docs/delta/api/acls-api.html b/docs/delta/api/acls-api.html index 260b3faaea..27579ff984 100644 --- a/docs/delta/api/acls-api.html +++ b/docs/delta/api/acls-api.html @@ -154,6 +154,8 @@
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Access Control Lists are rooted in the /v1/acls
collection.
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+An archive is a collection of resources stored inside an archive file. The archiving format chosen for this purpose is tar (or tarball). Archive resources are rooted in the /v1/archives/{org_label}/{project_label}/
collection.
An archive is a collection of resources stored inside an archive file. The archiving format chosen for this purpose is ZIP. Archive resources are rooted in the /v1/archives/{org_label}/{project_label}/
collection.
Each archive…
project
identifier by the label {project_label}
organization
identifier by the label {org_label}
When modifying archives, the caller must have archives/write
permissions on the current path of the project or the ancestor paths.
When reading archives, the caller must have resources/read
permissions on the current path of the project or the ancestor paths.
For both reading and modifying archives, the caller must have resources/read
permissions on the current path of the project or the ancestor paths.
Please visit Authentication & authorization section to learn more about it.
Contrarily to the rest of the platform resources, archives are not persisted resources, given their nature. Therefore there are no update, tag or deprecation operations available on archive resources.
@@ -318,6 +324,11 @@where…
{resource_id}
: Iri - the @id value of the resource to be added to the archive.{file_self}
: Uri - the _self
value of the file to be added to the archive. This is the same as the http location of the file in the Delta API.{project}
: String - the project (in the format ‘myorg/myproject’) where the specified resource belongs. This field is optional. It defaults to the current project.{path}
: Path - the relative path on the archive where this resource is going to stored
In order to decide whether we want to select a resource or a file, the @type
discriminator is used with the following possibilities:
Resource
: targets a resourceFile
: targets a fileFile
: targets a file using its project and idFileSelf
: targets a file using its address (_self
)This endpoint is used to describe the archive and to subsequently consume it.
@@ -364,10 +377,10 @@@id
value is found on the payload, this @id will be used.
@id
value is not found on the payload, an @id will be generated as follows: base:{UUID}
. The base
is the prefix
defined on the resource’s project ({project_label}
).The response will be an HTTP 303 Location redirect, which will point to the url where to consume the archive (tarball).
+The response will be an HTTP 303 Location redirect, which will point to the url where to consume the archive (ZIP).
The following diagram can help to understand the HTTP exchange
Example
-The following example shows how to create an archive containing 3 files. 2 of them are resources and the other is a file. As a response, the tarball will be offered.
+The following example shows how to create an archive containing 3 files. 2 of them are resources and the other is a file. As a response, the ZIP file will be offered.
Note that if the payload contains an @id different from the {archive_id}
, the request will fail.
When fetching an archive, the response format can be chosen through HTTP content negotiation. In order to fetch the archive metadata, the client can use any of the following MIME types. However, in order to fetch the archive content, the HTTP Accept
header should be provided:
*/*
or application/x-tar
will return a tar archive (or tarball)application/zip
will return a zip archiveThe limitations of the tar format makes the usage of archives difficult (among other things, the maximum file name is limited to 100 characters), so its support will be removed in a future release.
When fetching an archive, the response format can be chosen through HTTP content negotiation. In order to fetch the archive metadata, the client can use any of the following MIME types. However, in order to fetch the archive content, the HTTP Accept
header should be provided as application/zip
.
@@@
When downloading the archive, it is possible to ignore resources that does not exist by appending the query parameter ignoreNotFound=true
to the fetch link.
GET /v1/archives/{org_label}/{project_label}/{archive_id}
GET /v1/archives/{org_label}/{project_label}/{archive_id}?ignoreNotFound=true
Example
sourcecurl "http://localhost:8080/v1/archives/myorg/myproject/myarchive" \
- -H "Accept: application/x-tar" \
- -o output.tar
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Authentication is the process of validating that users are who they claim to be while authorization gives those users permission to access an API resource.
@@ -299,7 +306,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+When performing a request against Nexus Delta, clients can specify the desired format of the response they would like to receive. This is done through a mechanism called Content Negotiation. Nexus Delta uses the HTTP Accept Header in order to provide Content Negotiation capabilities.
@@ -298,7 +305,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Nexus Delta makes use of the HTTP status codes to report the outcome of each API call. The status codes are complemented by a consistent response data model for reporting client and system level failures.
@@ -281,7 +288,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Nexus provides a global events endpoint, which allows the users to access the stream of events for all the resources in Nexus, including ACLs, permissions, realms, etc.
@@ -330,7 +337,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Files are attachment resources rooted in the /v1/files/{org_label}/{project_label}/
collection.
Each file belongs to a project
identifier by the label {project_label}
inside an organization
identifier by the label {org_label}
.
When creating, updating and reading files, the caller must have the permissions defined on the storage associated to the file on the current path of the project or the ancestor paths.
Please visit Authentication & authorization section to learn more about it.
When using the endpoints described on this page, the responses will contain global metadata described on the Nexus Metadata page. In addition, the following files specific metadata can be present
+_bytes
: size of the file in bytes_digest
: algorithm and checksum used for file integrity_filename
: name of the file_location
: path where the file is stored on the underlying storage_mediaType
: MIME specifying the type of the file_origin
: whether the file attributes resulted from an action taken by a client or the Nexus Storage Service_storage
: @id
, @type
, and revision of the Storage used for the file_uuid
: UUID of the file_project
: address of the file’s project_incoming
: address to query to obtain the list of incoming links_outgoing
: address to query to obtain the list of outgoing linksAll the API calls modifying a file (creation, update, tagging, deprecation) can specify whether the file should be indexed synchronously or in the background. This behaviour is controlled using indexing
query param, which can be one of two values:
sync
- the file will be indexed synchronously and the API call won’t return until the indexing is finished
POST /v1/files/{org_label}/{project_label}?storage={storageId}
+POST /v1/files/{org_label}/{project_label}?storage={storageId}&tag={tagName}
-… where {storageId}
selects a specific storage backend where the file will be uploaded. This field is optional. When not specified, the default storage of the project is used.
+… where - {storageId}
selects a specific storage backend where the file will be uploaded. This field is optional. When not specified, the default storage of the project is used. - {tagName}
an optional label given to the file on its first revision.
The json payload:
- If the
@id
value is found on the payload, this @id will be used.
@@ -377,9 +403,9 @@
Create using PUT
This alternative endpoint to create a resource is useful in case the json payload does not contain an @id
but you want to specify one. The @id will be specified in the last segment of the endpoint URI.
-PUT /v1/files/{org_label}/{project_label}/{file_id}?storage={storageId}
+PUT /v1/files/{org_label}/{project_label}/{file_id}?storage={storageId}&tag={tagName}
-… where {storageId}
selects a specific storage backend where the file will be uploaded. This field is optional. When not specified, the default storage of the project is used.
+… where - {storageId}
selects a specific storage backend where the file will be uploaded. This field is optional. When not specified, the default storage of the project is used. - {tagName}
an optional label given to the file on its first revision.
Note that if the payload contains an @id different from the {file_id}
, the request will fail.
Example
- Response
-
source{
@@ -427,7 +453,7 @@
Link using POST
Brings a file existing in a storage to Nexus Delta as a file resource. This operation is supported for files using S3Storage
and RemoteDiskStorage
.
-POST /v1/files/{org_label}/{project_label}?storage={storageId}
+POST /v1/files/{org_label}/{project_label}?storage={storageId}&tag={tagName}
{
"path": "{path}",
"filename": "{filename}",
@@ -437,8 +463,9 @@
{storageId}
: String - Selects a specific storage backend that supports linking existing files. This field is optional. When not specified, the default storage of the project is used.
{path}
: String - the relative location (from the point of view of storage folder) on the remote storage where the file exists.
- {filenane}
: String - the name that will be given to the file during linking. This field is optional. When not specified, the original filename is retained.
+ {filename}
: String - the name that will be given to the file during linking. This field is optional. When not specified, the original filename is retained.
{mediaType}
: String - the MediaType fo the file. This field is optional. When not specified, Nexus Delta will attempt to detectput
+ {tagName}
an optional label given to the linked file resource on its first revision.
Example
@@ -498,7 +525,7 @@ Link using PUT
Brings a file existing in a storage to Nexus Delta as a file resource. This operation is supported for files using S3Storage
and RemoteDiskStorage
.
This alternative endpoint allows to specify the resource @id
.
-PUT /v1/files/{org_label}/{project_label}/{file_id}?storage={storageId}
+PUT /v1/files/{org_label}/{project_label}/{file_id}?storage={storageId}&tag={tagName}
{
"path": "{path}",
"filename": "{filename}",
@@ -509,8 +536,9 @@ sourcecurl -X PUT \
-H "Content-Type: application/json" \
- "http://localhost:8080/v1/files/myorg/myproject/mylink?storage=remote" -d \
+ "http://localhost:8080/v1/files/myorg/myproject/mylink?storage=remote&tag=mytag" -d \
'{
"path": "relative/path/to/myfile2.png"
}'
@@ -734,7 +762,7 @@ Deprecate
Locks the file, so no further operations can be performed.
Deprecating a file is considered to be an update as well.
-DELETE /v1/files/{org_label}/{project_label}?rev={previous_rev}
+DELETE /v1/files/{org_label}/{project_label}/{file_id}?rev={previous_rev}
… where {previous_rev}
is the last known revision number for the file.
Example
@@ -779,6 +807,54 @@ Undeprecate
+Unlocks a previously deprecated file. Further operations can then be performed. The file will again be found when listing/querying.
+Undeprecating a file is considered to be an update as well.
+PUT /v1/file/{org_label}/{project_label}/{file_id}/undeprecate?rev={previous_rev}
+
+… where {previous_rev}
is the last known revision number for the resource.
+Example
+
+ - Request
+ -
+
sourcecurl -X PUT \
+ "http://localhost:8080/v1/files/myorg/myproject/myfile/undeprecate?rev=4"
+ - Response
+ -
+
source{
+ "@context": [
+ "https://bluebrain.github.io/nexus/contexts/files.json",
+ "https://bluebrain.github.io/nexus/contexts/metadata.json"
+ ],
+ "@id": "http://localhost:8080/v1/resources/myorg/myproject/_/myfile",
+ "@type": "File",
+ "_bytes": 13896460,
+ "_constrainedBy": "https://bluebrain.github.io/nexus/schemas/files.json",
+ "_createdAt": "2021-05-12T07:30:54.576Z",
+ "_createdBy": "http://localhost:8080/v1/anonymous",
+ "_deprecated": false,
+ "_digest": {
+ "_algorithm": "SHA-256",
+ "_value": "4c9f4292e3c0c5fc23cd60722adb8a1535f1dd7f0cf9203140d61fb889eef3cf"
+ },
+ "_filename": "myfile2.pdf",
+ "_incoming": "http://localhost:8080/v1/files/myorg/myproject/myfile/incoming",
+ "_mediaType": "application/pdf",
+ "_origin": "Client",
+ "_outgoing": "http://localhost:8080/v1/files/myorg/myproject/myfile/outgoing",
+ "_project": "http://localhost:8080/v1/projects/myorg/myproject",
+ "_rev": 5,
+ "_self": "http://localhost:8080/v1/files/myorg/myproject/myfile",
+ "_storage": {
+ "@id": "https://bluebrain.github.io/nexus/vocabulary/diskStorageDefault",
+ "@type": "DiskStorage",
+ "_rev": 1
+ },
+ "_updatedAt": "2021-05-12T08:09:23.658Z",
+ "_updatedBy": "http://localhost:8080/v1/anonymous",
+ "_uuid": "3e86d93a-c196-407d-a13c-cea7168e32e3"
+}
+
Fetch
When fetching a file, the response format can be chosen through HTTP content negotiation. In order to fetch the file metadata, the client can use any of the following MIME types. However, in order to fetch the file content, the HTTP Accept
header */*
(or any MIME type that matches the file MediaType) should be provided.
GET /v1/files/{org_label}/{project_label}/{file_id}?rev={rev}&tag={tag}
@@ -879,6 +955,7 @@
Within an organization
This operation returns only files from projects defined in the organisation {org_label}
and where the caller has the resources/read
permission.
@@ -891,6 +968,7 @@ Within all projects
This operation returns only files from projects where the caller has the resources/read
permission.
@@ -903,6 +981,7 @@
Parameter description
@@ -915,6 +994,7 @@ {updatedBy}
: Iri - can be used to filter the resulting files based on the person which performed the last update
{search}
: String - can be provided to select only the files in the collection that have attribute values matching (containing) the provided string
{sort}
: String - can be used to sort files based on a payloads’ field. This parameter can appear multiple times to enable sorting by multiple fields. The default is done by _createdBy
and @id
.
+ {aggregations}
: Boolean - if true
then the response will only contain aggregations of the @type
and _project
fields; defaults to false
. See Aggregations.
Example
@@ -1077,7 +1157,7 @@
-v1.8.x
+Snapshot
diff --git a/docs/delta/api/graph-analytics-api.html b/docs/delta/api/graph-analytics-api.html
index b98d1cd0ee..4954e33df4 100644
--- a/docs/delta/api/graph-analytics-api.html
+++ b/docs/delta/api/graph-analytics-api.html
@@ -154,6 +154,8 @@
- Releases
+ - v1.9 Release Notes
+ - v1.8 To v1.9 Migration
- v1.8 Release Notes
- v1.7 To v1.8 Migration
- v1.7 Release Notes
@@ -162,14 +164,7 @@
- v1.5 To v1.6 Migration
- v1.5 Release Notes
- v1.4 To v1.5 Migration
- - v1.4 Release Notes
- - v1.3 To v1.4 Migration
- - v1.3 Release Notes
- - v1.2 To v1.3 Migration
- - v1.2 Release Notes
- - v1.1 Release Notes
- - v1.0 To v1.1 Migration
- - v1.0 Release Notes
+ - Older releases
- Roadmap
- Nexus Fusion
@@ -202,11 +197,15 @@
- Permissions
- Realms
- Access Control Lists
+ - User Permissions
- Organizations
- Projects
- Quotas
- Schemas
- Resources
+ - Trial
+ - Multi fetch
+ - Id Resolution
- Resolvers
- Views
@@ -226,6 +225,7 @@
- Global events
- Supervision
+ - Nexus Metadata
- Plugins
- Benchmarks
@@ -244,6 +244,7 @@
- Fetch relationships
- Fetch properties
- Fetch progress
+ - Search
- Internals
- Document structure
@@ -256,7 +257,7 @@
@@ -274,6 +275,7 @@
- Fetch relationships
- Fetch properties
- Fetch progress
+ - Search
- Internals
- Document structure
@@ -287,6 +289,13 @@
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Graph analytics is a feature introduced by the graph-analytics
plugin and rooted in the /v1/graph-analytics/{org_label}/{project_label}
collection.
POST /v1/graph-analytics/{org_label}/{project_label}/_search
+ {...}
+
+Search documents that are in a given project’s Graph Analytics view.
+The supported payload is defined on the ElasticSearch documentation.
+Example
+sourcecurl -XPOST \
+-H "Content-Type: application/json" \
+"http://localhost:8080/v1/graph-analytics/myorg/myproj/_search" -d \
+'{
+ "query": {
+ "term": {
+ "@id": "https://example.com/person"
+ }
+ }
+}'
source{
+ "hits": {
+ "hits": [
+ {
+ "_id": "http://example.com/person",
+ "_index": "delta_ga_myorg_myproj",
+ "_score": 1.3121864,
+ "_source": {
+ "@id": "http://example.com/person",
+ "@type": "http://schema.org/Person",
+ "properties": [
+ {
+ "@id": "http://example.com/epfl",
+ "dataType": "object",
+ "isInArray": false,
+ "path": "http://schema.org/worksFor"
+ }
+ ],
+ "references": [
+ {
+ "@id": "http://example.com/epfl",
+ "@type": [
+ "http://schema.org/EducationalOrganization"
+ ],
+ "dataType": "object",
+ "found": true,
+ "isInArray": false,
+ "path": "http://schema.org/worksFor"
+ }
+ ],
+ "relationships": [
+ {
+ "@id": "http://example.com/epfl",
+ "@type": [
+ "http://schema.org/EducationalOrganization"
+ ],
+ "dataType": "object",
+ "isInArray": false,
+ "path": "http://schema.org/worksFor"
+ }
+ ],
+ "remoteContexts": [
+ {
+ "@type": "ProjectRemoteContextRef",
+ "iri": "https://bbp.epfl.ch/contexts/person",
+ "resource": {
+ "id": "https://bbp.epfl.ch/contexts/person",
+ "project": "myorg/myproj",
+ "rev": 1
+ }
+ }
+ ],
+ "_createdAt": "2023-08-08T15:49:14.081Z",
+ "_createdBy": {
+ "@type": "User",
+ "realm": "internal",
+ "subject": "delta"
+ },
+ "_deprecated": false,
+ "_project": "myorg/myproj",
+ "_rev": 1,
+ "_updatedAt": "2023-08-08T15:49:14.081Z",
+ "_updatedBy": {
+ "@type": "User",
+ "realm": "internal",
+ "subject": "delta"
+ }
+ }
+ }
+ ],
+ "max_score": 1.3121864,
+ "total": {
+ "relation": "eq",
+ "value": 1
+ }
+ },
+ "timed_out": false,
+ "took": 0,
+ "_shards": {
+ "failed": 0,
+ "skipped": 0,
+ "successful": 1,
+ "total": 1
+ }
+}
In order to implement the described endpoints we needed a way to transform our data so that it would answer the desired questions in a performant manner.
The proposed solution was to stream our data, transform it and push it to a dedicated ElasticSearch index (one index per project). Then at query time we can run term aggregations in order to get the desired counts.
@@ -519,6 +636,17 @@isInArray
- Boolean: Flag to inform whether the current path (or its parent) is part of an array.
dataType
- String: The type of the value present in the current path. Possible values are: string, numeric and booleanfound
- Boolean: Flag to inform whether an @id inside references
has been resolved as a relationship.remoteContexts
- Json Object Array: A collection of remote contexts detected during the JSON-LD resolution for this resource. See the Resources - Fetch remote contexts operation to learn about the remote context types.+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Id Resolution allows to resolve a resource by providing only a resource identifier (the @id
value of a resource). In case there are multiple resources with the same identifier across different projects, the response provides all choices for disambiguation.
When performing a request, the caller must have resources/read
permission on the project each resource belongs to.
Please visit Authentication & authorization section to learn more about it.
GET /v1/resolve/{id}
+
+where…
+{id}
: the identifier of the resource to resolve (URL encoded value).The following example shows how to perform a request and possible responses. If the provided identifier can be resolved uniquely, the response is identical to that of the resource fetch operation. In case there are multiple choices, the response is that of a listing operation that filters for the resource identifier.
+sourcecurl "http://localhost:8080/v1/resolve/https%3A%2F%2Fexample.com%2FAlice"
source{
+ "@context": [
+ "https://bluebrain.github.io/nexus/contexts/metadata.json",
+ {
+ "@vocab": "https://schema.org/"
+ }
+ ],
+ "@id": "https://example.com/Alice",
+ "@type": "Person",
+ "name": "Alice",
+ "age": 42,
+ "_constrainedBy" : "https://bluebrain.github.io/nexus/schemas/unconstrained.json",
+ "_createdAt" : "1970-01-01T00:00:00Z",
+ "_createdBy" : "http://localhost/v1/anonymous",
+ "_deprecated" : false,
+ "_incoming" : "http://localhost/v1/resources/myorg/myproject/_/https%3A%2F%2Fexample.com%2FAlice/incoming",
+ "_outgoing" : "http://localhost/v1/resources/myorg/myproject/_/https%3A%2F%2Fexample.com%2FAlice/outgoing",
+ "_project" : "http://localhost/v1/projects/myorg/myproject",
+ "_rev" : 1,
+ "_schemaProject" : "http://localhost/v1/projects/myorg/myproject",
+ "_self" : "http://localhost/v1/resources/myorg/myproject/_/https%3A%2F%2Fexample.com%2FAlice",
+ "_updatedAt" : "1970-01-01T00:00:00Z",
+ "_updatedBy" : "http://localhost/v1/anonymous"
+}
source{
+ "@context" : [
+ "https://bluebrain.github.io/nexus/contexts/metadata.json",
+ "https://bluebrain.github.io/nexus/contexts/search.json"
+ ],
+ "_total" : 2,
+ "_results" : [
+ {
+ "@id" : "https://example.com/Alice",
+ "_constrainedBy" : "https://bluebrain.github.io/nexus/schemas/unconstrained.json",
+ "_createdAt" : "2023-09-22T13:55:33.493Z",
+ "_createdBy" : "http://localhost:8080/v1/realms/myrealm/users/bob",
+ "_deprecated" : false,
+ "_incoming" : "http://localhost:8080/v1/resources/myorg/myproject/_/https:%2F%2Fexample.com%2FAlice/incoming",
+ "_outgoing" : "http://localhost:8080/v1/resources/myorg/myproject/_/https:%2F%2Fexample.com%2FAlice/outgoing",
+ "_project" : "http://localhost:8080/v1/projects/myorg/myproject",
+ "_rev" : 1,
+ "_schemaProject" : "http://localhost:8080/v1/projects/myorg/myproject",
+ "_score" : 0.0,
+ "_self" : "http://localhost:8080/v1/resources/myorg/myproject/_/https:%2F%2Fexample.com%2FAlice",
+ "_updatedAt" : "2023-09-22T13:55:33.493Z",
+ "_updatedBy" : "http://localhost:8080/v1/realms/myrealm/users/bob"
+ },
+ {
+ "@id" : "https://example.com/Alice",
+ "_constrainedBy" : "https://bluebrain.github.io/nexus/schemas/unconstrained.json",
+ "_createdAt" : "2023-09-22T13:55:33.459Z",
+ "_createdBy" : "http://localhost:8080/v1/realms/myrealm/users/bob",
+ "_deprecated" : false,
+ "_incoming" : "http://localhost:8080/v1/resources/myorg/secondproject/_/https:%2F%2Fexample.com%2FAlice/incoming",
+ "_outgoing" : "http://localhost:8080/v1/resources/myorg/secondproject/_/https:%2F%2Fexample.com%2FAlice/outgoing",
+ "_project" : "http://localhost:8080/v1/projects/myorg/secondproject",
+ "_rev" : 1,
+ "_schemaProject" : "http://localhost:8080/v1/projects/myorg/secondproject",
+ "_score" : 0.0,
+ "_self" : "http://localhost:8080/v1/resources/myorg/secondproject/_/https:%2F%2Fexample.com%2FAlice",
+ "_updatedAt" : "2023-09-22T13:55:33.459Z",
+ "_updatedBy" : "http://localhost:8080/v1/realms/myrealm/users/bob"
+ }
+ ],
+ "_maxScore" : 0.0
+}
This endpoint is designed for the Nexus deployment at the Blue Brain Project and as such might not suit the needs of other deployments.
The resolve endpoint offers resource resolution on the resources the caller has access to. As such, if the client is a browser and does not have the ability to include an authorization header in the request, it is possible to use the proxy pass version of the resolve endpoint which will lead the client to a Nexus Fusion authentication page. This will allow to “inject” the user’s token in a subsequent resolve request made by Nexus Fusion.
+app.fusion.base
: String - base URL for Nexus Fusionapp.fusion.enable-redirects
: Boolean - needs to be true
in order for redirects to work (defaults to false
)app.fusion.resolve-base
: String - base URL to use when reconstructing the resource identifier in the proxy pass endpoint/v1/resolve-proxy-pass/{segment}
{resourceId}
= {resolveBase}/{segment}
{fusionBaseUrl}/resolve/{resourceId}
Fusion endpoint if the Accept: text/html
header is present/v1/resolve/{resourceId}
Delta endpoint otherwiseThe Nexus Fusion resolve page allows the user to authenticate (if they are not already authenticated) and will perform a call to the Nexus Delta /v1/resolve/{resourceId}
with the user’s authentication token.
All calls to the /v1/resolve-proxy-pass
endpoint lead to 303 See Other responses.
The example below assumes that:
+{fusionBaseUrl}
= http://localhost:8080/fusion
{segment}
= nexus/data/identifier
{resolveBase}
= https://example.com
sourcecurl "http://localhost:8080/v1/resolve-proxy-pass/nexus/data/identifier"
Acccept:text/html
is provided)
+ source303 See Other
+The response to the request can be found under <a href="http://localhost:8080/fusion/resolve/https:%2F%2Fexample.com%2Fnexus%2Fdata%2Fidentifier">this URI</a> using a
+GET method.
Accept:text/html
provided)
+ source303 See Other
+The response to the request can be found under <a href="https://localhost:8080/v1/resolve/https:%2F%2Fexample.com%2Fnexus%2Fdata%2Fidentifier">this URI</a> using a
+GET method.
In your networking setup, if a proxy pass is enabled to map https://example.com/nexus/data/*
to https://localhost:8080/v1/resolve-proxy-pass/nexus/data/*
, the proxy pass allows to de facto resolve resource with identifier of the type https://example.com/nexus/data/*
by simply querying their @id
.
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+The /v1/identities
endpoint allows user to retrieve the identities that the user has in the platform. Calling the endpoint without a token will result in only one identity returned: Anonymous
. Calling it with token should return multiple identities. There are four different types of identities:
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Nexus Delta exposes a RESTful interface over HTTP(S) for synchronous communication. The generally adopted transport format is JSON based, specifically JSON-LD. However, other response formats are supported through Content-Negotiation
@@ -314,7 +323,7 @@A realm provides with the necessary information to perform authentication against a certain OIDC provider .
+A realm provides with the necessary information to perform authentication against a certain OIDC provider .
In order to restrict applications’ access to data by placing restrictions on them, three parameters are important:
@@ -325,6 +334,9 @@An ACL defines the set of permissions that certain identities have on a concrete path.
+ACLs are the permission model used in Delta. Sometimes it can be more convenient to ask more basic questions about whether a user has a permission, rather than trying to determine this yourself from the ACLs API. This is what the user permissions API is for
+Operations on User Permissions
The top-level grouping resource in the platform, called organization
@@ -335,7 +347,7 @@A schema is a resource which defines a set of rules and constrains using SHACL.
+A schema is a resource which defines a set of rules and constrains using SHACL.
A resource is the most generic entity on the Knowledge Graph. Resources can be schemas
, resolvers
, views
, storages
, files
or data
.
An archive is a collection of resources stored inside an archive file. The archiving format chosen for this purpose is tar (or tarball).
+An archive is a collection of resources stored inside an archive file. The archiving format chosen for this purpose is ZIP file.
Nexus Delta is build using the event sourcing approach. This strategy captures all changes to an application state as a sequence of events.
@@ -380,7 +392,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+The Jira integration plugin comes from the need at BBP to have the Nexus platform and Jira to interact by linking Nexus resources and Jira issues.
@@ -347,7 +354,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+The multi-fetch operation allows to get in a given format multiple resources that can live in multiple projects.
+The response includes a resources array that contains the resources in the order specified in the request. The structure of the returned resources is similar to that returned by the fetch API. If there is a failure getting a particular resource, the error is included in place of the resource.
+This operation can be used to return every type of resource.
When performing a request, the caller must have resources/read
permission on the project each resource belongs to.
Please visit Authentication & authorization section to learn more about it.
GET|POST /v1/multi-fetch/resources
+
+{
+ "format": {format}
+ "resources": [
+ {
+ "id": "{id}",
+ "project": "{project}"
+ },
+ ...
+ ]
+}
+
+where…
+{format}
: String - the format we expect for the resources in the response. Accepts the following values: source (to get the original payload), annotated-source (to get the original payload with metadata), compacted, expanded, n-triples, dot{project}
: String - the project (in the format ‘myorg/myproject’) where the specified resource belongs. This field is optional. It defaults to the current project.{id}
: Iri - the @id value of the resource to be returned. Can contain a tag or a revision.The following example shows how to perform a multi-fetch and an example of response containing errors (missing permissions and resource not found). As a response, a regular json is returned containing the different resources in the requested format.
+sourcecurl -L \
+ -X GET \
+ -d '
+ {
+ "format": "source",
+ "resources" : [
+ {
+ "id": "https://bbp.epfl.ch/person/alex",
+ "project": "public/person"
+ },
+ {
+ "id": "https://bbp.epfl.ch/person/john-doe",
+ "project": "public/person"
+ },
+ {
+ "id": "https://bbp.epfl.ch/secret/xxx",
+ "project": "restricted/xxx"
+ }
+ ]
+ }
+'
source{
+ "format": "source",
+ "resources" : [
+ {
+ "id": "https://bbp.epfl.ch/person/alex",
+ "project": "public/person"
+ },
+ {
+ "id": "https://bbp.epfl.ch/person/john-doe",
+ "project": "public/person"
+ },
+ {
+ "id": "https://bbp.epfl.ch/secret/xxx",
+ "project": "restricted/xxx"
+ }
+ ]
+}
source{
+ "format": "source",
+ "resources": [
+ {
+ "@id": "https://bbp.epfl.ch/person/alex",
+ "project": "public/person",
+ "value": {
+ "@context": {
+ "@vocab": "https://bluebrain.github.io/nexus/vocabulary/"
+ },
+ "@id": "https://bluebrain.github.io/nexus/vocabulary/success",
+ "@type": "Person",
+ "bool": false,
+ "name": "Alex",
+ "number": 24
+ }
+ },
+ {
+ "@id": "https://bbp.epfl.ch/person/john-doe",
+ "project": "public/person",
+ "error": {
+ "@type": "NotFound",
+ "reason": "The resource 'https://bbp.epfl.ch/person/john-doe' was not found in project 'public/person'."
+ }
+ },
+ {
+ "@id": "https://bbp.epfl.ch/secret/xxx",
+ "project": "restricted/xxx",
+ "error": {
+ "@type": "AuthorizationFailed",
+ "reason": "The supplied authentication is not authorized to access this resource."
+ }
+ }
+ ]
+}
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Organizations are rooted in the /v1/orgs
path and are used to group and categorize sub-resources.
Reverses the deprecation of an organisation.
+Undeprecating an organization is considered to be an update as well.
+PUT /v1/orgs/{label}/undeprecate?rev={previous_rev}
+
+… where
+{label}
: String - is the user friendly name that identifies this organization.{previous_rev}
: Number - is the last known revision for the organization.Example
+sourcecurl -X PUT \
+ "http://localhost:8080/v1/orgs/myorg/undeprecate?rev=2"
source{
+ "@context": [
+ "https://bluebrain.github.io/nexus/contexts/organizations-metadata.json",
+ "https://bluebrain.github.io/nexus/contexts/metadata.json"
+ ],
+ "@id": "http://localhost:8080/v1/orgs/myorg",
+ "@type": "Organization",
+ "_constrainedBy": "https://bluebrain.github.io/nexus/schemas/organizations.json",
+ "_createdAt": "2021-05-10T13:31:24.223Z",
+ "_createdBy": "http://localhost:8080/v1/anonymous",
+ "_deprecated": false,
+ "_label": "myorg",
+ "_rev": 3,
+ "_self": "http://localhost:8080/v1/orgs/myorg",
+ "_updatedAt": "2021-05-10T13:32:31.197Z",
+ "_updatedBy": "http://localhost:8080/v1/anonymous",
+ "_uuid": "2393f414-8fff-4e79-b622-d7a887649952"
+}
Permanently delete an organization containing no projects. If there is a project, returns 409 Conflict.
+The caller must have organizations/delete
permissions.
DELETE /v1/orgs/{label}?prune=true
+
+… where
+{label}
: String - is the user friendly name that identifies this organization.prune
: a flag to make permanent deletion explicit. To avoid mistakes, if prune=true
then rev
cannot be present and a 400 Bad Request will be returned.Example
+sourcecurl -X DELETE \
+ "http://localhost:8080/v1/orgs/myorg?prune=true"
GET /v1/orgs/{label}
@@ -573,7 +638,7 @@ +You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Permissions are rooted in the /v1/permissions
collection.
storages/write
- default permissions for archives
-archives/write
default permissions for the version endpoint
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Projects belong to an organization
and are rooted in the corresponding /v1/projects/{org_label}
path. The purposes of projects are:
Reverses the deprecation of a project.
+Undeprecating a project is considered to be an update as well.
+PUT /v1/projects/{org_label}/{label}/undeprecate?rev={previous_rev}
+
+… where
+{previous_rev}
: Number - the last known revision for the organization.{label}
: String - the user friendly name that identifies this project.Example
+sourcecurl -X PUT \
+ "http://localhost:8080/v1/projects/myorg/myproject/undeprecate?rev=2"
source{
+ "@context": [
+ "https://bluebrain.github.io/nexus/contexts/projects-metadata.json",
+ "https://bluebrain.github.io/nexus/contexts/metadata.json"
+ ],
+ "@id": "http://localhost:8080/v1/projects/myorg/myproject",
+ "@type": "Project",
+ "_constrainedBy": "https://bluebrain.github.io/nexus/schemas/projects.json",
+ "_createdAt": "2021-05-10T15:43:36.567Z",
+ "_createdBy": "http://localhost:8080/v1/anonymous",
+ "_deprecated": false,
+ "_markedForDeletion": false,
+ "_effectiveApiMappings": [
+ {
+ "_namespace": "https://bluebrain.github.io/nexus/vocabulary/",
+ "_prefix": "nxv"
+ },
+ {
+ "_namespace": "https://bluebrain.github.io/nexus/vocabulary/defaultElasticSearchIndex",
+ "_prefix": "documents"
+ },
+ {
+ "_namespace": "https://bluebrain.github.io/nexus/vocabulary/defaultInProject",
+ "_prefix": "defaultResolver"
+ },
+ {
+ "_namespace": "https://bluebrain.github.io/nexus/schemas/shacl-20170720.ttl",
+ "_prefix": "schema"
+ },
+ {
+ "_namespace": "https://bluebrain.github.io/nexus/schemas/unconstrained.json",
+ "_prefix": "resource"
+ },
+ {
+ "_namespace": "https://bluebrain.github.io/nexus/schemas/unconstrained.json",
+ "_prefix": "_"
+ },
+ {
+ "_namespace": "https://bluebrain.github.io/nexus/schemas/views.json",
+ "_prefix": "view"
+ },
+ {
+ "_namespace": "https://bluebrain.github.io/nexus/schemas/storages.json",
+ "_prefix": "storage"
+ },
+ {
+ "_namespace": "https://bluebrain.github.io/nexus/schemas/files.json",
+ "_prefix": "file"
+ },
+ {
+ "_namespace": "https://bluebrain.github.io/nexus/schemas/resolvers.json",
+ "_prefix": "resolver"
+ },
+ {
+ "_namespace": "https://bluebrain.github.io/nexus/vocabulary/defaultSparqlIndex",
+ "_prefix": "graph"
+ },
+ {
+ "_namespace": "https://bluebrain.github.io/nexus/schemas/archives.json",
+ "_prefix": "archive"
+ },
+ {
+ "_namespace": "https://bluebrain.github.io/nexus/vocabulary/diskStorageDefault",
+ "_prefix": "defaultStorage"
+ }
+ ],
+ "_label": "myproject",
+ "_organizationLabel": "myorg",
+ "_organizationUuid": "41f169b8-7ae4-4878-a7ff-6309cb54ae2f",
+ "_rev": 3,
+ "_self": "http://localhost:8080/v1/projects/myorg/myproject",
+ "_updatedAt": "2021-05-10T15:47:40.659Z",
+ "_updatedBy": "http://localhost:8080/v1/anonymous",
+ "_uuid": "813cfa79-75fc-48eb-b931-abb21edb082b"
+}
Deletes the target project and all its resources.
Once the request is processed, an asynchronous task gets triggered and deletes the data generated from each resource type (e.g.: the indices generated by the view resources, the binaries generated by the file resources). During this time, no write operation can be performed at the project level nor at the resources level for the target project. When the task is finished, the target project can be recreated if desired.
@@ -1204,7 +1307,7 @@-Dplugins.project-deletion.enabled=true
-Dplugins.project-deletion.included-projects.1=".+""
and -Dplugins.project-deletion.excluded-projects.1=".+protected.+""
For details on available configuration, please visit the plugin configuration.
For details on available configuration, please visit the plugin configuration.
GET /v1/project-deletion/config
It returns the configuration of the project deletion plugin.
@@ -1237,7 +1340,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Quotas are rooted in /v1/quotas collection.
@@ -309,7 +316,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Realms are rooted in /v1/realms
collection.
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Resolvers are rooted in the /v1/resolvers/{org_label}/{project_label}
collection and are used in the following scenarios:
This operation returns only resolvers from projects defined in the organisation {org_label}
and where the caller has the resources/read
permission.
This operation returns only resolvers from projects where the caller has the resources/read
permission.
{updatedBy}
: Iri - can be used to filter the resulting resolvers based on the person which performed the last update
{search}
: String - can be provided to select only the resolvers in the collection that have attribute values matching (containing) the provided string{sort}
: String - can be used to sort resolvers based on a payloads’ field. This parameter can appear multiple times to enable sorting by multiple fields. The default is done by _createdBy
and @id
.{aggregations}
: Boolean - if true
then the response will only contain aggregations of the @type
and _project
fields; defaults to false
. See Aggregations.Example
Fetches the original source payload of a resource using the provided resolver.
+If the resolver segment ({resolver_id}
) is _
the resource is fetched from the first resolver in the requested project ({org_label}/{project_label}
). The resolvers are ordered by its priority field.
GET /v1/resolvers/{org_label}/{project_label}/{resolver_id}/{resource_id}/source?rev={rev}&tag={tag}
+
+where … - {resource_id}
: Iri - the @id value of the resource to be retrieved. - {rev}
: Number - the targeted revision to be fetched. This field is optional and defaults to the latest revision. - {tag}
: String - the targeted tag to be fetched. This field is optional.
{rev}
and {tag}
fields cannot be simultaneously present.
Example
+sourcecurl "http://localhost:8080/v1/resolvers/myorg/myproj/nxv:myresolver/fd8a2b32-170e-44e8-808f-44a8cbbc49b0/source"
source{
+ "@context": {
+ "ex": "http://localhost:8080/",
+ "@vocab": "http://localhost:8080/"
+ },
+ "@type": "ex:Custom",
+ "name": "Alex",
+ "number": 24,
+ "bool": false
+}
From Delta 1.5, it is possible to fetch SSEs for all resolvers or just resolvers in the scope of an organization or a project.
GET /v1/resolvers/events # for all resolver events in the application
@@ -999,7 +1037,7 @@
-v1.8.x
+Snapshot
@@ -1009,7 +1047,7 @@
diff --git a/docs/delta/api/resources-api.html b/docs/delta/api/resources-api.html
index b9e8cedc88..58d1da5f02 100644
--- a/docs/delta/api/resources-api.html
+++ b/docs/delta/api/resources-api.html
@@ -154,6 +154,8 @@
- Releases
+ - v1.9 Release Notes
+ - v1.8 To v1.9 Migration
- v1.8 Release Notes
- v1.7 To v1.8 Migration
- v1.7 Release Notes
@@ -162,14 +164,7 @@
- v1.5 To v1.6 Migration
- v1.5 Release Notes
- v1.4 To v1.5 Migration
- - v1.4 Release Notes
- - v1.3 To v1.4 Migration
- - v1.3 Release Notes
- - v1.2 To v1.3 Migration
- - v1.2 Release Notes
- - v1.1 Release Notes
- - v1.0 To v1.1 Migration
- - v1.0 Release Notes
+ - Older releases
- Roadmap
- Nexus Fusion
@@ -202,11 +197,15 @@
- Permissions
- Realms
- Access Control Lists
+ - User Permissions
- Organizations
- Projects
- Quotas
- Schemas
- Resources
+ - Trial
+ - Multi fetch
+ - Id Resolution
- Resolvers
- Views
@@ -226,6 +225,7 @@
- Global events
- Supervision
+ - Nexus Metadata
- Plugins
- Benchmarks
@@ -241,17 +241,20 @@
- Resources
+ - Nexus metadata
- Indexing
- Create using POST
- Create using PUT
- Update
- Refresh
- - Validate
- Tag
- Remove tag
- Deprecate
+ - Undeprecate
+ - Change schema
- Fetch
- Fetch original payload
+ - Fetch remote contexts
- Fetch tags
- List
@@ -259,6 +262,7 @@
- Within an organization
- Within all projects
- Parameter description
+ - Aggregations
- List filtering by schema
@@ -276,7 +280,7 @@
- Resources
+ - Nexus metadata
- Indexing
- Create using POST
- Create using PUT
- Update
- Refresh
- - Validate
- Tag
- Remove tag
- Deprecate
+ - Undeprecate
+ - Change schema
- Fetch
- Fetch original payload
+ - Fetch remote contexts
- Fetch tags
- List
@@ -309,6 +316,7 @@
- Within an organization
- Within all projects
- Parameter description
+ - Aggregations
- List filtering by schema
@@ -327,6 +335,13 @@
@@ -291,17 +295,20 @@
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Generic resources are rooted in the /v1/resources/{org_label}/{project_label}/{schema_id}
collection.
When modifying resources, the caller must have resources/write
permissions on the current path of the project or the ancestor paths.
When reading resources, the caller must have resources/read
permissions on the current path of the project or the ancestor paths.
Please visit Authentication & authorization section to learn more about it.
From Delta v1.5, remote contexts are only resolved during creates and updates. That means that when those get updated, the resources importing them must be also updated to take them into account the new version.
Please visit Authentication & authorization section to learn more about it.
Remote contexts are only resolved during creates and updates. That means that when those get updated, the resources importing them must be also updated to take them into account in a new version.
The json payload for create and update operations cannot contain keys beginning with underscore (_), as these fields are reserved for Nexus metadata
When using the endpoints described on this page, the responses will contain global metadata described on the Nexus Metadata page. In addition, the following resource specific metadata can be present
+_project
: address of the resource’s project_incoming
: address to query to obtain the list of incoming links_outgoing
: address to query to obtain the list of outgoing links_constrainedBy
: @id
of the schema used to validate the resource; the schema can only be identified uniquely together with _schemaProject
. If no schema has been used to validate the resource, it will indicate the unconstrained identifier._schemaProject
: address of the project where the _constrainedBy
schema is foundAll the API calls modifying a resource (creation, update, tagging, deprecation) can specify whether the resource should be indexed synchronously or in the background. This behaviour is controlled using indexing
query param, which can be one of two values:
sync
- the resource will be indexed synchronously and the API call won’t return until the indexing is finished
POST /v1/resources/{org_label}/{project_label}/{schema_id}
+POST /v1/resources/{org_label}/{project_label}/{schema_id}?tag={tag}
{...}
The json payload:
@@ -355,6 +380,12 @@ If the @id
value is found on the payload, this @id will be used.
- If the
@id
value is not found on the payload, an @id will be generated as follows: base:{UUID}
. The base
is the prefix
defined on the resource’s project ({project_label}
).
+The {schema_id}
segment allows to define an existing SHACL schema to validate the resource with:
+
+ - If
_
is provided, no SHACL validation will be performed
+ - If another value is provided, Nexus will attempt to resolve the schema then validate the expanded JSON-LD value generated from the provided payload.
+
+The {tag}
param is an optional tag associated with the first revision of this resource
Example
- Request
@@ -407,9 +438,11 @@
Create using PUT
This alternative endpoint to create a resource is useful in case the json payload does not contain an @id
but you want to specify one. The @id will be specified in the last segment of the endpoint URI.
-PUT /v1/resources/{org_label}/{project_label}/{schema_id}/{resource_id}
+PUT /v1/resources/{org_label}/{project_label}/{schema_id}/{resource_id}?tag={tag}
{...}
+… where {tag}
is an optional tag associated with the first revision of this resource.
+The {schema_id}
has the same behaviour as the creation using post operation.
Note that if the payload contains an @id different from the {resource_id}
, the request will fail.
Example
-Update
+Update
A update operation which does not result in a change of the existing resources is ignored
+If the submitted update does not result in changes compared to the current revision of the resource, it will be ignored and the current revision will be returned to the user.
+Nevertheless, if a tag is provided, it will still be applied to the current revision of the resource.
This operation overrides the payload.
In order to ensure a client does not perform any changes to a resource without having had seen the previous revision of the resource, the last revision needs to be passed as a query parameter.
-PUT /v1/resources/{org_label}/{project_label}/{schema_id}/{resource_id}?rev={previous_rev}
+PUT /v1/resources/{org_label}/{project_label}/{schema_id}/{resource_id}?rev={previous_rev}&tag={tag}
{...}
-… where {previous_rev}
is the last known revision number for the resource.
+… where
+
+ {previous_rev}
is the last known revision number for the resource.
+ {tag}
is an optional tag associated with the same revision as the current update. For example, if previous_rev
is 2, both the updated payload and tag
will be associated with revision 3; the new latest revision.
+
+The {schema_id}
segment allows to define an existing SHACL schema to validate the resource with:
+
+ - If
_
is provided, no SHACL validation will be performed with the latest version of its current schema
+ - If another value is provided, it has to match the identifier of the current schema as changing the schema of a resource is not currently supported. A different revision or tag of this schema can be provided though.
+
Example
-Refresh
+Refresh
A refresh operation which does not result in a change of the existing resources is ignored
+If the refresh does not result in changes compared to the current revision of the resource, the update will be ignored and the current revision will be returned to the user.
+If the submitted update does not result in changes compared to the current revision of the resource, it will be ignored and the current revision will be returned to the user.
+Nevertheless, if a tag is provided, it will still be applied to the current revision of the resource.
+@@@
This operation refreshes the compacted and expanded representations of the resource.
This is equivalent of doing an update with the same source as the last revision of the resource. It is useful when the schema or project contexts have changed, in order for the changes to be reflected in the resource.
PUT /v1/resources/{org_label}/{project_label}/{schema_id}/{resource_id}/refresh
@@ -549,24 +597,6 @@ <
"_updatedBy": "http://localhost:8080/v1/realms/myrealm/users/john"
}
-Validate
-This operation runs validation of a resource against a schema. This would be useful to test whether resources would match the shape of a new schema.
-GET /v1/resources/{org_label}/{project_label}/{schema_id}/{resource_id}/validate
-
-Example
-
- - Request
- -
-
sourcecurl "http://localhost:8080/v1/resources/myorg/myproj/myschema/base:fd8a2b32-170e-44e8-808f-44a8cbbc49b0/validate"
- - Response
- -
-
source{
- "@context" : "https://bluebrain.github.io/nexus/contexts/shacl-20170720.json",
- "@type" : "sh:ValidationReport",
- "conforms" : true,
- "targetedNodes" : 2
-}
-
Tag
Links a resource revision to a specific name.
Tagging a resource is considered to be an update as well.
@@ -689,6 +719,69 @@ Undeprecate
+Unlocks a previously deprecated resource. Further operations can then be performed. The resource will again be found when listing/querying.
+Undeprecating a resource is considered to be an update as well.
+PUT /v1/resources/{org_label}/{project_label}/{schema_id}/{resource_id}/undeprecate?rev={previous_rev}
+
+… where {previous_rev}
is the last known revision number for the resource.
+Example
+
+ - Request
+ -
+
sourcecurl -X PUT \
+ -H "Content-Type: application/json" \
+ "http://localhost:8080/v1/resources/myorg/myproj/myschema/fd8a2b32-170e-44e8-808f-44a8cbbc49b0/undeprecate?rev=3"
+ - Response
+ -
+
source{
+ "@context": "https://bluebrain.github.io/nexus/contexts/metadata.json",
+ "@id": "http://localhost:8080/v1/resources/myorg/myproj/fd8a2b32-170e-44e8-808f-44a8cbbc49b0",
+ "@type": "http://localhost:8080/Custom",
+ "_incoming": "http://localhost:8080/v1/resources/myorg/myproj/myschema/fd8a2b32-170e-44e8-808f-44a8cbbc49b0/incoming",
+ "_outgoing": "http://localhost:8080/v1/resources/myorg/myproj/myschema/fd8a2b32-170e-44e8-808f-44a8cbbc49b0/outgoing",
+ "_self": "http://localhost:8080/v1/resources/myorg/myproj/myschema/fd8a2b32-170e-44e8-808f-44a8cbbc49b0",
+ "_constrainedBy": "https://bluebrain.github.io/nexus/schemas/resource",
+ "_project": "http://localhost:8080/v1/projects/myorg/myproj",
+ "_schemaProject": "http://localhost:8080/v1/projects/myorg/myproj",
+ "_rev": 5,
+ "_deprecated": false,
+ "_createdAt": "2021-04-17T14:54:42.939Z",
+ "_createdBy": "http://localhost:8080/v1/realms/myrealm/users/john",
+ "_updatedAt": "2021-04-17T15:02:42.939Z",
+ "_updatedBy": "http://localhost:8080/v1/realms/myrealm/users/john"
+}
+
+Change schema
+This operation allows to only change the schema of a resource without providing any payload.
+PUT /v1/resources/{org_label}/{project_label}/{schema_id}/{resource_id}/update-schema
+
+Example
+
+ - Request
+ -
+
sourcecurl -X PUT \
+ "http://localhost:8080/v1/resources/myorg/myproj/myNewSchema/myId/update-schema"
+ - Response
+ -
+
source{
+ "@context": "https://bluebrain.github.io/nexus/contexts/metadata.json",
+ "@id": "http://localhost:8080/v1/resources/myorg/myproj/myId",
+ "@type": "http://localhost:8080/Custom",
+ "_incoming": "http://localhost:8080/v1/resources/myorg/myproj/myNewSchema/myId/incoming",
+ "_outgoing": "http://localhost:8080/v1/resources/myorg/myproj/myNewSchema/myId/outgoing",
+ "_self": "http://localhost:8080/v1/resources/myorg/myproj/myNewSchema/myId",
+ "_constrainedBy": "http://localhost:8080/v1/resources/myorg/myproj/_/myNewSchema",
+ "_project": "http://localhost:8080/v1/projects/myorg/myproj",
+ "_schemaProject": "http://localhost:8080/v1/projects/myorg/myproj",
+ "_rev": 2,
+ "_deprecated": false,
+ "_createdAt": "2021-04-17T14:54:42.939Z",
+ "_createdBy": "http://localhost:8080/v1/realms/myrealm/users/john",
+ "_updatedAt": "2021-04-17T14:56:42.939Z",
+ "_updatedBy": "http://localhost:8080/v1/realms/myrealm/users/john"
+}
+
Fetch
GET /v1/resources/{org_label}/{project_label}/{schema_id}/{resource_id}?rev={rev}&tag={tag}
@@ -698,6 +791,11 @@ {tag}
: String - the targeted tag to be fetched. This field is optional.
{rev}
and {tag}
fields cannot be simultaneously present.
+The {schema_id}
segment allows to pass the resource schema:
+
+ - If
_
is provided, the value is ignored
+ - If another value is provided, it must match the identifier of the resource schema.
+
Example
- Request
@@ -762,6 +860,47 @@ Fetch remote contexts
+Returns the remote contexts that have been detected during the JSON-LD resolution for this resource.
+These contexts can be:
+
+ - Static contexts that are statically defined in Nexus
+ - Project contexts that have been registered by Nexus, in this case the entry also provides the project this context lives and its revision at the time the JSON-LD resolution has been performed
+
+GET /v1/resources/{org_label}/{project_label}/{schema_id}/{resource_id}/remote-contexts?rev={rev}&tag={tag}
+
+where …
+
+ {rev}
: Number - the targeted revision to be fetched. This field is optional and defaults to the latest revision.
+ {tag}
: String - the targeted tag to be fetched. This field is optional.
+
+{rev}
and {tag}
fields cannot be simultaneously present.
+Example
+
+ - Request
+ -
+
sourcecurl "http://localhost:8080/v1/resources/org/proj/_/my-resource/remote-contexts"
+ - Response
+ -
+
source{
+ "@context": "https://bluebrain.github.io/nexus/contexts/remote-contexts.json",
+ "remoteContexts": [
+ {
+ "@type": "StaticContextRef",
+ "iri": "https://bluebrain.github.io/nexus/contexts/metadata.json"
+ },
+ {
+ "@type": "ProjectContextRef",
+ "iri": "https://localhost/nexus/context",
+ "resource": {
+ "id": "https://localhost/nexus/context",
+ "project": "org/proj",
+ "rev": 5
+ }
+ }
+ ]
+}
+
Fetch tags
GET /v1/resources/{org_label}/{project_label}/{schema_id}/{resource_id}/tags?rev={rev}&tag={tag}
@@ -797,6 +936,7 @@
This operation returns only resources from projects defined in the organisation {org_label}
and where the caller has the resources/read
permission.
This operation returns only resources from projects where the caller has the resources/read
permission.
A time range parameter allows to filter resources by their creation date or their last update date.
@@ -855,6 +1002,7 @@{deprecated}
: Boolean - can be used to filter the resulting resources based on their deprecation status
{rev}
: Number - can be used to filter the resulting resources based on their revision value{type}
: Iri - can be used to filter the resulting resources based on their @type
value. This parameter can appear multiple times, filtering further the @type
value.{typeOperator}
: String (and
/or
) - used to determine how multiple type
values affect the query, either requiring all to match (and
) or any to match (or
); defaults to or
. See De Morgan’s laws for inference rules.{createdBy}
: Iri - can be used to filter the resulting resources based on their creator{createdAt}
: Time range - can be used to filter the resulting resources based on their creation date{updatedBy}
: Iri - can be used to filter the resulting resources based on the person which performed the last update{schema}
: Iri - can be used to filter the resulting resources based on the conformant schema
{search}
: String - can be provided to select only the resources in the collection that have attribute values matching (containing) the provided string{sort}
: String - can be used to sort resources based on a payloads’ field. This parameter can appear multiple times to enable sorting by multiple fields. The default is done by _createdBy
and @id
.{aggregations}
: Boolean - if true
then the response will only contain aggregations of the @type
and _project
fields; defaults to false
. See AggregationsExample
sourcecurl "http://localhost:8080/v1/resources/myorg?aggregations=true"
source{
+ "@context": "https://bluebrain.github.io/nexus/contexts/aggregations.json",
+ "aggregations": {
+ "projects": {
+ "buckets": [
+ {
+ "doc_count": 5,
+ "key": "http://delta:8080/v1/projects/myorg/myproject"
+ },
+ {
+ "doc_count": 5,
+ "key": "http://delta:8080/v1/projects/myorg/myproject2"
+ }
+ ],
+ "doc_count_error_upper_bound": 0,
+ "sum_other_doc_count": 0
+ },
+ "types": {
+ "buckets": [
+ {
+ "doc_count": 6,
+ "key": "https://bluebrain.github.io/nexus/vocabulary/View"
+ },
+ {
+ "doc_count": 2,
+ "key": "https://bluebrain.github.io/nexus/vocabulary/CompositeView"
+ },
+ {
+ "doc_count": 2,
+ "key": "https://bluebrain.github.io/nexus/vocabulary/DiskStorage"
+ },
+ {
+ "doc_count": 2,
+ "key": "https://bluebrain.github.io/nexus/vocabulary/ElasticSearchView"
+ },
+ {
+ "doc_count": 2,
+ "key": "https://bluebrain.github.io/nexus/vocabulary/InProject"
+ },
+ {
+ "doc_count": 2,
+ "key": "https://bluebrain.github.io/nexus/vocabulary/Project"
+ },
+ {
+ "doc_count": 2,
+ "key": "https://bluebrain.github.io/nexus/vocabulary/Resolver"
+ },
+ {
+ "doc_count": 2,
+ "key": "https://bluebrain.github.io/nexus/vocabulary/SparqlView"
+ },
+ {
+ "doc_count": 2,
+ "key": "https://bluebrain.github.io/nexus/vocabulary/Storage"
+ }
+ ],
+ "doc_count_error_upper_bound": 0,
+ "sum_other_doc_count": 0
+ }
+ },
+ "total": 12
}
Aggregations are experimental and the API is subject to change.
Adding the aggregations=true
query parameter to a list query allows to aggregate the underlying resources by predefined terms. Currently, the following aggregations will be returned:
projects
: a bucket aggregation of the resources by the project they belong totypes
: a bucket aggregation of the @types
featured in the resourcesAggregation works on the same scopes as listing (all projects, organization, and project), and only aggregates the resources for which the caller has resource/read
permission.
This operation is only available at the project scope.
{type}
: Iri - can be used to filter the resulting resources based on their @type
value. This parameter can appear multiple times, filtering further the @type
value.
{createdBy}
: Iri - can be used to filter the resulting resources based on their creator{updatedBy}
: Iri - can be used to filter the resulting resources based on the person which performed the last update{aggregations}
: Boolean - if true
then the response will only contain aggregations of the @type
and _project
fields; defaults to false
. See AggregationsExample
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Schemas are rooted in the /v1/schemas/{org_label}/{project_label}
collection. They define a set of rules and constraints using SHACL. Once those schemas are present, other resources can be created against them. Those resources won’t be successfully created unless they match the required constraints defined on the schema.
This operation returns only schemas from projects defined in the organisation {org_label}
and where the caller has the resources/read
permission.
This operation returns only schemas from projects defined the organisation {org_label}
and where the caller has the resources/read
permission.
{updatedBy}
: Iri - can be used to filter the resulting schemas based on the person which performed the last update
{search}
: String - can be provided to select only the schemas in the collection that have attribute values matching (containing) the provided string{sort}
: String - can be used to sort schemas based on a payloads’ field. This parameter can appear multiple times to enable sorting by multiple fields. The default is done by _createdBy
and @id
.{aggregations}
: Boolean - if true
then the response will only contain aggregations of the @type
and _project
fields; defaults to false
. See Aggregations.Example
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Nexus provides global search functionality across all projects through the search plugin.
Nexus provides global search functionality across all projects through the search plugin.
The search plugin is experimental and its functionality and API can change without notice.
For instructions on how to configure global search in Nexus and how it works please visit the Search configuration page.
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Storages are rooted in the /v1/storages/{org_label}/{project_label}
collection and are used to describe where files are physically stored.
The Remote disk storage and it remote service implementation are now deprecated and will be removed in an upcoming release.
This storage type relies on a remote HTTP service that exposes basic file operations on an underlying POSIX file-system. This is particularly handy if your organization is running some kind of distributed network storage (such as Ceph, Gluster, GPFS, Lustre, …) that you don’t want to mount directly on the system where Nexus Delta runs.
-While there’s no formal specification for this service, you can check out or deploy our own implementation: Nexus remote storage service.
-In order to be able to use this storage, the configuration flag plugins.storage.storages.remote-disk.enabled
should be set to true
.
While there’s no formal specification for this service, you can check out or deploy our own implementation: Nexus remote storage service.
+In order to be able to use this storage, the configuration flag plugins.storage.storages.remote-disk.enabled
should be set to true
. More information about configuration
{
"@type": "RemoteDiskStorage",
"default": "{default}",
"endpoint": "{endpoint}",
- "credentials": "{credentials}",
"folder": "{folder}",
"readPermission": "{read_permission}",
"writePermission": "{write_permission}",
@@ -372,7 +378,6 @@
{default}
: Boolean - the flag to decide whether this storage is going to become the default storage for the target project or not.
{endpoint}
: Uri - the endpoint where the storage service is listening to requests. This field is optional, defaulting to the configuration flag plugins.storage.storages.remote-disk.default-endpoint
.
- {credentials}
: String - the service account access token to authenticate and authorize Nexus Delta client against the storage service. This field is optional, defaulting to the configuration flag plugins.storage.storages.remote-disk.default-credentials
.
{folder}
: String - the storage service bucket where files using this storage are going to be saved.
{read_permission}
: String - the permission a client must have in order to fetch files using this storage. This field is optional, defaulting to the configuration flag plugins.storage.storages.remote-disk.default-read-permission
(resources/read
).
{write_permission}
: String - the permission a client must have in order to create files using this storage. This field is optional, defaulting to the configuration flag plugins.storage.storages.remote-disk.default-write-permission
(files/write
).
@@ -385,8 +390,6 @@
{default}
: Boolean - the flag to decide whether this storage is going to become the default storage for the target project or not.
{endpoint}
: Uri - the Amazon S3 compatible service endpoint. This field is optional, defaulting to the configuration flag plugins.storage.storages.amazon.default-endpoint
.
- {access_key}
: String - the Amazon S3 compatible access key. This field is optional, defaulting to the configuration flag plugins.storage.storages.amazon.default-access-key
.
- {secret_key}
: String - the Amazon S3 compatible secret key. This field is optional, defaulting to the configuration flag plugins.storage.storages.amazon.default-secret-key
.
{region}
: String - the Amazon S3 compatible region. This field is optional, defaulting to the S3 default region configuration.
{read_permission}
: String - the permission a client must have in order to fetch files using this storage. This field is optional, defaulting to the configuration flag plugins.storage.storages.amazon.default-read-permission
(resources/read
).
{write_permission}
: String - the permission a client must have in order to create files using this storage. This field is optional, defaulting to the configuration flag plugins.storage.storages.amazon.default-write-permission
(files/write
).
@@ -771,6 +772,7 @@
This operation returns only storages from projects defined in the organisation {org_label}
and where the caller has the resources/read
permission.
This operation returns only storages from projects where the caller has the resources/read
permission.
{updatedBy}
: Iri - can be used to filter the resulting storages based on the person which performed the last update
{search}
: String - can be provided to select only the storages in the collection that have attribute values matching (containing) the provided string{sort}
: String - can be used to sort storages based on a payloads’ field. This parameter can appear multiple times to enable sorting by multiple fields. The default is done by _createdBy
and @id
.{aggregations}
: Boolean - if true
then the response will only contain aggregations of the @type
and _project
fields; defaults to false
. See Aggregations.Example
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+This endpoint returns information about the projections running on the current node.
@@ -341,7 +348,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Trial operations contain read-only operations designed to help users compose and validate their resources before effectively saving them in Nexus.
When performing a request, the caller must have resources/read
permission on the project each resource belongs to.
Please visit Authentication & authorization section to learn more about it.
This endpoint allows to create and get the output of a resource, optionally validating with an existing schema or a new one.
+It applies the same validation steps than the creation/update of resources, the main difference being that nothing is persisted.
+POST /v1/trial/resources/{org_label}/{project_label}
+
+{
+ "schema": {schema},
+ "resource": {resource}
+}
+
+Where:
+{schema}
: String/Json: The schema to validate the provided resource. If a string is provided, it will attempt to resolve it as an existing schema. If a json payload is provided, it will attempt to generate the schema and then use the result to validate the resource. This field is optional and defaults to no SHACL validation.{resource}
: Json: The resource payload to test and validateThe Json response will contain:
+Example
+sourcecurl -X PUT \
+ -H "Content-Type: application/json" \
+ "http://localhost:8080/trial/resources/myorg/myproj/" \
+ -d \
+'{
+ "schema": "https://bbp.epfl.ch/nexus/schema/morphology"
+ "resource": {
+ "@context": [
+ "https://neuroshapes.org",
+ "https://bluebrain.github.io/nexus/contexts/metadata.json",
+ {
+ "@vocab": "https://bluebrain.github.io/nexus/vocabulary/"
+ }
+ ],
+ "@id": "https://bbp.epfl.ch/nexus/data/morphology-001",
+ "@type": "Morphology",
+ "name": "Morphology 001"
+ }
+ }'
source{
+ "schema": "https://bbp.epfl.ch/nexus/schema/morphology"
+ "resource": {
+ "@context": [
+ "https://neuroshapes.org",
+ "https://bluebrain.github.io/nexus/contexts/metadata.json",
+ {
+ "@vocab": "https://bluebrain.github.io/nexus/vocabulary/"
+ }
+ ],
+ "@id": "https://bbp.epfl.ch/nexus/data/morphology-001",
+ "@type": "Morphology",
+ "name": "Morphology 001"
+ }
+}
source{
+ "result": {
+ "@context": [
+ "https://bluebrain.github.io/nexus/contexts/metadata.json",
+ {
+ "@vocab": "https://bluebrain.github.io/nexus/vocabulary/"
+ },
+ "https://neuroshapes.org"
+ ],
+ "@id": "https://bbp.epfl.ch/nexus/data/morphology-001",
+ "@type": "Morphology",
+ "name": "Morphology 001",
+ "_constrainedBy": "https://bbp.epfl.ch/nexus/schema/morphology",
+ "_createdAt": "2023-09-18T12:00:00Z",
+ "_createdBy": "http://localhost/v1/realms/wonderland/users/alice",
+ "_deprecated": false,
+ "_incoming": "http://localhost/v1/resources/myorg/myproj/_/https:%2F%2Fbluebrain.github.io%2Fnexus%2Fvocabulary%2FmyId/incoming",
+ "_outgoing": "http://localhost/v1/resources/myorg/myproj/_/https:%2F%2Fbluebrain.github.io%2Fnexus%2Fvocabulary%2FmyId/outgoing",
+ "_project": "http://localhost/v1/projects/myorg/myproj",
+ "_rev": 1,
+ "_schemaProject": "http://localhost/v1/projects/myorg/myproj",
+ "_self": "http://localhost/v1/resources/myorg/myproj/_/https:%2F%2Fbluebrain.github.io%2Fnexus%2Fvocabulary%2FmyId",
+ "_updatedAt": "2023-09-18T12:00:00Z",
+ "_updatedBy": "http://localhost/v1/realms/wonderland/users/alice"
+ }
+}
This operation runs validation of a resource against a schema. This would be useful to test whether resources would match the shape of a new schema.
+GET /v1/resources/{org_label}/{project_label}/{schema_id}/{resource_id}/validate
+
+Example
+sourcecurl "http://localhost:8080/v1/resources/myorg/myproj/myschema/base:fd8a2b32-170e-44e8-808f-44a8cbbc49b0/validate"
source{
+ "@context" : "https://bluebrain.github.io/nexus/contexts/shacl-20170720.json",
+ "@type" : "sh:ValidationReport",
+ "conforms" : true,
+ "targetedNodes" : 2
+}
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+A user of delta is given certain permissions. This is done using the ACLs API
+Sometimes for the sake of simplicity, it can be easier to ask whether the current user has a specific permission in a specific context. This is why the user permissions API exists
The described endpoints are experimental and the responses structure might change in the future.
This operation determines whether the current logged in user has a specific permission in a specific context
+HEAD /v1/user/permissions/{org_label}/{project_label}?permission={permission}
+
+where - {permission}
: String - the permission to check
This operation determines whether the current logged in user would be able to access files on a specific storage
+HEAD /v1/user/permissions/{org_label}/{project_label}?storage={storage_id}&type={access_type}
+
+where - {storage_id}
: String - the id of the storage - {access_type}
: String - the access type of the storage. Can be read
or write
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+GET /v1/version
-This endpoint returns information about the running Delta instance: Delta version, plugin versions and service dependency versions(e.g Cassandra).
+This endpoint returns information about the running Delta instance: Delta version, plugin versions and service dependency versions(e.g PostgreSQL).
source{
"@context": "https://bluebrain.github.io/nexus/contexts/version.json",
- "delta": "1.5.0",
+ "delta": "1.9.0",
"dependencies": {
"blazegraph": "2.1.6-RC",
- "postgresql": "15.3",
- "elasticsearch": "8.7.1",
- "remoteStorage": "1.8.0"
+ "postgresql": "15.5",
+ "elasticsearch": "8.11.1",
+ "remoteStorage": "1.9.0"
},
"plugins": {
- "archive": "1.8.0",
- "blazegraph": "1.8.0",
- "composite-views": "1.8.0",
- "elasticsearch": "1.8.0",
- "storage": "1.8.0"
+ "archive": "1.9.0",
+ "blazegraph": "1.9.0",
+ "composite-views": "1.9.0",
+ "elasticsearch": "1.9.0",
+ "storage": "1.9.0"
}
}
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+This view is an aggregate of ElasticSearchViews. The view itself does not create any index, but it references the already existing indices of the linked ElasticSearchViews.
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+This view is an aggregate of SparqlViews. The view itself does not create any namespace, but it references the already existing namespaces of the linked SparqlViews.
@@ -794,7 +801,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+A Composite Sink handles the following steps of composite view indexing
+These steps can be implemented in different ways. In Nexus Delta, there are two kinds of Composite Sink that can be selected via configuration.
+ +By default, Nexus Delta will use the Single Composite Sink. This sink performs one query to the Blazegraph common namespace for each resource in the project. The queries are done in chronological order (by the updatedAt
time of the resources).
We recommend reading through the search configuration example use case and the Composite View API reference to learn more about Composite Views.
+Starting with Delta 1.9, it is possible to configure Nexus Delta to use a Batch Composite Sink. This implementation of the Composite Sink can query the Blazegraph common namespace for multiple resource IDs at the same time.
We recommend to start using Composite Views with the default Single Composite Sink. Once you have a good understanding of it, the Batch Composite Sink can be used to enhance the performance of your deployment.
In order to enable the Batch Composite Sink, configure the following Nexus Delta property:
+plugins.composite-views.sink-config = batch
Furthermore, you can configure the maximum size of a batch and the maximum interval using:
+plugins.composite-views.{projection-plugin}-batch.max-elements = {max-elements}
plugins.composite-views.{projection-plugin}-batch.max-interval = {max-interval}
where:
+{projection-plugin}
is either elasticsearch
or blazegraph
. The batching options of a Composite Sink can be set separately for each target type.{max-elements}
is the maximum number of elements to batch at once (defaults to 10
){max-interval}
is the maximum interval of time to wait for {max-elements}
elementsIn order to use the Batch Composite Sink successfully, it is necessary to rework some aspects of a regular SPARQL construct query. We explain the changes through an example.
+Suppose we are in a situation where Composite Views are using the Single Composite Sink and have the following query:
+PREFIX schema: <http://schema.org/>
+PREFIX nxv: <https://bluebrain.github.io/nexus/vocabulary/>
+CONSTRUCT {
+ ?id nxv:name ?name ;
+ nxv:age ?age .
+ nxv:parent ?parent .
+} WHERE {
+ BIND({resource_id} AS ?id) .
+ ?id schema:name ?name .
+
+ OPTIONAL { ?id schema:age ?age }
+ OPTIONAL { ?id schema:parent ?parent . }
+}
+
+Using the default Single Composite Sink, Nexus Delta will query the resources Alice and Bob individually and obtain the following n-triples from Blazegraph:
+<http://people.com/Alice> <http://schema.org/name> <Alice>
+<http://people.com/Alice> <http://schema.org/parent> <http://people.com/Bob>
+
+<http://people.com/Bob> <http://schema.org/name> <Bob>
+<http://people.com/Bob> <http://schema.org/age> <42>
+
+In particular, note that when looking at the graph of Alice, we do not know the age of Bob.
+The first change to introduce in order to make this query work with batches of resources is to replace the BIND({resource_id} AS ?id) .
with VALUES ?id { {resources_id} }
. Nexus Delta will use this template to replace {resource_id}
with multiple resource in case it receives a batch of more than one element. The query is now:
PREFIX schema: <http://schema.org/>
+PREFIX nxv: <https://bluebrain.github.io/nexus/vocabulary/>
+CONSTRUCT {
+ ?id nxv:name ?name ;
+ nxv:age ?age .
+ nxv:parent ?parent .
+} WHERE {
+ VALUES ?id { {resource_id} } .
+ ?id schema:name ?name .
+
+ OPTIONAL { ?id schema:age ?age }
+ OPTIONAL { ?id schema:parent ?parent . }
+}
+
+With the Batch Composite Sink enabled, if Alice and Bob are batched together, this query will result in the following triples:
+<http://people.com/Alice> <http://schema.org/name> <Alice>
+<http://people.com/Alice> <http://schema.org/parent> <http://people.com/Bob>
+<http://people.com/Bob> <http://schema.org/name> <Bob>
+<http://people.com/Bob> <http://schema.org/age> <42>
+
+Note how the results are the merged result of the individual queries. While we were able to query several resources simultaneously, we are now facing a framing problem. If we try to frame http://people.com/Alice
, its graph now contains more information than before; it will now include the age of Bob, something that we did not request.
In order to solve this problem, we will introduce aliasing for the root resource IDs. The query will now become:
+PREFIX schema: <http://schema.org/>
+PREFIX nxv: <https://bluebrain.github.io/nexus/vocabulary/>
+CONSTRUCT {
+ ?alias nxv:name ?name ;
+ nxv:age ?age .
+ nxv:parent ?parent .
+} WHERE {
+ VALUES ?id { {resource_id} } .
+ BIND(IRI(CONCAT(STR(?id), '/alias')) AS ?alias) .
+
+ ?id schema:name ?name .
+
+ OPTIONAL { ?id schema:age ?age }
+ OPTIONAL { ?id schema:parent ?parent }
+}
+
+With this query, a batch query for both Alice and Bob will now yield:
+<http://people.com/Alice/alias> <http://schema.org/name> <Alice>
+<http://people.com/Alice/alias> <http://schema.org/parent> <http://people.com/Bob>
+<http://people.com/Bob/alias> <http://schema.org/name> <Bob>
+<http://people.com/Bob/alias> <http://schema.org/age> <42>
+
+You can see that the root node of Bob’s graph is now http://people.com/Bob/alias
, while Alice’s parent is http://people.com/Bob
. This distinction ensures that we cannot get Bob’s age by looking at Alice’s graph, thus reproducing the behavior that we had with the Single Composite Sink.
Nexus Delta takes care of framing these results so that the framed documents will be the same as with the Single Composite Sink, and will not contain any alias
keyword. For example, for the resource http://people.com/Alice
, Nexus Delta will obtain its graph by looking at the http://people.com/Alice/alias
root node, and use the resulting graph (removing the /alias
part) for JSON-LD framing.
To use the Batch Composite Sink the following changes are necessary:
+BIND({resource_id} AS ?id)
must become VALUES ?id { {resource_id} }
to allow for batches of resources to be queried from Blazegraph.BIND(IRI(CONCAT(STR(?id), '/alias')) AS ?alias)
needs to be added, and the relevant (?id
) “root nodes” replaced by ?alias
in the CONSTRUCT
part of the query. This is done in order to avoid any clashes between the graphs of several resources. Do note that in case you have several “root nodes” in the CONSTRUCT
part of your construct query, you might need several aliases.+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+This view is composed by multiple sources
and projections
.
{resourceSchema}
: Iri - Selects only resources that are validated against the provided schema Iri to perform the query. This field is optional.{resourceType}
: Iri - Select only resources of the provided type Iri to perform the query. This field is optional.{tag}
: String - Selects only resources with the provided tag to perform the query. This field is optional.{includeMetadata}
: Boolean - If true, the resource’s nexus metadata (_constrainedBy
, _deprecated
, …) will be included in the ElasticSearch document. The default value is false
.{includeDeprecated}
: Boolean - If true, deprecated resources are also indexed. The default value is false
.{permission}
: String - the permission necessary to query this projection. Defaults to views/query
.{projectionId}
: Iri - The identifier of the projection. This field is optional. When missing, a randomly generated Iri will be assigned.
{resourceSchema}
: Iri - Selects only resources that are validated against the provided schema Iri to perform the query. This field is optional.{resourceType}
: Iri - Select only resources of the provided type Iri to perform the query. This field is optional.{tag}
: String - Selects only resources with the provided tag to perform the query. This field is optional.{includeMetadata}
: Boolean - If true, the resource’s nexus metadata (_constrainedBy
, _deprecated
, …) will be stored in the ElasticSearch document. Otherwise it won’t. The default value is false
.{includeDeprecated}
: Boolean - If true, deprecated resources are also indexed. The default value is false
.{query}
: Sparql Construct Query - Defines the Sparql query to execute against the intermediate Sparql space for each target resource.{permission}
: String - the permission necessary to query this projection. Defaults to views/query
.The queries that projections perform to the intermediate Sparql space can be either executed per individual resource, or in batches containing multiple resources. To learn more about it, please refer to the Composite Sink page.
{
"@id": "{someid}",
@@ -2448,7 +2455,7 @@
-v1.8.x
+Snapshot
@@ -2471,13 +2478,13 @@
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+This view creates an ElasticSearch index
and stores the targeted Json resources into an ElasticSearch Document.
GET /v1/views/{org_label}/{project_label}/{view_id}/_mapping
+
+Retrieves the view’s ElasticSearch mapping.
+Example
+sourcecurl "http://localhost:8080/v1/views/myorg/myproj/myview/_mapping"
source{
+ "delta_f488d131-5f29-4721-979e-277a0850fed2_1": {
+ "mappings": {
+ "properties": {
+ "title": {
+ "type": "text",
+ "fields": {
+ "keyword": {
+ "type": "keyword"
+ }
+ }
+ },
+ "description": {
+ "type": "text"
+ }
+ }
+ }
+ }
+}
GET /v1/views/{org_label}/{project_label}/{view_id}/tags?rev={rev}&tag={tag}
@@ -1208,7 +1248,7 @@ +You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Views are rooted in the /v1/views/{org_label}/{project_label}
collection and are used to index the selected resources into a bucket.
async
- (default value) the view will be indexed asynchronously
sync
- the view will be indexed synchronously and the API call won’t return until the indexing is finishedViews are now stopped if no event has been detected for a given period (by default 30 minutes).
-This value can be updated by updating the configuration keys: - plugins.blazegraph.idle-timeout
- plugins.composite-views.idle-timeout
- plugins.elasticsearch.idle-timeout
Setting an infinite time (Inf
) will disable this feature.
There are three available endpoints to list views in different scopes.
This operation returns only views from projects defined in the organisation {org_label}
and where the caller has the resources/read
permission.
This operation returns only views from projects where the caller has the resources/read
permission.
{from}
: Number - is the parameter that describes the offset for the current query; defaults to 0
{size}
: Number - is the parameter that limits the number of results; defaults to 20
{updatedBy}
: Iri - can be used to filter the resulting views based on the person which performed the last update
{search}
: String - can be provided to select only the views in the collection that have attribute values matching (containing) the provided string{sort}
: String - can be used to sort views based on a payloads’ field. This parameter can appear multiple times to enable sorting by multiple fields. The default is done by _createdBy
and @id
.{aggregations}
: Boolean - if true
then the response will only contain aggregations of the @type
and _project
fields; defaults to false
. See Aggregations.Example
+This endpoint returns the indexing failures for the view {view_id}
. The caller must have view/write
permission on {org_label}/{project_label}
.
GET /v1/views/{org_label}/{project_label}/{view_id}/failures
+ ?from={from}
+ &size={size}
+ &instant={instant}
+
+{from}
: Number - is the parameter that describes the offset for the current query; defaults to 0
{size}
: Number - is the parameter that limits the number of results; defaults to 20
{instant}
: Time Range - can be used to filter the results based on the moment they were risenExample
+sourcecurl "http://localhost:8080/v1/views/myorg/myproj/myview/failures"
source{
+ "@context": [
+ "https://bluebrain.github.io/nexus/contexts/metadata.json",
+ "https://bluebrain.github.io/nexus/contexts/search.json",
+ "https://bluebrain.github.io/nexus/contexts/error.json"
+ ],
+ "_total": 2,
+ "_results": [
+ {
+ "errorType": "java.lang.Exception",
+ "id": "https://bluebrain.github.io/nexus/vocabulary/myid",
+ "message": "The resource could not be indexed for the following reason: '...'",
+ "offset": {
+ "@type": "At",
+ "value": 42
+ },
+ "project": "myorg/myproj",
+ "_rev": 1
+ },
+ {
+ "errorType": "java.lang.Exception",
+ "id": "https://bluebrain.github.io/nexus/vocabulary/myid",
+ "message": "The resource could not be indexed for the following reason: '...'",
+ "offset": {
+ "@type": "At",
+ "value": 42
+ },
+ "_rev": 1
+ }
+ ]
+}
This endpoint fetches the available indexing failures. The Last-Event-Id
is optional and provides the id of the indexing failure at which to start the stream; by default all indexing failures are fetched. The caller must have view/write
permission on {org_label}/{project_label}
.
GET /v1/views/{org_label}/{project_label}/{view_id}/failures/sse
+
+Example
+sourcecurl "http://localhost:8080/v1/views/myorg/myproj/myview/failures/sse"
sourcedata:{"@context":"https://bluebrain.github.io/nexus/contexts/error.json","errorType":"java.lang.Exception","id":"myid","message":"boom","offset":{"@type":"At","value":42}}
+event:IndexingFailure
+id:1
From Delta 1.5, it is possible to fetch SSEs for all resolvers or just resolvers in the scope of an organization or a project.
+From Delta 1.5, it is possible to fetch SSEs for all views or just views in the scope of an organization or a project.
GET /v1/views/events # for all view events in the application
GET /v1/views/{org_label}/events # for view events in the given organization
GET /v1/views/{org_label}/{project_label}/events # for view events in the given project
@@ -505,21 +592,6 @@
-This endpoint fetches the available indexing failures as SSEs. The Last-Event-Id
is optional and provides the id of the indexing failure at which to start the stream; by default all indexing failures are fetched. The caller must have view/write
permission on {org_label}/{project_label}
.
GET /v1/views/{org_label}/{project_label}/{view_id}/failures
-
-Example
-sourcecurl "http://localhost:8080/v1/views/myorg/myproj/myview/failures"
sourcedata:{"@context":"https://bluebrain.github.io/nexus/contexts/error.json","errorType":"java.lang.Exception","id":"myid","message":"boom","offset":{"@type":"At","value":42}}
-event:IndexingFailure
-id:1
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Pipes are the processing units of a pipeline for an Elasticsearch view.
@@ -400,8 +407,8 @@Please visit Plugins to learn about how to create/package/deploy a plugin.
Inside this plugin, you can then define additional pipes:
@@ -516,7 +523,7 @@The source code for the core pipes is available here and the associated unit tests here.
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+This view creates a Blazegraph namespace
and stores the targeted resources as RDF triples into a Blazegraph instance.
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Blue Brain Nexus is a collection of software components that address various organizational needs relating to data storage, management, analysis and consumption. It was designed to support the data-driven science iterative cycle at Blue Brain but its genericity allows for its use in arbitrary contexts.
@@ -305,7 +312,7 @@Resource identification is based on HTTP Internationalized Resource Identifiers (IRIs) and uniqueness is guaranteed within the scope of a project. This allows the system to be used in a multi-tenant configuration but at the same time it implies that project and organization identifiers are part of a resource addressing scheme.
In order to avoid limitations in URL lengths and for convenience, resource identifiers can be aliased and compacted (CURIE) using project level configurations.
The system supports OpenID Connect, OAuth 2.0 and JSON Web Tokens (JWTs) standards and can be configured to use identity providers that support these standards. Proof of identity can be provided by passing a Bearer JWT in the Authorization header of the HTTP requests when consuming the RESTful API.
+The system supports OpenID Connect, OAuth 2.0 and JSON Web Tokens (JWTs) standards and can be configured to use identity providers that support these standards. Proof of identity can be provided by passing a Bearer JWT in the Authorization header of the HTTP requests when consuming the RESTful API.
Nexus Delta can use LDAP as an identity management system through several off-the-shelf products that implement these protocols on top of LDAP, like for example Keycloak.
The authorization flow is as follows:
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+The main goal of the benchmarks is to analyze the hardware requirements for a Nexus deployment and to find potential issues and / or bottlenecks. In particular, we are most interested in the following metrics:
@@ -275,7 +282,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+For the newest benchmarks of Nexus version 1.4.2 see here.
@@ -718,7 +725,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+The tests were run against version 1.4.2 in January 2021 using Gatling version 3.5.0.
@@ -1127,7 +1134,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Blue Brain Nexus Delta is a low latency, scalable and secure system that realizes a range of functions to support data management and knowledge graph lifecycles.
It is a central piece of the Nexus ecosystem of software components as it offers a set of foundational capabilities to the other components (Nexus Fusion and Nexus Forge) around data and metadata storage, management, validation and consumption in a secure setting.
-Nexus Delta is developed in the open with a permissive licence (Apache License, version 2.0) using open standards and interoperable semantic web technologies like OpenID Connect, OAuth 2.0, RDF, JSON-LD, SHACL, Server-Sent Events.
+Nexus Delta is developed in the open with a permissive licence (Apache License, version 2.0) using open standards and interoperable semantic web technologies like OpenID Connect, OAuth 2.0, RDF, JSON-LD, SHACL, Server-Sent Events.
It is quite versatile as it is able to handle very small to very large amounts of data on-premise or in the cloud and can be used in a large spectrum of industries being completely domain agnostic.
Please refer to the architecture and api reference sections for more information about this component.
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Nexus Delta entities fall in two categories:
+Upon creation of these entities, Nexus Delta will create metadata fields which are described below.
+_self
: unique address of the entity across Nexus Delta
+ _self
address being discoverable in Nexus Delta’s different responses. Example: from a resources listing operation, the _self
endpoints listed can be used to fetch the underlying resourcesThe following metadata can help to audit an entity.
+_rev
: the revision number of the entity_deprecated
: boolean indicating whether or not the entity is deprecated_createdAt
: datetime at which the entity was first created_createdBy
: the user that first created the entity_updatedAt
: datetime at which the entity was last updated_updatedBy
: the user that last updated the entity+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Starting from version 1.5, Delta introduces the ability to extend its functionality using plugins. Plugins enable developers to add new functionality to Nexus Delta without the need to modify Delta itself. Plugins can introduce various new functionalities, including new API endpoints and indexing capabilities.
Plugins are still an experimental feature and Delta SDKs and dependent modules(rdf, sourcing) provide no binary compatibility guarantees.
Plugins used by Delta need to be packaged as a .jar
file containing the plugin code with all its dependencies. Delta loads plugins from .jar
files located in a directory specified by DELTA_PLUGINS
environment variable.
Plugins must define exactly one class which extends PluginDef trait.
+Plugins must define exactly one class which extends PluginDef trait.
The class must define following methods:
def info: PluginDescription
-this method returns instance of PluginDescription which defines the plugin name and version.
+this method returns instance of PluginDescription which defines the plugin name and version.
def initialize(locator: Locator): Task[Plugin]
this method can be used to initialize the plugin and returns an instance of a Plugin
, which can additionally define logic to stop the plugin gracefully.
sourcepackage ch.epfl.bluebrain.nexus.delta.testplugin
+import cats.effect.IO
import ch.epfl.bluebrain.nexus.delta.sdk.PriorityRoute
import ch.epfl.bluebrain.nexus.delta.sdk.model.ComponentDescription.PluginDescription
import ch.epfl.bluebrain.nexus.delta.sdk.model.{BaseUri, Name}
import ch.epfl.bluebrain.nexus.delta.sdk.plugin.{Plugin, PluginDef}
import izumi.distage.model.Locator
import izumi.distage.model.definition.ModuleDef
-import monix.bio.Task
case class TestPluginDef() extends PluginDef {
@@ -361,7 +368,7 @@
@@ -374,21 +381,21 @@ libraryDependencies += "ch.epfl.bluebrain.nexus" %% "delta-sdk" % deltaVersion % Provided
Delta uses distage library for dependency injection. Each plugin must define ModuleDef
to create instances of its own classes. All the dependencies provided by ModuleDef
s defined in Delta modules, as well as other plugins can be used here.
Delta uses distage library for dependency injection. Each plugin must define ModuleDef
to create instances of its own classes. All the dependencies provided by ModuleDef
s defined in Delta modules, as well as other plugins can be used here.
The plugin can also define instances of following traits/classes, which will be used in Delta:
/version
endpoint./resources
endpoints./version
endpoint./resources
endpoints.In order to avoid clashes between different plugins, Delta uses custom classloader to load plugin classes, which will load classes from the plugin first, then using application classloader and other plugins after that. It is therefore recommended to not include in the plugin jar any dependencies which are also provided by SDK. Libraries can be easily excluded from dependencies in sbt
:
Currently, following Delta functionality is provided by plugins:
Elasticsearch plugin is required in order to provide listings in the API, other plugins can be excluded if their functionality is not needed. All the above plugins are included in the Delta Docker image.
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Yes. As long as the client supports the ability to provide a Authentication
HTTP Header (for authentication purposes) on the HTTP request, any SPARQL client should work.
By default, the permissions used - for an authenticated user - when running Nexus Delta are the ones defined on the JVM property app.permissions.minimum. In order to change that behaviour, please create some ACLs for the path /
. For more details about ACLs creation, visit the ACLs page.
By default, the permissions used - for an authenticated user - when running Nexus Delta are the ones defined on the JVM property app.permissions.minimum. In order to change that behaviour, please create some ACLs for the path /
. For more details about ACLs creation, visit the ACLs page.
Blue Brain Nexus can be used from Jupyter Notebooks using Nexus Forge or Nexus Python SDK. Alternatively, you can also use any Python HTTP client and use Nexus REST API directly from the Jupyter Notebook. Please consider looking at our tutorial to learn how to user Nexus Forge on the Sandbox. Other examples are provided in the folder Notebooks.
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Blue Brain Nexus Forge is a domain-agnostic, generic and extensible Python framework enabling non-expert users to create and manage knowledge graphs by making it easy to:
@@ -271,7 +278,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Nexus Fusion is a server-side rendered single-page webapp powered by React.js. It is written in Typescript, and therefore must be transpiled into native browser JavaScript, during a build step, before being served.
-The build step produces a server artifact to run on a Node.js server, with client-side assets and Javascript.
-We produce a docker image that is able to serve the compiled assets directly, which is available on Dockerhub.
+Nexus Fusion is a partially server-side rendered single-page web app powered by React.js. It is written in TypeScript, and therefore must be transpiled into native browser JavaScript, during a build step, before being served.
+The build step produces a server artifact to run on a Node.js server, with client-side assets and JavaScript.
+We produce a Docker image that is able to serve the compiled assets directly, which is available on Dockerhub.
Although the application is served by a Node.js server, the client communicates directly to Nexus Delta using Nexus.js.
Nexus Fusion has undergone a significant restructuring to transition from a SubApp-based architecture to a page-based structure. This change will prepare the application for a full migration to file-system based routing technologie and help seperating the application main features.
-Pages serve a specific top level entity or functionality concerns of diverse users and activities, and provide varying access privileges to each. The main features that can be accessed from the home page are: Organizations, Projects, Studios and My data.
-The page Project, is responsible for handling all aspects of managing a single project.
-The search page has been removed, but every element that was previously available on the search page now has a link on the home page. These links redirect users to the appropriate global search type page.
-The pages in Nexus Fusion are part of the source code and reside in the src/pages
folder.
While all the previous feature is still available in the current version of Nexus Fusion, we expect its functionality to evolve and change in the upcoming release.
+Nexus Fusion has undergone a significant restructuring to transition from a SubApp-based architecture to a page-based structure. This change will prepare the application for a full migration to file-system based routing system and help separate the application’s main features.
+Pages serve a specific top level entity or functionality concerns of diverse users and activities, and provide varying access privileges to each. The main features that can be accessed from the home page are: Organizations, Projects, Studios and My data.
+The page Project, is responsible for handling all aspects of managing a single project.
The search page has been removed, but every element that was previously available on the search page now has a link on the home page. These links redirect users to the appropriate global search type page.
The pages in Nexus Fusion are part of the source code and reside in the src/pages
folder. While all the previous features are still available in the current version of Nexus Fusion, we expect their functionality to evolve and change in the upcoming releases.
Plugins are ways to render resources. You can find more about them here. It is important to note that the plugin repository is hosted separately from Nexus Fusion. Nexus Fusion will request a Plugin Manifest from this repository at run-time, and fetch plugins to render during run time based on a config. Both the plugins, the configuration, and the manifest should be hosted somewhere Nexus Fusion can request it.
You can customize the Header of Nexus Fusion by setting up the following environment variables:
+Customize the Nexus codebase appearance by setting environment variables. Defaults are provided by Nexus Fusion if not set.
+LOGO_IMG
: HTTPS URL for the application logo. Recommended: SVG format with transparent background. Ideal size: ~35 px height, max 250 px width. Click here for an example SVG.LOGO_LINK
: HTTPS URL redirecting from the logo. Example: https://www.epfl.chLANDING_VIDEO
: HTTPS URL for a landing page video. Requirements: MP4 format, H.264 codec, ~10MB, 1920×1080 resolution. Click here for an example video. Ensure that the video’s main color is not too bright, as the text on top of it will be white.LANDING_POSTER_IMG
: HTTPS URL for a loading image on the landing page, displayed while the video loads. Click here for an example poster image. Please use a PNG or JPG image that matches the size and color of the video. Aim for the image to have the same dimensions as the video. Ensure that the image size is kept under 200 KB to improve loading time and user experience.MAIN_COLOR
: Main background color in hex code. Example color: #062d68+
ORGANIZATION_IMG
: Provide the HTTPS URL for an image on the organization page. Click here for an example image. The recommended dimensions are approximately 1500 × 450 pixels. Please ensure that the image size is kept under 200 KB to improve loading time and user experience.PROJECTS_IMG
: Share the HTTPS URL for an image on the projects page. Click here for an example image. The recommended dimensions are approximately 1500 × 450 pixels. Please ensure that the image size is kept under 200 KB to improve loading time and user experience.STUDIOS_IMG
: Provide the HTTPS URL for an image on the studios page. Click here for an example image. The recommended dimensions are approximately 1500 × 450 pixels. Please ensure that the image size is kept under 200 KB to improve loading time and user experience.LOGO_IMG
: Url for an image to be used as application logo in the Header, for example, https://www.epfl.ch/logo-img.png
LOGO_LINK
: Url for the logo, for example, https://www.epfl.ch
ORGANIZATION_IMG
: Url for the organization page foreground image, for example, https://www.epfl.ch/default-org-img.png
PROJECTS_IMG
: Url for the projects page foreground image, for example,https://www.epfl.ch/default-projects-img.png
STUDIOS_IMG
: Url for the studios page foreground image, for example, https://www.epfl.ch/default-studios-img.png
LANDING_VIDEO
: Url for video in the the landing page, for example, https://www.epfl.ch/landing-page-video.mp4
LANDING_POSTER_IMG
: Url for the video’s poster image in landing page (replace the video when loading, for example,https://www.epfl.ch/landing-page-poster-img.png
MAIN_COLOR
: Url for the organization page, for example “#062d68”Forge templates
button by providing the url as FORGE_LINK
, for example, https://some-url.hi
FORGE_LINK
: HTTPS URL for the Forge templates
button (only if using Nexus Forge). Example: https://some-url.comThe full list of environment variables can be found here.
+For a comprehensive list of environment variables, see the Environment Variables List.
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Nexus Fusion is the web interface of Blue Brain Nexus, powered by Nexus Delta. It offers prebuilt data visualization, querying, data manipulation, and administration capabilities to help integrate, edit and visualize data and resources inside Nexus Delta projects.
@@ -292,7 +299,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+The My Data page within the Nexus Fusion platform serves as a central repository for all datasets that have been created or updated by the logged in user. This feature enables users to easily access and manage the data that they are interested in, without the need to navigate through irrelevant data.
@@ -301,7 +308,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+The Nexus Fusion platform provides users with the ability to manage a high-level entity that encompasses all projects within their ecosystem. This feature is accessible through the organizations page, which presents users with a comprehensive overview of all the organizations managed within a given Blue Brain Nexus deployment. Users who possess the necessary permissions are also able to create new organizations through this interface.
@@ -293,7 +300,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+A resource returned by the Nexus Delta API is usually a JSON-LD document. By default, Nexus Fusion displays JSON-LD in a code editor. If the user has edit right to the document, they can update the resource in the editor.
@@ -443,7 +450,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+The Nexus Fusion project page allows you to manage your project in a given Blue Brain Nexus deployment. In particular, you can:
@@ -476,7 +483,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+The concept of a Project in Nexus Fusion represents a fundamental unit of data management that empowers users to effectively organize and manage a distinct set of data. The Projects page, available in two distinct modes, enables users to browse either all projects within the Nexus ecosystem or those specific to a particular organization.
@@ -266,7 +273,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Search requires Delta configuration. See the Delta Search documentation for details: Delta Search Documentation
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+A Studio
is a collection of persistent queries organized in a table layout for users to quickly access relevant data in a customizable way. Studio
authors can create a studio to match a specific topic, and create organization schemes called Workspaces and Dashboards to access various aspects of that data.
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+The Studios space within the Nexus Fusion platform provides data curators with a powerful tool to showcase their data and can effectively visualize and communicate complex data sets to a broad audience. By developing custom plugins with JavaScript, curators can tailor the presentation of query results to meet specific requirements, including the formatting of charts, graphs, and other data visualization tools.
@@ -265,7 +272,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+The Nexus ecosystem is a data catalog with several technologies and techniques used to get insight and explore the data within the ecosystem.
@@ -265,7 +272,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Nexus Delta service can be highly customized using configuration file(s). Many things can be adapted to your deployment needs: port where the service is running, timeouts, pagination defaults, etc.
+Nexus Delta service can be highly customized using configuration file(s). Many things can be adapted to your deployment needs: port where the service is running, timeouts, pagination defaults, etc.
There are 3 ways to modify the default configuration:
DELTA_EXTERNAL_CONF
which defines the path to a HOCON file. The configuration keys that are defined here can be overridden by the other methods.JAVA_OPTS
environment variable: -Xms4g -Xmx4g
. The recommended values should be changed accordingly with the usage of Nexus Delta, the number of projects and the resources/schemas size.
In order to successfully run Nexus Delta there is a minimum set of configuration flags that need to be specified
The http
section of the configuration defines the binding address and port where the service will be listening.
The http
section of the configuration defines the binding address and port where the service will be listening.
The configuration flag akka.http.server.parsing.max-content-length
can be used to control the maximum payload size allowed for Nexus Delta resources. This value applies to all posted resources except for files.
The database
section of the configuration defines the postgres specific configuration. As Nexus Delta uses three separate pools (‘read’, ‘write’, ‘streaming’), it is recommended to set the host, port, database name, username, and password via the app.defaults.database
field, as it will apply to all pools. It is however possible to accommodate more advanced setups by configuring each pool separately by changing its respective app.database.{read|write|streaming}
fields.
The database
section of the configuration defines the postgres specific configuration. As Nexus Delta uses three separate pools (‘read’, ‘write’, ‘streaming’), it is recommended to set the host, port, database name, username, and password via the app.defaults.database
field, as it will apply to all pools. It is however possible to accommodate more advanced setups by configuring each pool separately by changing its respective app.database.{read|write|streaming}
fields.
The pool size can be set using the app.defaults.database.access.pool-size
setting for all pools, or individually for each pool (app.database.{read|write|streaming}.access.pool-size
).
A default Postgres deployment will limit the number of connections to 100, unless configured otherwise. See the Postgres Connection and Authentication documentation.
Before running Nexus Delta, the expected tables should be created.
+Before running Nexus Delta, the init scripts should be run in the lexicographical order.
It is possible to let Nexus Delta automatically create them using the following configuration parameters: app.database.tables-autocreate=true
.
Auto creation of tables is included as a development convenience and should be avoided in production.
The underlying Apache Jena parser used to validate incoming data is configurable using the json-ld-api
field to enable different levels of strictness.
The underlying Apache Jena parser used to validate incoming data is configurable using the json-ld-api
field to enable different levels of strictness.
Nexus Delta uses a service account to perform automatic tasks under the hood. Examples of it are:
The service-account
section of the configuration defines the service account configuration.
The service-account
section of the configuration defines the service account configuration.
Automatic project provisioning allows to create a dedicated project for users the first time they connect to Delta that is to say the first time, they query the project listing endpoints.
The generated project label will be:
@@ -363,49 +368,90 @@automatic-provisioning
section of the configuration defines the project provisioning configuration.
-Nexus Delta uses symmetric encryption to secure sensitive data information (tokens and passwords).
-The encryption
section of the configuration defines the encryption configuration.
The automatic-provisioning
section of the configuration defines the project provisioning configuration.
When fetching a resource, Nexus Delta allows to return a redirection to its representation in Fusion by providing text/html
in the Accept
header.
The fusion
section of the configuration defines the fusion configuration.
The fusion
section of the configuration defines the fusion configuration.
Projections in Nexus Delta are asynchronous processes that can replay the event log and process this information. For more information on projections, please refer to the Architecture page.
-The projections
section of the configuration allows to configure the projections.
The projections
section of the configuration allows to configure the projections.
In case of failure in a projection, Nexus Delta records the failure information inside the public.failed_elem_logs
PostgreSQL table, which can be used for analysis, and ultimately resolution of the failures. The configuration allows to set how long the failure information is stored for (app.projections.failed-elem-ttl
), and how often the projection deleting the expired failures is awoken (app.projections.delete-expired-every
).
Since 1.5.0, Nexus Delta supports plugins. Jar files present inside the local directory defined by the DELTA_PLUGINS
environment variable are loaded as plugins into the Delta service.
Each plugin configuration is rooted under plugins.{plugin_name}
. All plugins have a plugins.{plugin_name}.priority
configuration flag used to determine the order in which the routes are handled in case of collisions.
For more information about plugins, please refer to the Plugins page.
The elasticsearch plugin configuration can be found here.
+The elasticsearch plugin configuration can be found here.
The most important flags are: * plugins.elasticsearch.base
which defines the endpoint where the Elasticsearch service is running. * plugins.elasticsearch.credentials.username
and plugins.elasticsearch.credentials.password
to allow to access to a secured Elasticsearch cluster. The user provided should have the privileges to create/delete indices and read/index from them.
Please refer to the Elasticsearch configuration which describes the different steps to achieve this.
The blazegraph plugin configuration can be found here.
+The blazegraph plugin configuration can be found here.
The most important flag is plugins.blazegraph.base
which defines the endpoint where the Blazegraph service is running.
The plugins.blazegraph.slow-queries
section of the Blazegraph configuration defines what is considered a slow Blazegraph query, which will get logged in the public.blazegraph_queries
PostgreSQL table. This can be used to understand which Blazegraph queries can be improved.
The plugins.blazegraph.slow-queries
section of the Blazegraph configuration defines what is considered a slow Blazegraph query, which will get logged in the public.blazegraph_queries
PostgreSQL table. This can be used to understand which Blazegraph queries can be improved.
The composite views plugin configuration can be found here.
+The composite views plugin configuration can be found here.
There are several configuration flags related to tweaking the range of values allowed for sources, projections and rebuild interval.
+Authentication for remote sources can be specified in three different ways. The value of plugins.composite-views.remote-source-credentials
should be speficied in the same way as remote storages, as shown here
The storage plugin configuration can be found here.
+The storage plugin configuration can be found here.
Nexus Delta supports 3 types of storages: ‘disk’, ‘amazon’ (s3 compatible) and ‘remote’.
plugins.storage.storages.disk.default-volume
, which defines the default location in the Nexus Delta filesystem where the files using that storage are going to be saved.plugins.storage.storages.amazon.default-endpoint
, plugins.storage.storages.amazon.default-access-key
and plugins.storage.storages.amazon.default-secret-key
.plugins.storage.storages.remote-disk.default-endpoint
(the endpoint where the remote storage service is running) and plugins.storage.storages.remote-disk.default-credentials
(the Bearer token to authenticate to the remote storage service).plugins.storage.storages.remote-disk.default-endpoint
(the endpoint where the remote storage service is running) and plugins.storage.storages.remote-disk.credentials
(the method to authenticate to the remote storage service).When the media type is not provided by the user, Delta relies on automatic detection based on the file extension in order to provide one.
+From 1.9, it is possible to provide a list of extensions with an associated media type to compute the media type.
+This list can be defined at files.media-type-detector.extensions
:
files {
+ # Allows to define default media types for the given file extensions
+ media-type-detector {
+ extensions {
+ custom = "application/custom"
+ ntriples = "application/n-triples"
+ }
+ }
+}
+
+The media type resolution process follow this order stopping at the first successful step:
+Content-Type
header from the file creation/update requestapplication/octet-stream
Authentication for remote storage can be specified in three different ways. The value of plugins.storage.storages.remote-disk.credentials
can be:
{
+ type: "client-credentials"
+ user: "username"
+ password: "password"
+ realm: "internal"
+}
+
+This configuration tells Delta to log into the internal
realm (which should have already been defined) with the user
and password
credentials, which will give Delta an access token to use when making requests to remote storage
{
+ type: "anonymous"
+}
+
+{
+ type: "jwt-token"
+ token: "long-living-auth-token"
+}
+
The archive plugin configuration can be found here.
+The archive plugin configuration can be found here.
The Jira plugin configuration can be found here.
+The Jira plugin configuration can be found here.
Setting up the Jira plugin needs to set up the endpoint of the Jira instance but also the consumer key, the consumer secret and the private key required to interact with Jira (more details including the configuration steps in Jira here).
For monitoring, Nexus Delta relies on Kamon.
Kamon can be disabled by passing the environment variable KAMON_ENABLED
set to false
.
Delta configuration for Kamon is provided in the monitoring
section. For a more complete description on the different options available, please look at the Kamon website.
Delta configuration for Kamon is provided in the monitoring
section. For a more complete description on the different options available, please look at the Kamon website.
Delta provides the Kamon instrumentation for:
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+If you wish to quickly try out Nexus, we provide a public sandbox. For a more in-depth test-drive of Nexus on your machine, we recommend the Docker Compose approach. For a production deployment on your in-house or cloud infrastructure, please refer to our deployment guide.
@@ -410,19 +417,19 @@$ curl http://localhost/v1/version | jq
{
"@context": "https://bluebrain.github.io/nexus/contexts/version.json",
- "delta": "1.8.0",
+ "delta": "1.9.0",
"dependencies": {
"blazegraph": "2.1.6-SNAPSHOT",
- "elasticsearch": "8.8.0",
- "postgres": "15.3"
+ "elasticsearch": "8.11.1",
+ "postgres": "15.5"
},
"environment": "dev",
"plugins": {
- "archive": "1.8.0",
- "blazegraph": "1.8.0",
- "composite-views": "1.8.0",
- "elasticsearch": "1.8.0",
- "storage": "1.8.0"
+ "archive": "1.9.0",
+ "blazegraph": "1.9.0",
+ "composite-views": "1.9.0",
+ "elasticsearch": "1.9.0",
+ "storage": "1.9.0"
}
}
@@ -571,7 +578,7 @@ +You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Nexus provides global search functionality across all projects through the search plugin.
Nexus provides global search functionality across all projects through the search plugin.
The search plugin is experimental and its functionality and API can change without notice.
There are several aspects that have been taken into consideration when adding global search capabilities in Nexus:
This completes the configuration required to enable Global Search capabilities into Nexus. It may seem quite cumbersome, but all the configuration options presented are required for a good end-user experience. A different, more complex configuration example can be found in the test project.
+This completes the configuration required to enable Global Search capabilities into Nexus. It may seem quite cumbersome, but all the configuration options presented are required for a good end-user experience. A different, more complex configuration example can be found in the test project.
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+In this tutorial, you will use the core features of the Nexus ecosystem through our sandbox. This requires minimal technical knowledge but the ability to install a Python library and run a jupyter notebook.
@@ -331,9 +338,9 @@A jupyter notebook is available for this part of the tutorial and can be spawn easily using Google Colab, binder, or locally:
For local execution, Nexus Forge can be installed using these instructions. Make sure that the jupyter notebook|lab is launched in the same virtual environment where Nexus Forge is installed. Alternatively, set up a specialized kernel.
If you want to try some other examples of Nexus Forge, you can use these notebooks.
@@ -381,8 +388,8 @@The goal of this notebook is to learn how to connect to an Elasticsearch view and run queries against it.
@@ -392,8 +399,8 @@In this tutorial, we demonstrate how to consume structured data published on the web according to the Linked data principles to extend and enrich a knowledge graph.
@@ -409,8 +416,8 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Welcome to our Nexus tutorial! Nexus is an open-source data and metadata management suite. With Nexus, your data is catalogued and indexed as a knowledge graph, all interconnected.
@@ -392,7 +399,7 @@This tutorial is also part of the edX EPFL Simulation Neuroscience MOOC.
In Step 0, you’ll learn to setup your Python environment if you want to run the Jupyter notebooks locally. You can skip this step if you use Binder or Google Colaboratory (preferred).
In Step 1, you’ll learn about the Nexus Sandbox deployment, a dedicated environment for this tutorial. You’ll use Nexus Fusion to login and access your dedicated project instantiated in Nexus Delta.
@@ -494,11 +501,11 @@In the code available, we will integrate data from two sources: the AIBS and MouseLight (see Step 3). We will provide mappers for both data sources. You can check the mappers directly on GitHub. There are two mappers for the AIBS, one for neuron morphologies and another for electrophysiology data. For MouseLight, there’s only one mapper, as both morphologies and traces are in the same data source.
+In the code available, we will integrate data from two sources: the AIBS and MouseLight (see Step 3). We will provide mappers for both data sources. You can check the mappers directly on GitHub. There are two mappers for the AIBS, one for neuron morphologies and another for electrophysiology data. For MouseLight, there’s only one mapper, as both morphologies and traces are in the same data source.
To run the notebook locally, open your terminal, clone the Nexus repository, go to the notebook directory, and run Jupyter:
@@ -533,8 +540,8 @@To run the notebook locally, open your terminal, clone the Nexus repository, go to the notebook directory, and run Jupyter:
@@ -937,8 +944,8 @@To run the notebook locally, open your terminal, clone the Nexus repository, go to the notebook directory, and run Jupyter:
@@ -954,7 +961,7 @@jupyter notebook MOOC_Content_based_Recommender_System_using_Blue_Brain_Nexus.ipynb
Well done! You have now completed the last part of this tutorial. To learn more, scroll down or navigate to our documentation, or start contributing to our Github repositories.
-If you have reached this tutorial via the Simulation Neuroscience MOOC, you can now head back to the edX platform and complete this week assignment.
+If you have reached this tutorial via the Simulation Neuroscience MOOC, you can now head back to the edX platform and complete this week assignment.
Nexus can be used to manage more than neuroscience data. If you want to try it, head over to our MovieLens Tutorial!
@@ -966,7 +973,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+This section will help you understand knowledge graphs and related standards and technologies.
@@ -424,7 +431,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Blue Brain Nexus is an ecosystem that allows you to organize and better leverage your data through the use of a Knowledge Graph. In addition to the products listed here, you’ll find a rich ecosystem of libraries and tools.
@@ -325,7 +332,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+This section of the documentation lists the significant BlueBrain Nexus releases across all services and web applications.
The latest stable release is v1.8.0 released on 14.06.2023.
+_
anymore@type
and project
_self
A detailed list of changes included in the release can be found in the release notes.
@@ -591,152 +604,8 @@settings
(this allows the customization of ElasticSearch index with number of shards and replicas, tokenizers, filters etc.).
A detailed list of changes included in the release can be found in the release notes.
-This is a bugfix release, addressing the following issues related to views lifecycle:
-This is a bugfix release, addressing the following issues:
-The release is backwards compatible with v1.x.y releases in terms of API. If you’re upgrading from v1.3.x please visit the migration instructions.
-Summary of the significant changes:
-delta
;A detailed list of changes included in the release can be found in the release notes.
-The release is backwards compatible with v1.x.y releases in terms of API. If you’re upgrading from v1.2.x please visit the migration instructions.
-Summary of the significant changes:
-.../offset
that presents the current view offset, or collection of offsets in case of CompositeViews. The offset has the same value used with Server Sent Events as means of keeping track of the current event replay progress. Deleting this resource with instruct the system to rebuild the indices of the selected view;sort
query param that accepts ElasticSearch document field names (...?sort=_createdAt&sort=-_createdBy
). The ordering defaults to ascending, but can be switched for descending by prefixing the -
character to the field name.A detailed list of changes included in the release can be found in the release notes.
-This is a bugfix release, addressing two specific issues:
-FileAttributesUpdated
event deserialization which was causing indexing problems for situations where the deployment included a remote storage service to handle files.kamon-akka-remote
dependency which was causing problems in clustered deployments due to binary compatibility issues.The release adds two major features:
-The API is backwards compatible with v1.1.
-Summary of the significant changes:
-Storage service related updates:
-KG updates:
-DigestViewCoordinator
to AttributesViewCoordinator
. This async process now updates all the FileAttributes.Dependency updates:
-The release addresses bug fixing and is backwards compatible with v1.0 in terms of API. If you’re upgrading from v1.0 please visit the migration instructions.
-Summary of the significant changes:
-Storage service related updates:
-KG Fixes:
-ProjectViewCoordinator
and DigestViewCoordinator
actors now create child actors (better management of actors lifecycle).Fixed library dependency issues:
-The release addresses bug fixing and is backwards compatible with v1.0 in terms of API. If you’re upgrading from v1.0 please visit the migration instructions.
-Summary of the significant changes:
-The release is backwards compatible with v1.0 in terms of API. If you’re upgrading from v1.0 please visit the migration instructions.
-Summary of the significant changes:
-A detailed list of changes included in the release can be found in the release notes.
-This is the first major release of Blue Brain Nexus after almost two years of development.
-Also referred to as “Nexus V1”, this initial release is our first big milestone in our quest to build a Knowledge Graph platform uniquely combining flexible graph database, powerful search engine and scalable data store to enable:
-A detailed list of changes included in the release can be found in the release notes.
+The release notes of older versions are available here.
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+The v1.5 release of Delta includes a lot of improvements like storing the compacted/expanded forms of resources to get immutability and improve performance.
@@ -356,7 +363,7 @@ nodetool repair
Create a materialized view on the messages table on the original keyspace delta_1_4
:
Create a materialized view on the messages table on the original keyspace delta_1_4
:
CREATE MATERIALIZED VIEW {delta_1_4}.ordered_messages AS
SELECT ser_manifest, ser_id, event
FROM delta.messages
@@ -722,7 +729,7 @@
Once migration is completed and the previous checks have been performed, Delta can be restarted in normal mode.
-The previous Delta instance can be stopped. After the restart, a complete reindexing of the data in elasticsearch/blazegraph will take place.
+The previous Delta instance can be stopped. After the restart, a complete reindexing of the data in elasticsearch/blazegraph will take place.
Check the statistics views for different views, they should progressively tend to get to 0 remaining events.
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+@@ -327,15 +334,15 @@<
Plugins support painless extensibility and evolution of different features of the Nexus Delta API while keeping isolation and stability to the rest of the feature set.
In order to adopt and validate this new design, we have identified a set features that have been implemented as plugins:
-
- archives API Reference | code
-- SPARQL views API Reference | code
-- Elasticsearch views API Reference | code
-- composite views API Reference | code
-- files and storages API Reference | code
+- archives API Reference | code
+- SPARQL views API Reference | code
+- Elasticsearch views API Reference | code
+- composite views API Reference | code
+- files and storages API Reference | code
If you’re interested into building your own plugin, please visit the plugins section.
JSON-LD
+Note-Please read carefully this section. Some changes introduced might have breaking consequences on clients consuming the Nexus Delta API if, and only if, your client does not already use a JSON-LD aware library
Please read carefully this section. Some changes introduced might have breaking consequences on clients consuming the Nexus Delta API if, and only if, your client does not already use a JSON-LD aware library
JSON-LD has been our primary representation format since the beginnings of Nexus Delta. It is a powerful format that allows clients to use simple JSON while it adds support for Linked Data, opening the door to work with knowledge graphs.
Our previous JSON-LD support was, in some cases, inconsistent with the specification. We have fixed that, introducing the following changes:
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+The v1.6 release of Delta includes a change on how project statistics are computed which requires to run a query against the Cassandra primary store.
@@ -291,7 +298,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+@@ -359,7 +366,7 @@Secure the Delta Elasticsearch client
With 1.6, it is now possible to define credentials to query a secured Elasticsearch cluster: Elasticsearch configuration.
Dedicated Blazegraph client for user-defined queries
-User-defined queries could bring instability to Blazegraph because of their complexity. In 1.6, a dedicated client with a timeout which triggers a circuit breaker in Blazegraph has been introduced in order to solve that problem.
+User-defined queries could bring instability to Blazegraph because of their complexity. In 1.6, a dedicated client with a timeout which triggers a circuit breaker in Blazegraph has been introduced in order to solve that problem.
Synchronous indexing
From 1.6, the different types of resources can be indexed directly after creation/modification without waiting for the background indexing process to pick it up.
Graph analytics
@@ -368,7 +375,7 @@Listings
The different types of resources can now be listed at the organization level and within all the projects the current user has access to.
Configurable RDF parser
-The RDF parser allowing to validate incoming data can now be configured in order to be more or less strict.
+The RDF parser allowing to validate incoming data can now be configured in order to be more or less strict.
Automatic project provisioning
When enabled, a dedicated project is created for the current user on its first access to the Nexus platform.
How to enable and configure it is detailed here
@@ -383,7 +390,7 @@Nexus Fusion
Features
-
- Revamped Fusion search. Fusion search will now use delta search end point and will provide a improved UI with sorting, filtering, pagination etc.
+- Revamped Fusion search. Fusion search will now use delta search end point and will provide a improved UI with sorting, filtering, pagination etc.
- Real-time updates. Fusion now uses ‘indexing=sync’ option in delta API. This means all the updates made through fusion will be indexed in delta synchronously.
- Sub apps other than studio will be hidden in sidebar for anonymous users.
- Fusion will show a warning message when projects are due for deletion.
@@ -402,7 +409,7 @@-v1.8.x +Snapshot
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
++You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+@@ -433,7 +440,7 @@-v1.8.x +Snapshot
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+The v1.8 release of Delta introduces PostgreSQL as sole option for the Delta primary event store.
@@ -453,12 +460,12 @@Run the commands from the Delta 1.8.x schema.ddl file to create the tables necessary for Delta operations.
+Run the commands from the Delta 1.8.x schema scripts to create the tables necessary for Delta operations.
For the migration, there are two ways to make sure that the new Delta 1.8 indices do not overlap the ones from Delta 1.7 (for both Elasticsearch and Blazegraph):
app.defaults.indexing.prefix
field, which sets the default prefix value for both Elasticsearch and Blazegraph indices.app.defaults.indexing.prefix
field, which sets the default prefix value for both Elasticsearch and Blazegraph indices.Elasticsearch defaults to a maximum of 1000 shards per node in the cluster. If you intend to keep both old and new indices for the time of the migration, ensure that you can double your amount of shards within this limit. It is also possible to raise this limit by configuring cluster.max_shards_per_node
. In both cases, monitor your resource usage, and consider allocating more resources to Elasticsearch for the time of the migration.
Blazegraph stores data using a journaling system, which is append-only. Consider backing up your existing Blazegraph journal (by making a copy of the blazegraph.jnl
file), and running with a clean instance of Blazegraph. This can allow a significant reduction in the size of the journal as only the latest state will be saved.
Delta 1.8 contains several bug fixes and improvements that result in differences in the Blazegraph counts. As such it is expected to find differences in the migration_blazegraph_count
table. Additionally, the default Blazegraph query timeout is 1 minute, so that the count may not be able to complete for large projects.
Delta 1.8 contains several bug fixes and improvements that result in differences in the Blazegraph counts. As such it is expected to find differences in the migration_blazegraph_count
table. Additionally, the default Blazegraph query timeout is 1 minute, so that the count may not be able to complete for large projects.
The calls must be authenticated using the same service account as the one defined in Delta.
Before running the tests, the following command needs to be run in your PostgreSQL instance:
CREATE TABLE IF NOT EXISTS public.migration_resources_diff_offset(
@@ -847,7 +854,7 @@
-v1.8.x
+Snapshot
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+@@ -390,7 +397,7 @@Views
Better reporting of indexing errors
Indexing errors for the different views are now available as Server Sent Events via the API making it easier to identify and fix them by either updating the resource or updating the view.
- +Errors related to indexing into Elasticsearch have also been improved to give more precise and relevant feedback.
Only trigger reindexing when indexing is impacted
Indexing views will now only restart if its new configuration changes the way, data is impacted. For example, updating the name, the description or the permission of the view won’t trigger reindexing anymore.
@@ -409,7 +416,7 @@More information for resources More information for schemas
Validate
This new validate operation allows to check a resource against a provided schema. This operation is read-only and allows to test a schema against different resources
- +Annotated original payload
It is now possible to fetch in a single the original payload of a resource alongside its metadata: More information
Supervision API
@@ -508,7 +515,7 @@-v1.8.x +Snapshot
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Reindexing can consume a lot of CPU/memory if many views are indexing in parallel, please proceed by restarting only a subset of them at a time so as to not overload the system.
As those links are not shortened anymore (see here), we encourage Nexus deployments moving to 1.9 to reindex your different views containing metadata.
+This operation can be performing by deleting the offset for the different views by using the different endpoints:
+ +Nexus Fusion also allows to restart the indexing for the different views.
+The introduction of partial reindexing for composite views required a model change of composite views.
+If your Nexus deployment relies on composite views or search, the following environment variable need to be set when launching Delta v1.9 for the first time:
+MIGRATE_COMPOSITE_VIEWS
: trueThe following logs should appear in Delta:
+2023-08-24 07:38:24 INFO c.e.b.n.d.p.c.m.MigrateCompositeViews - Starting composite views migration
+2023-08-24 07:38:28 INFO c.e.b.n.d.p.c.m.MigrateCompositeViews - '500' composite views events have been migrated
+2023-08-24 07:38:28 INFO c.e.b.n.d.p.c.m.MigrateCompositeViews - '31' composite views states have been migrated
+2023-08-24 07:38:28 INFO c.e.b.n.d.p.c.m.MigrateCompositeViews - '62' composite views states have been migrated
+2023-08-24 07:38:34 INFO c.e.b.n.d.p.c.m.MigrateCompositeViews - Composite views migration is now complete with (500,62) events/states updated
+
+After this completes, the environment variable can be removed and Delta restarted.
+As the mapping and setting of default Elasticsearch views have changed, it is necessary to update the views. For this purpose, set the following environment variable when launching Nexus Delta 1.9.0 for the first time
+RESET_DEFAULT_ES_VIEWS
: trueThe following logs should appear in Delta for each of your project:
+2023-11-28 16:30:14 INFO c.e.b.n.d.p.e.i.ElasticSearchDefaultViewsResetter - Created a new defaultElasticSearchView in project 'myorg/myproject'.
+
+Currently, both remote storages and composite views with remote sources can hold auth tokens for communications with these remote endpoints. In v1.9, we now expect authentication to be dealt with in the Delta configuration
+A remote storage might currently have a token stored in the storage definition. In 1.9, this should be moved to plugins.storage.storages.remote-disk.credentials
, as shown here
A composite view might currently have a token stored in the view definition. In 1.9, this should be moved to plugins.composite-views.remote-source-credentials
, as shown here
+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+TODO add link to full changes on github
+If you are currently using Nexus Delta from 1.8.x, please visit the migration instructions for the necessary steps to successfully upgrade.
+Those three links are returned in the response of operations related to the different resources. Until 1.9, they were curied/shortened according to the base and the API mappings defined in the project configuration.
+This caused consistency problems as whenever the base or the API mapping were changing, those shortened links were not valid anymore:
+Therefore since 1.9, _self
, _incoming
and outgoing
links are not shortened anymore allowing them to remain stable in time even if the project configuration changes.
Multiple resources can now be retrieved within a single call with the multi-fetch operation.
+ +Until 1.9, operations about resources were resulting in writes in the primary store and indexing in the knowledge graph. However, users may also want to first experiment/refine/polish/validate a resource and schema and only create/update the resource when they are happy with it.
+ +It is now forbidden for JSON payloads to contain fields beginning with underscore (_). This can be disabled be setting app.resources.decoding-option
to lenient
, however it is not recommended as specification of this data in payloads can have unexpected consequences in both data and the user-interface
Listings by types can now be controlled with the typeOperator
query parameter. Possible values are or
and and
, which will match any and all types, respectively.
It is now possible to aggregate resources by @type
or project
.
This feature is also available on files, resolvers, schemas, storages, and views.
+ +When creating/updating a resource, Nexus Delta now keeps track of the remote contexts that have been resolved during the operation.
+ +The schema of a resource can now be changed:
+More information about the dedicated endpoint
+Create and update operations on resources accept now an optional tag allowing to tag the resulting resource in the same call.
+ +During update and refresh operations if no change is detected, no new revision will be created and the last existing revision will be returned.
+This was introduced to prevent users to create a large number of revisions for resources when no change is actually introduced.
+ +Previously deprecated resources can now be undeprecated.
+ +Previously deprecated projects can now be undeprecated.
+ +Indexing errors can now be listed and filtered for a given view.
+ +To improve indexing performance, the types defined in the FilterByType pipe, Sparql View payload, or the Composite View source payload are filtered in PostgreSQL rather than in Nexus Delta. This avoids querying for data just to discard it straight away.
+The different requests to Elasticsearch are now compressed by default allowing to reduce the I/Os especially during indexing.
+New features enhance performance of indexing of composite views, as well as authentication changes for views which incorporate projects on remote Delta instances
+Construct queries to the intermediate space can now be batched.
+ +When a composite view is updated, a diff is performed and only the necessary spaces are reindexed.
+For example, if a source is added/updated/removed, all the spaces (intermediate space and projection spaces) get reindexed. However if a projection is added/updated/removed, now only the space related to this projection is reindexed which reduces the number of operations and consequently speed up the indexing.
+Preserving the intermediate space also reduces the fragmentation of the Blazegraph journal.
+ +Rather than storing an auth token in the composite view, it is now expected to specify an authentication method in the Delta configuration
+ +This field was never used so as there is no plan for it, it is now removed.
+The Elasticsearch mapping of an Elasticsearch View can be retrieved.
+ +_self
Files can be added to an archive using a _self
.
Annotated source is now available as an output format when creating an archive.
+ +resources/read
permission for archive creationCreating an archive now requires only the resources/read
permission instead of archives/write
.
Tarball archives are no longer supported due to unnecessary restrictions. ZIP is now the only allowed format and clients should send application/zip
in the Accept
header when creating archives.
Previously it was only possible to deprecate organizations at a specific revision. Now organizations containing no projects can be deleted by specifying a prune parameter: DELETE /v1/org/{label}?prune=true
Previously deprecated organizations can now be undeprecated.
+ +Storages can no longer be created with credentials that would get stored:
+accessKey
/secretKey
.credentials
.These should instead be defined in the Delta configuration.
+ +Previously deprecated files can now be undeprecated.
+ +The automatic detection of the media type can now be customized at the Delta level.
+NB: The media type provided by the user still prevails over automatic detection.
+ +Create and update operations on files accept now an optional tag allowing to tag the resulting file in the same call.
+ +The Elasticsearch views behind Graph analytics can now be queried using the _search
endpoint.
Remote contexts used during the JSON-LD resolution for a resource are now indexed in the Graph analytics views and can be found via the search endpoint
+ +It is now possible to perform ID resolution by providing only the identifier of a resource.
+ +It is now possible to query whether the current logged in user has a specific permission in a specific context.
+ +TODO
+The Nexus storage app also benefits from the changes about automatic file detection.
+ +This callback has been replaced by a local validation of the token.
+TODO
++You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+Check our GitHub Milestones for the latest updates and priorities for Nexus.
@@ -267,7 +274,7 @@+You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
++You are browsing the docs for the snapshot version of Nexus, +the latest release is available here +
+