-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Access Control Lists
Access Control Lists are rooted in the /v1/acls
collection.
diff --git a/docs/delta/api/acls-api.html b/docs/delta/api/acls-api.html index 27579ff984..e9c85e3d3a 100644 --- a/docs/delta/api/acls-api.html +++ b/docs/delta/api/acls-api.html @@ -259,7 +259,7 @@
@@ -293,13 +293,6 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Access Control Lists are rooted in the /v1/acls
collection.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-An archive is a collection of resources stored inside an archive file. The archiving format chosen for this purpose is ZIP. Archive resources are rooted in the /v1/archives/{org_label}/{project_label}/
collection.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Authentication is the process of validating that users are who they claim to be while authorization gives those users permission to access an API resource.
@@ -306,7 +299,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-When performing a request against Nexus Delta, clients can specify the desired format of the response they would like to receive. This is done through a mechanism called Content Negotiation. Nexus Delta uses the HTTP Accept Header in order to provide Content Negotiation capabilities.
@@ -305,7 +298,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Nexus Delta makes use of the HTTP status codes to report the outcome of each API call. The status codes are complemented by a consistent response data model for reporting client and system level failures.
@@ -288,7 +281,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Nexus provides a global events endpoint, which allows the users to access the stream of events for all the resources in Nexus, including ACLs, permissions, realms, etc.
@@ -337,7 +330,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Files are attachment resources rooted in the /v1/files/{org_label}/{project_label}/
collection.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Graph analytics is a feature introduced by the graph-analytics
plugin and rooted in the /v1/graph-analytics/{org_label}/{project_label}
collection.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Id Resolution allows to resolve a resource by providing only a resource identifier (the @id
value of a resource). In case there are multiple resources with the same identifier across different projects, the response provides all choices for disambiguation.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-The /v1/identities
endpoint allows user to retrieve the identities that the user has in the platform. Calling the endpoint without a token will result in only one identity returned: Anonymous
. Calling it with token should return multiple identities. There are four different types of identities:
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Nexus Delta exposes a RESTful interface over HTTP(S) for synchronous communication. The generally adopted transport format is JSON based, specifically JSON-LD. However, other response formats are supported through Content-Negotiation
@@ -392,7 +385,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-The Jira integration plugin comes from the need at BBP to have the Nexus platform and Jira to interact by linking Nexus resources and Jira issues.
@@ -354,7 +347,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-The multi-fetch operation allows to get in a given format multiple resources that can live in multiple projects.
@@ -402,7 +395,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Organizations are rooted in the /v1/orgs
path and are used to group and categorize sub-resources.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Permissions are rooted in the /v1/permissions
collection.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Projects belong to an organization
and are rooted in the corresponding /v1/projects/{org_label}
path. The purposes of projects are:
-Dplugins.project-deletion.enabled=true
-Dplugins.project-deletion.included-projects.1=".+""
and -Dplugins.project-deletion.excluded-projects.1=".+protected.+""
For details on available configuration, please visit the plugin configuration.
For details on available configuration, please visit the plugin configuration.
GET /v1/project-deletion/config
It returns the configuration of the project deletion plugin.
@@ -1340,7 +1333,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Quotas are rooted in /v1/quotas collection.
@@ -316,7 +309,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Realms are rooted in /v1/realms
collection.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Resolvers are rooted in the /v1/resolvers/{org_label}/{project_label}
collection and are used in the following scenarios:
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Generic resources are rooted in the /v1/resources/{org_label}/{project_label}/{schema_id}
collection.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Schemas are rooted in the /v1/schemas/{org_label}/{project_label}
collection. They define a set of rules and constraints using SHACL. Once those schemas are present, other resources can be created against them. Those resources won’t be successfully created unless they match the required constraints defined on the schema.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Nexus provides global search functionality across all projects through the search plugin.
Nexus provides global search functionality across all projects through the search plugin.
The search plugin is experimental and its functionality and API can change without notice.
For instructions on how to configure global search in Nexus and how it works please visit the Search configuration page.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Storages are rooted in the /v1/storages/{org_label}/{project_label}
collection and are used to describe where files are physically stored.
The Remote disk storage and it remote service implementation are now deprecated and will be removed in an upcoming release.
This storage type relies on a remote HTTP service that exposes basic file operations on an underlying POSIX file-system. This is particularly handy if your organization is running some kind of distributed network storage (such as Ceph, Gluster, GPFS, Lustre, …) that you don’t want to mount directly on the system where Nexus Delta runs.
-While there’s no formal specification for this service, you can check out or deploy our own implementation: Nexus remote storage service.
+While there’s no formal specification for this service, you can check out or deploy our own implementation: Nexus remote storage service.
In order to be able to use this storage, the configuration flag plugins.storage.storages.remote-disk.enabled
should be set to true
. More information about configuration
{
"@type": "RemoteDiskStorage",
@@ -963,7 +956,7 @@
-Snapshot
+v1.9.x
diff --git a/docs/delta/api/supervision-api.html b/docs/delta/api/supervision-api.html
index d7ec7715e9..af23ce0fcf 100644
--- a/docs/delta/api/supervision-api.html
+++ b/docs/delta/api/supervision-api.html
@@ -250,7 +250,7 @@
@@ -275,13 +275,6 @@
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-This endpoint returns information about the projections running on the current node.
@@ -348,7 +341,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Trial operations contain read-only operations designed to help users compose and validate their resources before effectively saving them in Nexus.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-A user of delta is given certain permissions. This is done using the ACLs API
@@ -314,7 +307,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-GET /v1/version
@@ -295,7 +288,7 @@ <
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-This view is an aggregate of ElasticSearchViews. The view itself does not create any index, but it references the already existing indices of the linked ElasticSearchViews.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-This view is an aggregate of SparqlViews. The view itself does not create any namespace, but it references the already existing namespaces of the linked SparqlViews.
@@ -801,7 +794,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-A Composite Sink handles the following steps of composite view indexing
@@ -418,7 +411,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-This view is composed by multiple sources
and projections
.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-This view creates an ElasticSearch index
and stores the targeted Json resources into an ElasticSearch Document.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Views are rooted in the /v1/views/{org_label}/{project_label}
collection and are used to index the selected resources into a bucket.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Pipes are the processing units of a pipeline for an Elasticsearch view.
@@ -407,8 +400,8 @@Please visit Plugins to learn about how to create/package/deploy a plugin.
Inside this plugin, you can then define additional pipes:
@@ -523,7 +516,7 @@The source code for the core pipes is available here and the associated unit tests here.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-This view creates a Blazegraph namespace
and stores the targeted resources as RDF triples into a Blazegraph instance.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Blue Brain Nexus is a collection of software components that address various organizational needs relating to data storage, management, analysis and consumption. It was designed to support the data-driven science iterative cycle at Blue Brain but its genericity allows for its use in arbitrary contexts.
@@ -312,7 +305,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-The main goal of the benchmarks is to analyze the hardware requirements for a Nexus deployment and to find potential issues and / or bottlenecks. In particular, we are most interested in the following metrics:
@@ -282,7 +275,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-For the newest benchmarks of Nexus version 1.4.2 see here.
@@ -725,7 +718,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-The tests were run against version 1.4.2 in January 2021 using Gatling version 3.5.0.
@@ -1134,7 +1127,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Blue Brain Nexus Delta is a low latency, scalable and secure system that realizes a range of functions to support data management and knowledge graph lifecycles.
@@ -274,7 +267,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Nexus Delta entities fall in two categories:
@@ -315,7 +308,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Starting from version 1.5, Delta introduces the ability to extend its functionality using plugins. Plugins enable developers to add new functionality to Nexus Delta without the need to modify Delta itself. Plugins can introduce various new functionalities, including new API endpoints and indexing capabilities.
Plugins are still an experimental feature and Delta SDKs and dependent modules(rdf, sourcing) provide no binary compatibility guarantees.
Plugins used by Delta need to be packaged as a .jar
file containing the plugin code with all its dependencies. Delta loads plugins from .jar
files located in a directory specified by DELTA_PLUGINS
environment variable.
Plugins must define exactly one class which extends PluginDef trait.
+Plugins must define exactly one class which extends PluginDef trait.
The class must define following methods:
def info: PluginDescription
-this method returns instance of PluginDescription which defines the plugin name and version.
+this method returns instance of PluginDescription which defines the plugin name and version.
def initialize(locator: Locator): Task[Plugin]
this method can be used to initialize the plugin and returns an instance of a Plugin
, which can additionally define logic to stop the plugin gracefully.
libraryDependencies += "ch.epfl.bluebrain.nexus" %% "delta-sdk" % deltaVersion % Provided
Delta uses distage library for dependency injection. Each plugin must define ModuleDef
to create instances of its own classes. All the dependencies provided by ModuleDef
s defined in Delta modules, as well as other plugins can be used here.
Delta uses distage library for dependency injection. Each plugin must define ModuleDef
to create instances of its own classes. All the dependencies provided by ModuleDef
s defined in Delta modules, as well as other plugins can be used here.
The plugin can also define instances of following traits/classes, which will be used in Delta:
/version
endpoint./resources
endpoints./version
endpoint./resources
endpoints.In order to avoid clashes between different plugins, Delta uses custom classloader to load plugin classes, which will load classes from the plugin first, then using application classloader and other plugins after that. It is therefore recommended to not include in the plugin jar any dependencies which are also provided by SDK. Libraries can be easily excluded from dependencies in sbt
:
Currently, following Delta functionality is provided by plugins:
Elasticsearch plugin is required in order to provide listings in the API, other plugins can be excluded if their functionality is not needed. All the above plugins are included in the Delta Docker image.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Yes. As long as the client supports the ability to provide a Authentication
HTTP Header (for authentication purposes) on the HTTP request, any SPARQL client should work.
By default, the permissions used - for an authenticated user - when running Nexus Delta are the ones defined on the JVM property app.permissions.minimum. In order to change that behaviour, please create some ACLs for the path /
. For more details about ACLs creation, visit the ACLs page.
By default, the permissions used - for an authenticated user - when running Nexus Delta are the ones defined on the JVM property app.permissions.minimum. In order to change that behaviour, please create some ACLs for the path /
. For more details about ACLs creation, visit the ACLs page.
Blue Brain Nexus can be used from Jupyter Notebooks using Nexus Forge or Nexus Python SDK. Alternatively, you can also use any Python HTTP client and use Nexus REST API directly from the Jupyter Notebook. Please consider looking at our tutorial to learn how to user Nexus Forge on the Sandbox. Other examples are provided in the folder Notebooks.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Blue Brain Nexus Forge is a domain-agnostic, generic and extensible Python framework enabling non-expert users to create and manage knowledge graphs by making it easy to:
@@ -278,7 +271,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
--You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Nexus Fusion is the web interface of Blue Brain Nexus, powered by Nexus Delta. It offers prebuilt data visualization, querying, data manipulation, and administration capabilities to help integrate, edit and visualize data and resources inside Nexus Delta projects.
@@ -299,7 +292,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-The My Data page within the Nexus Fusion platform serves as a central repository for all datasets that have been created or updated by the logged in user. This feature enables users to easily access and manage the data that they are interested in, without the need to navigate through irrelevant data.
@@ -308,7 +301,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-The Nexus Fusion platform provides users with the ability to manage a high-level entity that encompasses all projects within their ecosystem. This feature is accessible through the organizations page, which presents users with a comprehensive overview of all the organizations managed within a given Blue Brain Nexus deployment. Users who possess the necessary permissions are also able to create new organizations through this interface.
@@ -300,7 +293,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-A resource returned by the Nexus Delta API is usually a JSON-LD document. By default, Nexus Fusion displays JSON-LD in a code editor. If the user has edit right to the document, they can update the resource in the editor.
@@ -450,7 +443,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-The Nexus Fusion project page allows you to manage your project in a given Blue Brain Nexus deployment. In particular, you can:
@@ -483,7 +476,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-The concept of a Project in Nexus Fusion represents a fundamental unit of data management that empowers users to effectively organize and manage a distinct set of data. The Projects page, available in two distinct modes, enables users to browse either all projects within the Nexus ecosystem or those specific to a particular organization.
@@ -273,7 +266,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Search requires Delta configuration. See the Delta Search documentation for details: Delta Search Documentation
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-A Studio
is a collection of persistent queries organized in a table layout for users to quickly access relevant data in a customizable way. Studio
authors can create a studio to match a specific topic, and create organization schemes called Workspaces and Dashboards to access various aspects of that data.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-The Studios space within the Nexus Fusion platform provides data curators with a powerful tool to showcase their data and can effectively visualize and communicate complex data sets to a broad audience. By developing custom plugins with JavaScript, curators can tailor the presentation of query results to meet specific requirements, including the formatting of charts, graphs, and other data visualization tools.
@@ -272,7 +265,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-The Nexus ecosystem is a data catalog with several technologies and techniques used to get insight and explore the data within the ecosystem.
@@ -272,7 +265,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Nexus Delta service can be highly customized using configuration file(s). Many things can be adapted to your deployment needs: port where the service is running, timeouts, pagination defaults, etc.
+Nexus Delta service can be highly customized using configuration file(s). Many things can be adapted to your deployment needs: port where the service is running, timeouts, pagination defaults, etc.
There are 3 ways to modify the default configuration:
DELTA_EXTERNAL_CONF
which defines the path to a HOCON file. The configuration keys that are defined here can be overridden by the other methods.JAVA_OPTS
environment variable: -Xms4g -Xmx4g
. The recommended values should be changed accordingly with the usage of Nexus Delta, the number of projects and the resources/schemas size.
In order to successfully run Nexus Delta there is a minimum set of configuration flags that need to be specified
The http
section of the configuration defines the binding address and port where the service will be listening.
The http
section of the configuration defines the binding address and port where the service will be listening.
The configuration flag akka.http.server.parsing.max-content-length
can be used to control the maximum payload size allowed for Nexus Delta resources. This value applies to all posted resources except for files.
The database
section of the configuration defines the postgres specific configuration. As Nexus Delta uses three separate pools (‘read’, ‘write’, ‘streaming’), it is recommended to set the host, port, database name, username, and password via the app.defaults.database
field, as it will apply to all pools. It is however possible to accommodate more advanced setups by configuring each pool separately by changing its respective app.database.{read|write|streaming}
fields.
The database
section of the configuration defines the postgres specific configuration. As Nexus Delta uses three separate pools (‘read’, ‘write’, ‘streaming’), it is recommended to set the host, port, database name, username, and password via the app.defaults.database
field, as it will apply to all pools. It is however possible to accommodate more advanced setups by configuring each pool separately by changing its respective app.database.{read|write|streaming}
fields.
The pool size can be set using the app.defaults.database.access.pool-size
setting for all pools, or individually for each pool (app.database.{read|write|streaming}.access.pool-size
).
A default Postgres deployment will limit the number of connections to 100, unless configured otherwise. See the Postgres Connection and Authentication documentation.
Before running Nexus Delta, the init scripts should be run in the lexicographical order.
+Before running Nexus Delta, the init scripts should be run in the lexicographical order.
It is possible to let Nexus Delta automatically create them using the following configuration parameters: app.database.tables-autocreate=true
.
Auto creation of tables is included as a development convenience and should be avoided in production.
The underlying Apache Jena parser used to validate incoming data is configurable using the json-ld-api
field to enable different levels of strictness.
The underlying Apache Jena parser used to validate incoming data is configurable using the json-ld-api
field to enable different levels of strictness.
Nexus Delta uses a service account to perform automatic tasks under the hood. Examples of it are:
The service-account
section of the configuration defines the service account configuration.
The service-account
section of the configuration defines the service account configuration.
Automatic project provisioning allows to create a dedicated project for users the first time they connect to Delta that is to say the first time, they query the project listing endpoints.
The generated project label will be:
@@ -368,32 +361,32 @@automatic-provisioning
section of the configuration defines the project provisioning configuration.
+The automatic-provisioning
section of the configuration defines the project provisioning configuration.
When fetching a resource, Nexus Delta allows to return a redirection to its representation in Fusion by providing text/html
in the Accept
header.
The fusion
section of the configuration defines the fusion configuration.
The fusion
section of the configuration defines the fusion configuration.
Projections in Nexus Delta are asynchronous processes that can replay the event log and process this information. For more information on projections, please refer to the Architecture page.
-The projections
section of the configuration allows to configure the projections.
The projections
section of the configuration allows to configure the projections.
In case of failure in a projection, Nexus Delta records the failure information inside the public.failed_elem_logs
PostgreSQL table, which can be used for analysis, and ultimately resolution of the failures. The configuration allows to set how long the failure information is stored for (app.projections.failed-elem-ttl
), and how often the projection deleting the expired failures is awoken (app.projections.delete-expired-every
).
Since 1.5.0, Nexus Delta supports plugins. Jar files present inside the local directory defined by the DELTA_PLUGINS
environment variable are loaded as plugins into the Delta service.
Each plugin configuration is rooted under plugins.{plugin_name}
. All plugins have a plugins.{plugin_name}.priority
configuration flag used to determine the order in which the routes are handled in case of collisions.
For more information about plugins, please refer to the Plugins page.
The elasticsearch plugin configuration can be found here.
+The elasticsearch plugin configuration can be found here.
The most important flags are: * plugins.elasticsearch.base
which defines the endpoint where the Elasticsearch service is running. * plugins.elasticsearch.credentials.username
and plugins.elasticsearch.credentials.password
to allow to access to a secured Elasticsearch cluster. The user provided should have the privileges to create/delete indices and read/index from them.
Please refer to the Elasticsearch configuration which describes the different steps to achieve this.
The blazegraph plugin configuration can be found here.
+The blazegraph plugin configuration can be found here.
The most important flag is plugins.blazegraph.base
which defines the endpoint where the Blazegraph service is running.
The plugins.blazegraph.slow-queries
section of the Blazegraph configuration defines what is considered a slow Blazegraph query, which will get logged in the public.blazegraph_queries
PostgreSQL table. This can be used to understand which Blazegraph queries can be improved.
The plugins.blazegraph.slow-queries
section of the Blazegraph configuration defines what is considered a slow Blazegraph query, which will get logged in the public.blazegraph_queries
PostgreSQL table. This can be used to understand which Blazegraph queries can be improved.
The composite views plugin configuration can be found here.
+The composite views plugin configuration can be found here.
There are several configuration flags related to tweaking the range of values allowed for sources, projections and rebuild interval.
Authentication for remote sources can be specified in three different ways. The value of plugins.composite-views.remote-source-credentials
should be speficied in the same way as remote storages, as shown here
The storage plugin configuration can be found here.
+The storage plugin configuration can be found here.
Nexus Delta supports 3 types of storages: ‘disk’, ‘amazon’ (s3 compatible) and ‘remote’.
plugins.storage.storages.disk.default-volume
, which defines the default location in the Nexus Delta filesystem where the files using that storage are going to be saved.The archive plugin configuration can be found here.
+The archive plugin configuration can be found here.
The Jira plugin configuration can be found here.
+The Jira plugin configuration can be found here.
Setting up the Jira plugin needs to set up the endpoint of the Jira instance but also the consumer key, the consumer secret and the private key required to interact with Jira (more details including the configuration steps in Jira here).
For monitoring, Nexus Delta relies on Kamon.
Kamon can be disabled by passing the environment variable KAMON_ENABLED
set to false
.
Delta configuration for Kamon is provided in the monitoring
section. For a more complete description on the different options available, please look at the Kamon website.
Delta configuration for Kamon is provided in the monitoring
section. For a more complete description on the different options available, please look at the Kamon website.
Delta provides the Kamon instrumentation for:
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-If you wish to quickly try out Nexus, we provide a public sandbox. For a more in-depth test-drive of Nexus on your machine, we recommend the Docker Compose approach. For a production deployment on your in-house or cloud infrastructure, please refer to our deployment guide.
@@ -578,7 +571,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Nexus provides global search functionality across all projects through the search plugin.
Nexus provides global search functionality across all projects through the search plugin.
The search plugin is experimental and its functionality and API can change without notice.
There are several aspects that have been taken into consideration when adding global search capabilities in Nexus:
This completes the configuration required to enable Global Search capabilities into Nexus. It may seem quite cumbersome, but all the configuration options presented are required for a good end-user experience. A different, more complex configuration example can be found in the test project.
+This completes the configuration required to enable Global Search capabilities into Nexus. It may seem quite cumbersome, but all the configuration options presented are required for a good end-user experience. A different, more complex configuration example can be found in the test project.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-In this tutorial, you will use the core features of the Nexus ecosystem through our sandbox. This requires minimal technical knowledge but the ability to install a Python library and run a jupyter notebook.
@@ -338,9 +331,9 @@A jupyter notebook is available for this part of the tutorial and can be spawn easily using Google Colab, binder, or locally:
For local execution, Nexus Forge can be installed using these instructions. Make sure that the jupyter notebook|lab is launched in the same virtual environment where Nexus Forge is installed. Alternatively, set up a specialized kernel.
If you want to try some other examples of Nexus Forge, you can use these notebooks.
@@ -388,8 +381,8 @@The goal of this notebook is to learn how to connect to an Elasticsearch view and run queries against it.
@@ -399,8 +392,8 @@In this tutorial, we demonstrate how to consume structured data published on the web according to the Linked data principles to extend and enrich a knowledge graph.
@@ -416,8 +409,8 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Welcome to our Nexus tutorial! Nexus is an open-source data and metadata management suite. With Nexus, your data is catalogued and indexed as a knowledge graph, all interconnected.
@@ -501,11 +494,11 @@In the code available, we will integrate data from two sources: the AIBS and MouseLight (see Step 3). We will provide mappers for both data sources. You can check the mappers directly on GitHub. There are two mappers for the AIBS, one for neuron morphologies and another for electrophysiology data. For MouseLight, there’s only one mapper, as both morphologies and traces are in the same data source.
+In the code available, we will integrate data from two sources: the AIBS and MouseLight (see Step 3). We will provide mappers for both data sources. You can check the mappers directly on GitHub. There are two mappers for the AIBS, one for neuron morphologies and another for electrophysiology data. For MouseLight, there’s only one mapper, as both morphologies and traces are in the same data source.
To run the notebook locally, open your terminal, clone the Nexus repository, go to the notebook directory, and run Jupyter:
@@ -540,8 +533,8 @@To run the notebook locally, open your terminal, clone the Nexus repository, go to the notebook directory, and run Jupyter:
@@ -944,8 +937,8 @@To run the notebook locally, open your terminal, clone the Nexus repository, go to the notebook directory, and run Jupyter:
@@ -973,7 +966,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-This section will help you understand knowledge graphs and related standards and technologies.
@@ -431,7 +424,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Blue Brain Nexus is an ecosystem that allows you to organize and better leverage your data through the use of a Knowledge Graph. In addition to the products listed here, you’ll find a rich ecosystem of libraries and tools.
@@ -332,7 +325,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-This section of the documentation lists the significant BlueBrain Nexus releases across all services and web applications.
-The latest stable release is v1.8.0 released on 14.06.2023.
-_
anymore@type
and project
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-For archival purposes, the release notes of older versions are available here.
@@ -270,7 +263,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-The v1.5 release of Delta includes a lot of improvements like storing the compacted/expanded forms of resources to get immutability and improve performance.
@@ -750,7 +743,7 @@Check the statistics views for different views, they should progressively tend to get to 0 remaining events.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-@@ -334,11 +327,11 @@<
Plugins support painless extensibility and evolution of different features of the Nexus Delta API while keeping isolation and stability to the rest of the feature set.
In order to adopt and validate this new design, we have identified a set features that have been implemented as plugins:
-
- archives API Reference | code
-- SPARQL views API Reference | code
-- Elasticsearch views API Reference | code
-- composite views API Reference | code
-- files and storages API Reference | code
+- archives API Reference | code
+- SPARQL views API Reference | code
+- Elasticsearch views API Reference | code
+- composite views API Reference | code
+- files and storages API Reference | code
If you’re interested into building your own plugin, please visit the plugins section.
JSON-LD
diff --git a/docs/releases/v1.5-to-v1.6-migration.html b/docs/releases/v1.5-to-v1.6-migration.html index 1c8a2b0c16..ae9d9c68d4 100644 --- a/docs/releases/v1.5-to-v1.6-migration.html +++ b/docs/releases/v1.5-to-v1.6-migration.html @@ -250,7 +250,7 @@ @@ -275,13 +275,6 @@Note@@ -432,7 +425,7 @@-Snapshot +v1.9.x
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-The v1.6 release of Delta includes a change on how project statistics are computed which requires to run a query against the Cassandra primary store.
@@ -298,7 +291,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-@@ -366,7 +359,7 @@Secure the Delta Elasticsearch client
With 1.6, it is now possible to define credentials to query a secured Elasticsearch cluster: Elasticsearch configuration.
Dedicated Blazegraph client for user-defined queries
-User-defined queries could bring instability to Blazegraph because of their complexity. In 1.6, a dedicated client with a timeout which triggers a circuit breaker in Blazegraph has been introduced in order to solve that problem.
+User-defined queries could bring instability to Blazegraph because of their complexity. In 1.6, a dedicated client with a timeout which triggers a circuit breaker in Blazegraph has been introduced in order to solve that problem.
Synchronous indexing
From 1.6, the different types of resources can be indexed directly after creation/modification without waiting for the background indexing process to pick it up.
Graph analytics
@@ -375,7 +368,7 @@Listings
The different types of resources can now be listed at the organization level and within all the projects the current user has access to.
Configurable RDF parser
-The RDF parser allowing to validate incoming data can now be configured in order to be more or less strict.
+The RDF parser allowing to validate incoming data can now be configured in order to be more or less strict.
Automatic project provisioning
When enabled, a dedicated project is created for the current user on its first access to the Nexus platform.
How to enable and configure it is detailed here
@@ -390,7 +383,7 @@Nexus Fusion
Features
-
- Revamped Fusion search. Fusion search will now use delta search end point and will provide a improved UI with sorting, filtering, pagination etc.
+- Revamped Fusion search. Fusion search will now use delta search end point and will provide a improved UI with sorting, filtering, pagination etc.
- Real-time updates. Fusion now uses ‘indexing=sync’ option in delta API. This means all the updates made through fusion will be indexed in delta synchronously.
- Sub apps other than studio will be hidden in sidebar for anonymous users.
- Fusion will show a warning message when projects are due for deletion.
@@ -409,7 +402,7 @@-Snapshot +v1.9.x
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
--You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-@@ -440,7 +433,7 @@-Snapshot +v1.9.x
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-The v1.8 release of Delta introduces PostgreSQL as sole option for the Delta primary event store.
@@ -460,12 +453,12 @@Run the commands from the Delta 1.8.x schema scripts to create the tables necessary for Delta operations.
+Run the commands from the Delta 1.8.x schema scripts to create the tables necessary for Delta operations.
For the migration, there are two ways to make sure that the new Delta 1.8 indices do not overlap the ones from Delta 1.7 (for both Elasticsearch and Blazegraph):
app.defaults.indexing.prefix
field, which sets the default prefix value for both Elasticsearch and Blazegraph indices.app.defaults.indexing.prefix
field, which sets the default prefix value for both Elasticsearch and Blazegraph indices.Elasticsearch defaults to a maximum of 1000 shards per node in the cluster. If you intend to keep both old and new indices for the time of the migration, ensure that you can double your amount of shards within this limit. It is also possible to raise this limit by configuring cluster.max_shards_per_node
. In both cases, monitor your resource usage, and consider allocating more resources to Elasticsearch for the time of the migration.
Blazegraph stores data using a journaling system, which is append-only. Consider backing up your existing Blazegraph journal (by making a copy of the blazegraph.jnl
file), and running with a clean instance of Blazegraph. This can allow a significant reduction in the size of the journal as only the latest state will be saved.
Delta 1.8 contains several bug fixes and improvements that result in differences in the Blazegraph counts. As such it is expected to find differences in the migration_blazegraph_count
table. Additionally, the default Blazegraph query timeout is 1 minute, so that the count may not be able to complete for large projects.
Delta 1.8 contains several bug fixes and improvements that result in differences in the Blazegraph counts. As such it is expected to find differences in the migration_blazegraph_count
table. Additionally, the default Blazegraph query timeout is 1 minute, so that the count may not be able to complete for large projects.
The calls must be authenticated using the same service account as the one defined in Delta.
Before running the tests, the following command needs to be run in your PostgreSQL instance:
CREATE TABLE IF NOT EXISTS public.migration_resources_diff_offset(
@@ -854,7 +847,7 @@
-Snapshot
+v1.9.x
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-@@ -515,7 +508,7 @@-Snapshot +v1.9.x
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
--You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-TODO add link to full changes on github
+++This is an incremental software release that adds several improvements across the board. The API is backwards compatible with v1.8.
+With v1.9, Nexus Delta also changes its underlying runtime allowing to it to work properly, switching from Monix which is not maintained anymore to Cats Effect.
+
For the detailed list of updates in this release, see the list of addressed issues since v1.8.
If you are currently using Nexus Delta from 1.8.x, please visit the migration instructions for the necessary steps to successfully upgrade.
The different requests to Elasticsearch are now compressed by default allowing to reduce the I/Os especially during indexing.
New features enhance performance of indexing of composite views, as well as authentication changes for views which incorporate projects on remote Delta instances
+New features enhance performance of indexing of composite views, as well as authentication changes for views which incorporate projects on remote Delta instances.
Construct queries to the intermediate space can now be batched.
@@ -558,7 +555,7 @@This callback has been replaced by a local validation of the token.
TODO
+For more information about the Forge, please refer to the documentation.
-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-Check our GitHub Milestones for the latest updates and priorities for Nexus.
@@ -274,7 +267,7 @@-You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
--You are browsing the docs for the snapshot version of Nexus, -the latest release is available here -
-