diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-read-latency.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-read-latency.png deleted file mode 100644 index c5e7536759..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-read-latency.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-read-throughput.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-read-throughput.png deleted file mode 100644 index cf04646881..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-read-throughput.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-write-latency.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-write-latency.png deleted file mode 100644 index c4f0c9cea6..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-write-latency.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-write-novalidation-latency.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-write-novalidation-latency.png deleted file mode 100644 index 1cc00a2677..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-write-novalidation-latency.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-write-novalidation-throughput.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-write-novalidation-throughput.png deleted file mode 100644 index 0cf59e2cc2..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-write-novalidation-throughput.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-write-throughput.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-write-throughput.png deleted file mode 100644 index 03b562b402..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.2.1/benchmarks-write-throughput.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-read-latency.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-read-latency.png deleted file mode 100644 index cd2aa9b971..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-read-latency.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-read-throughput.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-read-throughput.png deleted file mode 100644 index d262ca4304..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-read-throughput.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-write-latency.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-write-latency.png deleted file mode 100644 index 6720897c5a..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-write-latency.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-write-novalidation-latency.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-write-novalidation-latency.png deleted file mode 100644 index 5ab36f5140..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-write-novalidation-latency.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-write-novalidation-throughput.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-write-novalidation-throughput.png deleted file mode 100644 index e804c97430..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-write-novalidation-throughput.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-write-throughput.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-write-throughput.png deleted file mode 100644 index 8c5ad37288..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/with-auth/benchmarks-write-throughput.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-read-latency.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-read-latency.png deleted file mode 100644 index 4f9ad98155..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-read-latency.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-read-throughput.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-read-throughput.png deleted file mode 100644 index fd12ecb9c2..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-read-throughput.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-write-latency.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-write-latency.png deleted file mode 100644 index c65a303cb6..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-write-latency.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-write-novalidation-latency.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-write-novalidation-latency.png deleted file mode 100644 index 980d1e98be..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-write-novalidation-latency.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-write-novalidation-throughput.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-write-novalidation-throughput.png deleted file mode 100644 index 00f065c612..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-write-novalidation-throughput.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-write-throughput.png b/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-write-throughput.png deleted file mode 100644 index e1e7cac353..0000000000 Binary files a/docs/src/main/paradox/docs/delta/benchmarks/assets/v1.4.2/without-auth/benchmarks-write-throughput.png and /dev/null differ diff --git a/docs/src/main/paradox/docs/delta/benchmarks/index.md b/docs/src/main/paradox/docs/delta/benchmarks/index.md deleted file mode 100644 index e8046b1e30..0000000000 --- a/docs/src/main/paradox/docs/delta/benchmarks/index.md +++ /dev/null @@ -1,24 +0,0 @@ -@@@ index -* @ref:[Benchmarks v1.4.2](v1.4.2.md) -* @ref:[Benchmarks v1.2.1](v1.2.1.md) -@@@ - -# Benchmarks - -The main goal of the benchmarks is to analyze the hardware requirements for a Nexus deployment and to find potential -issues and / or bottlenecks. In particular, we are most interested in the following metrics: - -* **throughput** - how many requests per second the system can handle -* **latency** - the time the system needed to provide response to the requests - -... and how they were affected by different factors, especially: - -* **data volume** - how does the volume of the data in the system affect the performance -* **hardware configuration and scalability** - does assigning more hardware increase the performance of the system and - can the system scale both horizontally and vertically. -* **clustering** - what are the effects of changing from a single node to clustered deployment, as well as, what are the effects of adding more nodes to the cluster. - -The test scenarios and scripts can be found in the @link:[nexus-benchmarks](https://github.com/BlueBrain/nexus-benchmarks){ open=new } -repository. - -The latest benchmarks were run against Nexus Delta v1.4.2, see @ref:[benchmarks](v1.4.2.md). \ No newline at end of file diff --git a/docs/src/main/paradox/docs/delta/benchmarks/v1.2.1.md b/docs/src/main/paradox/docs/delta/benchmarks/v1.2.1.md deleted file mode 100644 index c464542c75..0000000000 --- a/docs/src/main/paradox/docs/delta/benchmarks/v1.2.1.md +++ /dev/null @@ -1,196 +0,0 @@ -# Benchmarks v1.2.1 - -For the newest benchmarks of Nexus version 1.4.2 see @ref[here](v1.4.2.md). - -The tests were run against version 1.2.1 in October 2019 using @link:[Gatling](https://gatling.io/){ open=new } version 3.2.1. - -# Environment - -The system was deployed on AWS EKS using the following basic configuration: - -* 8 compute nodes of 16 vCPU, 64 GB RAM, 2 x 300 GB NVMe disks -* nginx ingress controller -* 1 Keycloak pod using MariaDB as its storage -* 6 Cassandra pods each with 4 vCPU, 6 GB HEAP, local NVMe storage, replication factor of 3 -* 3 Elasticsearch pods each with 4 vCPU, 8 GB HEAP, local NVMe storage -* 1 Blazegraph pod with 8 vCPU, 16 GB HEAP, local NVMe storage -* 1 Prometheus pod and 1 Jaeger pod - -The services were deployed as follows: - -* 3 pod IAM cluster, each node using 4 vCPU, 4 GB HEAP -* 3 pod Admin cluster, each node using 4 vCPU, 4 GB HEAP -* 1 to 6 pods KG cluster, each node with 8 vCPU, 8 GB HEAP - -# Data volume - -Prior to the test execution a reasonable amount of data was injected into the system to ensure the system behaves well -under a typical volume, specifically 120,000,000 resources were created across 27 projects using an exponential -distribution. The total number of triples (22 for each resource + 11 system metadata) was approximately: 4,000,000,000. - -The following resource template was used for both pre-created resources and the resources created during the tests: -``` -{ - "@type": [ - "http://www.w3.org/ns/prov#Activity", - "https://neuroshapes.org/StimulusExperiment" - ], - "http://www.w3.org/ns/prov#used": { - "@id": "https://nexus-sandbox.io/neurosciencegraph/data/049b569d-98ba-4a54-a9cb-45f5c5cd3b40", - "@type": [ - "http://www.w3.org/ns/prov#Entity", - "https://neuroshapes.org/PatchedCell" - ], - "http://schema.org/name": "Some PatchedCell" - }, - "http://www.w3.org/ns/prov#wasAssociatedWith": { - "@id": "https://nexus-sandbox.io/v1/realms/random/users/someuser", - "@type": "http://www.w3.org/ns/prov#Agent" - }, - "https://neuroshapes.org/hadProtocol": { - "@id": "https://nexus-sandbox.io/neurosciencegraph/data/3f189851-c3de-470d-894e-68abe116bfe4", - "@type": [ - "https://neuroshapes.org/Protocol", - "https://neuroshapes.org/ExperimentalProtocol", - "http://www.w3.org/ns/prov#Entity" - ], - "http://schema.org/name": "Some Protocol" - }, - "https://neuroshapes.org/stimulus": { - "https://neuroshapes.org/stimulusType": { - "@id": "http://stimulusont.org/H20S8", - "http://www.w3.org/2000/01/rdf-schema#label": "H20S8" - } - } -} -``` - -# Scenarios - -The main focus of the tests is to identify the throughput and latency for the primary operations as most of the -secondary operations would either have the same results or be irrelevant. For example: - -* tagging or deprecating a resource is equivalent to performing a create without validation -* updating a resource is equivalent to creating a new one (with or without validation) -* accessing the incoming or outgoing link is irrelevant as the system just delegates the operation to the triple store - -The primary operations are: - -* create without validation -* create with validation using a schema that imports others as transitive dependencies -* create with validation using a schema with the same collection of shapes but without any dependencies -* fetch a resource by id while injecting the metadata in the resource RDF graph representation and returning it as json -* fetch a resource source json representation - -Each test has been run for 5 minutes using a concurrency level (users) of 16 for each KG node -(e.g.: 6 KG nodes -> 96 users). - -# Results - -## Highlights and conclusions - -The system scales almost linearly with the number of nodes in terms of throughput and the latency remains within -acceptable values. - -In most cases the increase of latency is minimal as adding additional nodes to the cluster increases -the necessary data exchange between nodes when handling requests. The chance for the required data to be handled by the -node that accepts each request decreases from 100% (single node) to ~16% (6 node cluster) and if executing the request -implies interaction with multiple resources (like in the case of creating a resource with validation where the schema -has import definitions) the chances drop close to 1%. - -The latency increase for creating resources with validation using a self contained (single) schema is caused by the -fact that the system is CPU bound while performing heavy operations. The the values for the 50 and 95 percentiles show -that there's a higher deviation in the values for this operation. This is explained by nodes processing different -number of validations depending on how requests are distributed between the members of the cluster. - -![Read - Throughput (requests / second) for varied number of nodes](assets/v1.2.1/benchmarks-read-throughput.png) - -![Read - Latency (ms) for varied number of nodes](assets/v1.2.1/benchmarks-read-latency.png) - -![Write no validation - Throughput (requests / second) for varied number of nodes](assets/v1.2.1/benchmarks-write-novalidation-throughput.png) - -![Write no validation - Latency (ms) for varied number of nodes](assets/v1.2.1/benchmarks-write-novalidation-latency.png) - -![Write - Throughput (requests / second) for varied number of nodes](assets/v1.2.1/benchmarks-write-throughput.png) - -![Write - Latency (ms) for varied number of nodes](assets/v1.2.1/benchmarks-write-latency.png) - -## Raw results - -The following sections show the exact numbers for throughput and latency of the system for each test configuration. - -### Create Resource, No Validation - -The test evaluates the throughput and latency at 50, 75, 95 and 99 percentiles when creating new unconstrained -resources. - -| Nodes | Users | Throughput (req/s) | p50 (ms) | p75 (ms) | p95 (ms) | p99 (ms) | -|-------|-------|--------------------|----------|----------|----------|----------| -| 1 | 16 | 781 | 16 | 21 | 42 | 57 | -| 2 | 32 | 1281 | 20 | 29 | 50 | 66 | -| 4 | 64 | 2044 | 25 | 37 | 65 | 89 | -| 6 | 96 | 2730 | 40 | 62 | 96 | 130 | - -### Create Resource, Validation, CrossProject Resolution, Modular Schema - -The test evaluates the throughput and latency at 50, 75, 95 and 99 percentiles when creating new resources constrained -by schemas hosted in a separate project than the one used for creation. The schema uses transitive imports of other -schemas and contexts. - -| Nodes | Users | Throughput (req/s) | p50 (ms) | p75 (ms) | p95 (ms) | p99 (ms) | -|-------|-------|--------------------|----------|----------|----------|----------| -| 1 | 16 | 26 | 618 | 638 | 666 | 699 | -| 2 | 32 | 46 | 682 | 712 | 768 | 977 | -| 4 | 64 | 88 | 719 | 753 | 822 | 1067 | -| 6 | 96 | 127 | 743 | 806 | 898 | 1051 | - -### Create Resource, Validation, InProject Resolution, Modular Schema - -The test evaluates the throughput and latency at 50, 75, 95 and 99 percentiles when creating new resources constrained -by schemas hosted in the same project as the one used for creation. The schema uses transitive imports of other -schemas and contexts. - -| Nodes | Users | Throughput (req/s) | p50 (ms) | p75 (ms) | p95 (ms) | p99 (ms) | -|-------|-------|--------------------|----------|----------|----------|----------| -| 1 | 16 | 28 | 569 | 590 | 620 | 650 | -| 2 | 32 | 48 | 655 | 685 | 736 | 926 | -| 4 | 64 | 97 | 658 | 687 | 739 | 867 | -| 6 | 96 | 125 | 758 | 799 | 879 | 977 | - -### Create Resource, Validation, InProject Resolution, Single Schema - -The test evaluates the throughput and latency at 50, 75, 95 and 99 percentiles when creating new resources constrained -by a schema hosted in the same project as the one used for creation. The schema has all the necessary shapes defined in -the same resource and does not reference other schemas or contexts. - -| Nodes | Users | Throughput (req/s) | p50 (ms) | p75 (ms) | p95 (ms) | p99 (ms) | -|-------|-------|--------------------|----------|----------|----------|----------| -| 1 | 16 | 82 | 201 | 228 | 272 | 316 | -| 2 | 32 | 138 | 211 | 301 | 410 | 504 | -| 4 | 64 | 258 | 153 | 317 | 668 | 838 | -| 6 | 96 | 364 | 147 | 273 | 879 | 1198 | - -### Fetch Resource - -The test evaluates the throughput and latency at 50, 75, 95 and 99 percentiles when getting a resource by its id. It -shows the impact of transforming the original json representation into an RDF graph, injecting the system metadata -(e.g. rev, deprecated, project, org etc.) and presenting the result in a JSON-LD representation. - -| Nodes | Users | Throughput (req/s) | p50 (ms) | p75 (ms) | p95 (ms) | p99 (ms) | -|-------|-------|--------------------|----------|----------|----------|----------| -| 1 | 16 | 777 | 20 | 21 | 28 | 33 | -| 2 | 32 | 1518 | 20 | 22 | 28 | 45 | -| 4 | 64 | 3000 | 20 | 22 | 30 | 47 | -| 6 | 96 | 4393 | 19 | 25 | 40 | 61 | - -### Fetch Resource Source - -The test evaluates the throughput and latency at 50, 75, 95 and 99 percentiles when getting a resource by its id in its -original representation. - -| Nodes | Users | Throughput (req/s) | p50 (ms) | p75 (ms) | p95 (ms) | p99 (ms) | -|-------|-------|--------------------|----------|----------|----------|----------| -| 1 | 16 | 1050 | 15 | 16 | 17 | 26 | -| 2 | 32 | 2103 | 15 | 16 | 18 | 30 | -| 4 | 64 | 3821 | 16 | 17 | 19 | 37 | -| 6 | 96 | 5648 | 16 | 17 | 21 | 37 | \ No newline at end of file diff --git a/docs/src/main/paradox/docs/delta/benchmarks/v1.4.2.md b/docs/src/main/paradox/docs/delta/benchmarks/v1.4.2.md deleted file mode 100644 index 200ec1e681..0000000000 --- a/docs/src/main/paradox/docs/delta/benchmarks/v1.4.2.md +++ /dev/null @@ -1,276 +0,0 @@ -# Benchmarks v1.4.2 - -The tests were run against version 1.4.2 in January 2021 using @link:[Gatling](https://gatling.io/){ open=new } version 3.5.0. - -These are the most recent benchmarks. For older benchmarks of Nexus version 1.2.1 see @ref:[here](v1.2.1.md). - -# Environment - -The system was deployed on AWS EKS using the following configuration: - -* Up to 12 compute nodes of 16 vCPU, 64 GB RAM -* [AWS EBS storage](https://aws.amazon.com/ebs/) - gp3, 10 000 IOPS, throughput 250 MB/s - -The services were deployed as follows: - -* 1 to 12 Delta pods with 8 vCPU, 8GB HEAP -* 1 to 12 Cassandra pods each with 3.5 vCPU, 12 GB HEAP, 250 GB EBS storage -* 1 Elasticsearch pod with 2 vCPU, 8 GB HEAP, 50GB gp2 storage -* 1 Blazegraph pod with 2 vCPU, 4 GB HEAP, 50GB gp2 storage -* 1 Keycloak pod using its internal database H2 as storage, 5GB gp2 storage -* 1 Prometheus pod and 1 Grafana pod, 50GB gp2 storage -* 1 Nginx ingress controller - -# Data volume - -Prior to the test execution a reasonable amount of data was injected into the system to ensure the system behaves well -under a typical volume, specifically 120,000,000 resources were created across 27 projects using an exponential -distribution. The total number of triples (22 for each resource + 11 system metadata) was approximately: 4,000,000,000. - -The following resource template was used for both pre-created resources and the resources created during the tests: -``` -{ - "@type": [ - "http://www.w3.org/ns/prov#Activity", - "https://neuroshapes.org/StimulusExperiment" - ], - "http://www.w3.org/ns/prov#used": { - "@id": "https://nexus-sandbox.io/neurosciencegraph/data/049b569d-98ba-4a54-a9cb-45f5c5cd3b40", - "@type": [ - "http://www.w3.org/ns/prov#Entity", - "https://neuroshapes.org/PatchedCell" - ], - "http://schema.org/name": "Some PatchedCell" - }, - "http://www.w3.org/ns/prov#wasAssociatedWith": { - "@id": "https://nexus-sandbox.io/v1/realms/random/users/someuser", - "@type": "http://www.w3.org/ns/prov#Agent" - }, - "https://neuroshapes.org/hadProtocol": { - "@id": "https://nexus-sandbox.io/neurosciencegraph/data/3f189851-c3de-470d-894e-68abe116bfe4", - "@type": [ - "https://neuroshapes.org/Protocol", - "https://neuroshapes.org/ExperimentalProtocol", - "http://www.w3.org/ns/prov#Entity" - ], - "http://schema.org/name": "Some Protocol" - }, - "https://neuroshapes.org/stimulus": { - "https://neuroshapes.org/stimulusType": { - "@id": "http://stimulusont.org/H20S8", - "http://www.w3.org/2000/01/rdf-schema#label": "H20S8" - } - } -} -``` - -# Scenarios - -The main focus of the tests is to identify the throughput and latency for the primary operations as most of the -secondary operations would either have the same results or be irrelevant. For example: - -* tagging or deprecating a resource is equivalent to performing a create without validation -* updating a resource is equivalent to creating a new one (with or without validation) -* accessing the incoming or outgoing link is irrelevant as the system just delegates the operation to the triple store - -The primary operations are: - -* create without validation -* create with validation using a schema that imports others as transitive dependencies -* create with validation using a schema with the same collection of shapes but without any dependencies -* fetch a resource by id while injecting the metadata in the resource RDF graph representation and returning it as json -* fetch a resource source json representation - -Each test has been run for 5 minutes using a concurrency level of 32 for each Delta node -(e.g.: 6 Delta nodes -> 192 users), except for the single node test where a concurrency level of 16 was used. - -# Results - -## Highlights and conclusions - -The system scales almost linearly with the number of nodes in terms of throughput and the latency remains within -acceptable values. - -In most cases the increase of latency is minimal as adding additional nodes to the cluster increases -the necessary data exchange between nodes when handling requests. The chance for the required data to be handled by the -node that accepts each request decreases from 100% (single node) to ~16% (6 node cluster) and if executing the request -implies interaction with multiple resources (like in the case of creating a resource with validation where the schema -has import definitions) the chances drop close to 1%. - -The latency increase for creating resources with validation using a self contained (single) schema is caused by the -fact that the system is CPU bound while performing heavy operations. The the values for the 95th percentiles show -that there's a higher deviation in the values for this operation. This is explained by nodes processing different -number of validations depending on how requests are distributed between the members of the cluster. - -As the benchmarks were performed using EBS (remote) storage for the Cassandra instances this may have influenced the latency during the benchmarks. However, when re-running the benchmarks the results were very similar. In addition, when studying the overall performance of the system and comparing it to the previous benchmarks which used local storage the performance still behaves as expected. - -It can be also be observed that enabling authentication does not add any significant amount of latency to the benchmarks. - -![Read - Throughput (requests / second) for varied number of nodes](assets/v1.4.2/without-auth/benchmarks-read-throughput.png) - -![Read - Latency (ms) for varied number of nodes](assets/v1.4.2/without-auth/benchmarks-read-latency.png) - -![Write no validation - Throughput (requests / second) for varied number of nodes](assets/v1.4.2/without-auth/benchmarks-write-novalidation-throughput.png) - -![Write no validation - Latency (ms) for varied number of nodes](assets/v1.4.2/without-auth/benchmarks-write-novalidation-latency.png) - -![Write - Throughput (requests / second) for varied number of nodes](assets/v1.4.2/without-auth/benchmarks-write-throughput.png) - -![Write - Latency (ms) for varied number of nodes](assets/v1.4.2/without-auth/benchmarks-write-latency.png) - -## Raw results - -The following sections show the exact numbers for throughput and latency of the system for each test configuration. - -### Create Resource, No Validation - -The test evaluates the throughput and latency at 50, 75, 95 and 99 percentiles when creating new unconstrained resources. - -| Nodes | Concurrency | Throughput (req/s) | p50 (ms) | p75 (ms) | p95 (ms) | p99 (ms) | -|-------|-------------|--------------------|----------|----------|----------|----------| -| 1 | 16 | 712 | 20 | 24 | 33 | 81 | -| 2 | 64 | 1328 | 44 | 54 | 83 | 110 | -| 4 | 128 | 2594 | 42 | 56 | 97 | 125 | -| 6 | 192 | 3270 | 52 | 71 | 110 | 143 | -| 9 | 288 | 4539 | 52 | 77 | 130 | 172 | -| 12 | 384 | 5578 | 60 | 86 | 132 | 174 | - -### Create Resource, Validation, InProject Resolution, Modular Schema - -The test evaluates the throughput and latency at 50, 75, 95 and 99 percentiles when creating new resources constrained by schemas hosted in the same project as the one used for creation. The schema uses transitive imports of other schemas and contexts. - -| Nodes | Concurrency | Throughput (req/s) | p50 (ms) | p75 (ms) | p95 (ms) | p99 (ms) | -|-------|-------------|--------------------|-----------|-----------|-----------|-----------| -| 1 | 8 | 24 | 329 | 340 | 365 | 437 | -| 2 | 32 | 58 | 552 | 596 | 640 | 709 | -| 4 | 64 | 116 | 548 | 587 | 654 | 793 | -| 6 | 96 | 164 | 573 | 614 | 707 | 915 | -| 9 | 144 | 237 | 598 | 638 | 724 | 899 | -| 12 | 192 | 288 | 656 | 702 | 798 | 1044 | - -### Create Resource, Validation, InProject Resolution, Single Schema - -The test evaluates the throughput and latency at 50, 75, 95 and 99 percentiles when creating new resources constrained by a schema hosted in the same project as the one used for creation. The schema has all the necessary shapes defined in the same resource and does not reference other schemas or contexts. - -| Nodes | Concurrency | Throughput (req/s) | p50 (ms) | p75 (ms) | p95 (ms) | p99 (ms) | -|-------|-------------|--------------------|-----------|-----------|-----------|-----------| -| 1 | 8 | 73 | 105 | 122 | 136 | 157 | -| 2 | 32 | 141 | 216 | 277 | 358 | 428 | -| 4 | 64 | 266 | 178 | 297 | 546 | 692 | -| 6 | 96 | 367 | 169 | 246 | 779 | 1159 | -| 9 | 144 | 532 | 175 | 240 | 919 | 1350 | -| 12 | 192 | 626 | 170 | 218 | 1379 | 2216 | - -### Fetch Resource - -The test evaluates the throughput and latency at 50, 75, 95 and 99 percentiles when getting a resource by its id. It -shows the impact of transforming the original json representation into an RDF graph, injecting the system metadata -(e.g. rev, deprecated, project, org etc.) and presenting the result in a JSON-LD representation. - -| Nodes | Concurrency | Throughput (req/s) | p50 (ms) | p75 (ms) | p95 (ms) | p99 (ms) | -|-------|-------------|---------------------|----------|----------|----------|----------| -| 1 | 16 | 1513 | 10 | 11 | 12 | 19 | -| 2 | 64 | 2687 | 22 | 25 | 36 | 45 | -| 4 | 128 | 5386 | 22 | 24 | 34 | 45 | -| 6 | 192 | 9063 | 19 | 23 | 35 | 47 | -| 9 | 288 | 11997 | 22 | 26 | 40 | 53 | -| 12 | 384 | 17011 | 20 | 25 | 36 | 50 | - -### Fetch Resource Source - -The test evaluates the throughput and latency at 50, 75, 95 and 99 percentiles when getting a resource by its id in its -original representation. - -| Nodes | Concurrency | Throughput (req/s) | p50 (ms) | p75 (ms) | p95 (ms) | p99 (ms) | -|-------|-------------|---------------------|----------|----------|----------|----------| -| 1 | 16 | 1539 | 10 | 10 | 11 | 18 | -| 2 | 64 | 3135 | 19 | 22 | 32 | 39 | -| 4 | 128 | 5615 | 21 | 23 | 38 | 47 | -| 6 | 192 | 8726 | 20 | 22 | 32 | 45 | -| 9 | 288 | 12354 | 22 | 26 | 36 | 50 | -| 12 | 384 | 17815 | 20 | 24 | 34 | 46 | - -## Results with authentication - -![Read - Throughput (requests / second) for varied number of nodes (with authentication)](assets/v1.4.2/with-auth/benchmarks-read-throughput.png) - -![Read - Latency (ms) for varied number of nodes (with authentication)](assets/v1.4.2/with-auth/benchmarks-read-latency.png) - -![Write no validation - Throughput (requests / second) for varied number of nodes (with authentication)](assets/v1.4.2/with-auth/benchmarks-write-novalidation-throughput.png) - -![Write no validation - Latency (ms) for varied number of nodes (with authentication)](assets/v1.4.2/with-auth/benchmarks-write-novalidation-latency.png) - -![Write - Throughput (requests / second) for varied number of nodes (with authentication)](assets/v1.4.2/with-auth/benchmarks-write-throughput.png) - -![Write - Latency (ms) for varied number of nodes (with authentication)](assets/v1.4.2/with-auth/benchmarks-write-latency.png) - -## Raw results with authentication - -The following sections show the exact numbers for throughput and latency of the system for each test configuration when enabling authentication. It can be noted that the authentication does not add any significant performance penalty. - -### Create Resource, No Validation (with authentication) - -The test evaluates the throughput and latency at 50, 75, 95 and 99 percentiles when creating new unconstrained resources. - -| Nodes | Concurrency | Throughput (req/s) | p50 (ms) | p75 (ms) | p95 (ms) | p99 (ms) | -|-------|-------------|--------------------|----------|----------|----------|-----------| -| 1 | 16 | 683 | 21 | 25 | 36 | 67 | -| 2 | 64 | 1273 | 46 | 57 | 85 | 110 | -| 4 | 128 | 2557 | 44 | 58 | 94 | 118 | -| 6 | 192 | 3113 | 54 | 74 | 120 | 154 | -| 9 | 288 | 4232 | 60 | 85 | 128 | 169 | -| 12 | 384 | 5114 | 68 | 94 | 144 | 189 | - -### Create Resource, Validation, InProject Resolution, Modular Schema (with authentication) - -The test evaluates the throughput and latency at 50, 75, 95 and 99 percentiles when creating new resources constrained by schemas hosted in the same project as the one used for creation. The schema uses transitive imports of other schemas and contexts. - -| Nodes | Concurrency | Throughput (req/s) | p50 (ms) | p75 (ms) | p95 (ms) | p99 (ms) | -|-------|-------------|--------------------|-----------|-----------|-----------|-----------| -| 1 | 8 | 24 | 327 | 337 | 357 | 374 | -| 2 | 32 | 58 | 549 | 592 | 634 | 707 | -| 6 | 96 | 168 | 565 | 607 | 680 | 784 | -| 9 | 144 | 230 | 596 | 646 | 801 | 1431 | -| 12 | 192 | 289 | 647 | 695 | 803 | 1124 | - -### Create Resource, Validation, InProject Resolution, Single Schema (with authentication) - -The test evaluates the throughput and latency at 50, 75, 95 and 99 percentiles when creating new resources constrained by a schema hosted in the same project as the one used for creation. The schema has all the necessary shapes defined in the same resource and does not reference other schemas or contexts. - -| Nodes | Concurrency | Throughput (req/s) | p50 (ms) | p75 (ms) | p95 (ms) | p99 (ms) | -|-------|-------------|--------------------|-----------|-----------|-----------|-----------| -| 1 | 8 | 74 | 104 | 121 | 134 | 146 | -| 2 | 32 | 139 | 214 | 286 | 372 | 445 | -| 4 | 64 | 270 | 174 | 295 | 543 | 674 | -| 6 | 96 | 383 | 168 | 244 | 727 | 998 | -| 9 | 144 | 622 | 171 | 219 | 1393 | 2249 | -| 12 | 192 | 24 | 329 | 340 | 365 | 437 | - -### Fetch Resource (with authentication) - -The test evaluates the throughput and latency at 50, 75, 95 and 99 percentiles when getting a resource by its id. It -shows the impact of transforming the original json representation into an RDF graph, injecting the system metadata -(e.g. rev, deprecated, project, org etc.) and presenting the result in a JSON-LD representation. - -| Nodes | Concurrency | Throughput (req/s) | p50 (ms) | p75 (ms) | p95 (ms) | p99 (ms) | -|-------|-------------|---------------------|----------|----------|----------|----------| -| 1 | 16 | 1496 | 11 | 11 | 11 | 14 | -| 2 | 64 | 3016 | 20 | 24 | 33 | 37 | -| 4 | 128 | 5459 | 22 | 24 | 33 | 44 | -| 6 | 192 | 9293 | 19 | 22 | 34 | 48 | -| 9 | 288 | 12118 | 22 | 25 | 40 | 53 | -| 12 | 384 | 14268 | 24 | 30 | 45 | 61 | - -### Fetch Resource Source (with authentication) - -The test evaluates the throughput and latency at 50, 75, 95 and 99 percentiles when getting a resource by its id in its -original representation. - -| Nodes | Concurrency | Throughput (req/s) | p50 (ms) | p75 (ms) | p95 (ms) | p99 (ms) | -|-------|-------------|---------------------|----------|----------|----------|----------| -| 1 | 16 | 1494 | 11 | 11 | 11 | 14 | -| 2 | 64 | 2982 | 20 | 24 | 34 | 37 | -| 4 | 128 | 5690 | 21 | 23 | 33 | 44 | -| 6 | 192 | 8971 | 19 | 23 | 35 | 49 | -| 9 | 288 | 12491 | 21 | 25 | 36 | 49 | -| 12 | 384 | 14363 | 25 | 30 | 42 | 54 | diff --git a/docs/src/main/paradox/docs/delta/index.md b/docs/src/main/paradox/docs/delta/index.md index 59395b6ec6..be4ff2c6e3 100644 --- a/docs/src/main/paradox/docs/delta/index.md +++ b/docs/src/main/paradox/docs/delta/index.md @@ -4,7 +4,6 @@ - @ref:[API Reference](api/index.md) - @ref:[Nexus Metadata](metadata.md) - @ref:[Plugins](plugins/index.md) -- @ref:[Benchmarks](benchmarks/index.md) @@@ diff --git a/docs/src/main/paradox/docs/faq.md b/docs/src/main/paradox/docs/faq.md index 6b0c16a47f..439a9cdf77 100644 --- a/docs/src/main/paradox/docs/faq.md +++ b/docs/src/main/paradox/docs/faq.md @@ -43,9 +43,6 @@ strategies, retries and progress Blue Brain Nexus leverages scalable open source technologies, therefore limitations and performance depends heavily on the deployment setup where Nexus is running. -To get an idea about the ingestion capabilities, we have run @ref:[Benchmarks](delta/benchmarks/v1.4.2.md) where we were able -to ingest over 3.5 billion triples representing 120 million resources. - ### What is a Knowledge Graph? A Knowledge Graph is a modern approach to enabling the interlinked representations of entities (real-world objects, diff --git a/docs/src/main/paradox/docs/releases/index.md b/docs/src/main/paradox/docs/releases/index.md index 30e8cf4adc..2e3d9611b1 100644 --- a/docs/src/main/paradox/docs/releases/index.md +++ b/docs/src/main/paradox/docs/releases/index.md @@ -10,8 +10,6 @@ - @ref:[v1.7 To v1.8 Migration](v1.7-to-v1.8-migration.md) - @ref:[v1.7 Release Notes](v1.7-release-notes.md) - @ref:[v1.6 To v1.7 Migration](v1.6-to-v1.7-migration.md) -- @ref:[v1.6 Release Notes](v1.6-release-notes.md) -- @ref:[v1.5 To v1.6 Migration](v1.5-to-v1.6-migration.md) - @ref:[Older releases](older-releases.md) @@@ @@ -340,60 +338,6 @@ Tools will be provided to enable migration from Cassandra to PostgreSQL for exis A detailed list of changes included in the release can be found in the @ref:[release notes](v1.7-release-notes.md). -## v1.6.1 (29.10.2021) - -This release contains bugfixes and minor improvements: - -- Graph-analytics returns edges for non-existing nodes @link:[#2871](https://github.com/BlueBrain/nexus/issues/2871) -- Graph analytics is trying to resolve every link to all types of resources - @link:[#2852](https://github.com/BlueBrain/nexus/issues/2852) -- Composite key values cache is not distributed across nodes - @link:[#2909](https://github.com/BlueBrain/nexus/issues/2909) -- Shortcut Acl permission check (project -> org -> root) when address matches early - @link:[#2916](https://github.com/BlueBrain/nexus/issues/2916) -- Resource-view opens up as a side panel @link:[#2617](https://github.com/BlueBrain/nexus/issues/2617) -- User can see all data, when the search query is empty @link:[#2875](https://github.com/BlueBrain/nexus/issues/2875) -- A loading spinner shows up when there is a delay in fetching search results - @link:[#2880](https://github.com/BlueBrain/nexus/issues/2880) -- Label 'none of' in filter was previously mis labelled as 'any of' - @link:[#2872](https://github.com/BlueBrain/nexus/issues/2872) -- The behaviour of 'none of' filter has been fixed to avoid confusion with other filters - @link:[#2898](https://github.com/BlueBrain/nexus/issues/2898) -- Preview plugin big fix to prevent it from crashing for certain file paths - @link:[#2884](https://github.com/BlueBrain/nexus/issues/2884) -- Search bar matches query content @link:[#2874](https://github.com/BlueBrain/nexus/issues/2874) - -**Full Changelogs**: - -- Delta: @link:[v1.6.0...v1.6.1](https://github.com/BlueBrain/nexus/compare/v1.6.0...v1.6.1) -- Fusion: @link:[v1.6.0...v1.6.1](https://github.com/BlueBrain/nexus-web/compare/v1.6.0...v1.6.1) - -## v1.6.0 (13.10.2021) - -### Deprecations - -- Nexus client -- Indexing data in other systems -- Remote storages - -### New features / enhancements - -- Introduced a plugin to search among different projects -- Introduced PDF, CSV, TSV and Youtube Viewer Fusion Plugins -- Add basic authentication to access a secured Elasticsearch cluster -- Handle user-defined queries to Blazegraph with a dedicated client -- Introduced a plugin to analyze properties and relationships of resources within a project -- Synchronous indexing -- Listing of resources outside the project scope -- The RDF parser to validate resources is now configurable -- Automatic project provisioning -- Introduced quotas on projects -- Project deletion (on demand and automatic) -- Tagging resources after deprecation -- View passivation - -A detailed list of changes included in the release can be found in the @ref:[release notes](v1.6-release-notes.md). - ## Older releases The release notes of older versions are available @link:[here](https://bluebrainnexus.io/v1.6.x/docs/releases/index.html). \ No newline at end of file diff --git a/docs/src/main/paradox/docs/releases/older-releases.md b/docs/src/main/paradox/docs/releases/older-releases.md index 1c062e2fe6..3924ff0ffc 100644 --- a/docs/src/main/paradox/docs/releases/older-releases.md +++ b/docs/src/main/paradox/docs/releases/older-releases.md @@ -1,2 +1,2 @@ # Older releases -For archival purposes, the release notes of older versions are available @link:[here](https://bluebrainnexus.io/v1.6.x/docs/releases/index.html). +For archival purposes, the release notes of older versions are available @link:[here](https://bluebrainnexus.io/v1.7.x/docs/releases/index.html). diff --git a/docs/src/main/paradox/docs/releases/v1.5-to-v1.6-migration.md b/docs/src/main/paradox/docs/releases/v1.5-to-v1.6-migration.md deleted file mode 100644 index ec8dd39514..0000000000 --- a/docs/src/main/paradox/docs/releases/v1.5-to-v1.6-migration.md +++ /dev/null @@ -1,11 +0,0 @@ -# v1.5 To v1.6 Migration - -The v1.6 release of Delta includes a change on how project statistics are computed which requires to run a query against the Cassandra primary store. - -## Delete the project statistics row in the projections_progress table - -This command has to be run after stopping Delta 1.5 and before starting Delta 1.6 - - DELETE FROM {delta_1_5}.projections_progress where projection_id = 'ProjectsCounts'; - -Note that as project statistics will be recomputed according to all events in the system, it may take some time for the statistics computation to reach back to the most recent events. \ No newline at end of file diff --git a/docs/src/main/paradox/docs/releases/v1.6-release-notes.md b/docs/src/main/paradox/docs/releases/v1.6-release-notes.md deleted file mode 100644 index 1d0ab0d68f..0000000000 --- a/docs/src/main/paradox/docs/releases/v1.6-release-notes.md +++ /dev/null @@ -1,116 +0,0 @@ -# v1.6 Release Notes - -> This is an incremental software release that adds several improvements across the board. The API is backwards -> compatible to v1.5. - -For the detailed list of updates in this release, see the -@link:[list of addressed issues](https://github.com/BlueBrain/nexus/issues?q=is%3Aissue+is%3Aclosed+created%3A2021-05-01..2021-10-13+){ open=new } -since v1.5. - -If you are currently using Nexus Delta from 1.5.x, please -visit the @ref:[migration instructions](v1.5-to-v1.6-migration.md) for the necessary steps to successfully upgrade. - -## Nexus Delta - -The main goal of this release is to allow users to search across multiple projects through @ref:[global search](../running-nexus/search-configuration.md) - -Secondary goals include: - -- The possibility for Delta to query a secured Elasticsearch cluster -- The introduction of a dedicated Blazegraph client for user-defined queries with the possibility to define a timeout to improve Blazegraph stability -- Synchronous indexing -- The analysis of properties and relationships of resources within a project through the @ref:[graph analytics plugin](../delta/api/graph-analytics-api.md) -- The listing of resources within an organization and within all projects -- The configuration of the RDF parser -- @ref:[The deletion of projects and all its resources](../delta/api/projects-api.md#delete) -- Automatic project provisioning -- @ref:[The introduction of quotas](../delta/api/quotas.md) -- Tagging resources after deprecation - -A series of bugs have also been fixed with this new release. - -### Deprecations - -These two features are now marked as deprecated and will be removed in an upcoming release: - -- Nexus client -- Indexing data in other systems -- Remote storages - -### Global search - -Before 1.6, searching across several projects was not available in Nexus. The first version of Global search included in this release -aims to fill this blank. - -Global search is composed of 2 parts: - -- A Delta plugin which allows to index and query resources according to a given configuration -- A matching plugin in Fusion allows to query the endpoints provided by Delta to filter/sort results and customize how these results are displayed to the users - -More details about search and how to configure it is provided @ref:[here](../running-nexus/search-configuration.md) - -Please note that this first version is experimental in both Delta and Fusion and that improvements and possibly breaking -changes may be introduced in the upcoming releases. - -### Secure the Delta Elasticsearch client - -With 1.6, it is now possible to define credentials to query a secured Elasticsearch cluster: @ref:[Elasticsearch configuration](../running-nexus/configuration/index.md#elasticsearch-views-plugin-configuration). - -### Dedicated Blazegraph client for user-defined queries - -User-defined queries could bring instability to Blazegraph because of their complexity. In 1.6, a @link:[dedicated client with a timeout](https://github.com/BlueBrain/nexus/blob/$git.branch$/delta/plugins/blazegraph/src/main/resources/blazegraph.conf#L34) which triggers a circuit breaker in Blazegraph has been introduced in order to solve that problem. - -### Synchronous indexing - -From 1.6, the different types of resources can be @ref:[indexed directly after creation/modification](../delta/api/resources-api.md#indexing) without waiting for the background indexing process to pick it up. - -### Graph analytics - -This @ref:[new plugin](../delta/api/graph-analytics-api.md) helps users to inspect the properties and relationships of the different resources in a given project. - -Only the Delta endpoints are available in this release but a Nexus Fusion plugin will be released soon to make use of it. - -### Listings - -The different types of resources can now be listed at the @ref:[organization level](../delta/api/resources-api.md#within-an-organization) and @ref:[within all the projects](../delta/api/resources-api.md#within-all-projects) the current user has access to. - -### Configurable RDF parser - -The RDF parser allowing to validate incoming data can now be @link:[configured](https://github.com/BlueBrain/nexus/blob/$git.branch$/delta/app/src/main/resources/app.conf#L83) in order to be more or less strict. - -### Automatic project provisioning - -When enabled, a dedicated project is created for the current user on its first access to the Nexus platform. - -### Deletion of projects - -The deletion of projects and all its enclosed resources is now possible by enabling a configuration flag via a @ref:[dedicated endpoint](../delta/api/projects-api.md#delete). - -### Quotas - -It is now possible to define via static configurations @ref:[quotas on projects](../delta/api/quotas.md). - -### Tagging resources after deprecation - -The different types of resources can now be tagged after their deprecation. - -### View passivation - -Views are now stopped if no new event has been processed after a configured period. - -## Nexus Fusion - -### Features - -- Revamped [Fusion search](https://github.com/BlueBrain/nexus/blob/$git.branch$/docs/src/main/paradox/docs/fusion/search.md). Fusion search will now use delta search end point and will provide a improved UI with sorting, filtering, pagination etc. -- Real-time updates. Fusion now uses 'indexing=sync' option in delta API. This means all the updates made through fusion will be indexed in delta synchronously. -- Sub apps other than studio will be hidden in sidebar for anonymous users. -- Fusion will show a warning message when projects are due for deletion. - -### Fixes - -- Fixes on video player plugin. - -## Nexus Forge - -See @link:[Nexus Forge](https://nexus-forge.readthedocs.io/en/latest/index.html){ open=new } documentation for release notes. diff --git a/docs/src/main/paradox/docs/running-nexus/index.md b/docs/src/main/paradox/docs/running-nexus/index.md index b79ed4716b..4247cee1f5 100644 --- a/docs/src/main/paradox/docs/running-nexus/index.md +++ b/docs/src/main/paradox/docs/running-nexus/index.md @@ -213,10 +213,9 @@ Details on how to deploy, run and monitor the different components are described ### Choice of hardware -Depending on the target throughput, usage profiles and data volume the hardware specification can vary greatly; please -take a look at the @ref:[benchmarks section](../delta/benchmarks/v1.4.2.md) to get an idea of what you should expect in terms -of throughput with various hardware configurations. When the usage profiles are unknown a couple of rules of thumb -should narrow the scope: +Depending on the target throughput, usage profiles and data volume the hardware specification can vary greatly. + +When the usage profiles are unknown a couple of rules of thumb should narrow the scope: 1. Nexus uses a collection of data stores (@link:[PostgreSQL](https://www.postgresql.org/){ open=new }, @link:[Elasticsearch](https://www.elastic.co/elasticsearch){ open=new },