Skip to content

Update dependency @apollo/gateway to v2.10.1 [SECURITY] #650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

renovate[bot]
Copy link
Contributor

@renovate renovate bot commented Apr 7, 2025

This PR contains the following updates:

Package Change Age Adoption Passing Confidence
@apollo/gateway (source) 2.10.0 -> 2.10.1 age adoption passing confidence

GitHub Vulnerability Alerts

CVE-2025-32031

Impact

Summary

A vulnerability in Apollo Gateway allowed queries with deeply nested and reused named fragments to be prohibitively expensive to query plan, specifically due to internal optimizations being frequently bypassed. This could lead to excessive resource consumption and denial of service.

Details

The query planner includes an optimization that significantly speeds up planning for applicable GraphQL selections. However, queries with deeply nested and reused named fragments can generate many selections where this optimization does not apply, leading to significantly longer planning times. Because the query planner does not enforce a timeout, a small number of such queries can render gateway inoperable.

Fix/Mitigation

  • A new Query Optimization Limit metric has been added:
    • This metric approximates the number of selections that cannot be skipped by the existing optimization.
    • The metric is checked against a limit to prevent excessive computation.

Given the complexity of query planning optimizations, we will continue refining these solutions based on real-world performance and accuracy tests.

Patches

This has been remediated in @apollo/gateway version 2.10.1.

Workarounds

No known direct workarounds exist.

References

Query Planning Documentation

Acknowledgements

We appreciate the efforts of the security community in identifying and improving the performance and security of query planning mechanisms.

CVE-2025-32030

Impact

Summary

A vulnerability in Apollo Gateway allowed queries with deeply nested and reused named fragments to be prohibitively expensive to query plan, specifically during named fragment expansion. This could lead to excessive resource consumption and denial of service.

Details

Named fragments were being expanded once per fragment spread during query planning, leading to exponential resource usage when deeply nested and reused fragments were involved.

Fix/Mitigation

A new Query Fragment Expansion Limit metric has been introduced:

  • This metric computes the number of selections a query would have if its fragment spreads were fully expanded.
  • The metric is checked against a limit to prevent excessive computation.

Patches

This has been remediated in @apollo/gateway version 2.10.1.

Workarounds

No known direct workarounds exist.

References

Query Planning Documentation

Acknowledgements

We appreciate the efforts of the security community in identifying and improving the performance and security of query planning mechanisms.


Release Notes

apollographql/federation (@​apollo/gateway)

v2.10.1

Compare Source


Configuration

📅 Schedule: Branch creation - "" (UTC), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.


  • If you want to rebase/retry this PR, check this box

This PR was generated by Mend Renovate. View the repository job log.

@renovate renovate bot added the dependencies Pull requests that update a dependency file label Apr 7, 2025
Copy link

github-actions bot commented Apr 7, 2025

Overview for: federation/constant-vus-over-time

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 300 VUs over 30s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
cosmo 189 5855 total, 0 failed avg: 848ms, p95: 2207ms
apollo-router 172 5373 total, 1 failed avg: 1024ms, p95: 2735ms ❌ 1 failed requests, 1 non-200 responses, 1 unexpected GraphQL errors
grafbase 171 5301 total, 0 failed avg: 929ms, p95: 2910ms
hive-gateway-bun 82 2784 total, 0 failed avg: 3342ms, p95: 5622ms
hive-gateway 77 2613 total, 0 failed avg: 3603ms, p95: 22939ms
apollo-server 75 2584 total, 0 failed avg: 3685ms, p95: 31925ms
mercurius 62 2028 total, 0 failed avg: 4549ms, p95: 7375ms
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 17505      ✗ 0    
     data_received..................: 514 MB  17 MB/s
     data_sent......................: 6.9 MB  225 kB/s
     http_req_blocked...............: avg=1.58ms   min=1.32µs  med=3.02µs   max=2.43s p(90)=4.73µs   p(95)=254.07µs
     http_req_connecting............: avg=1.17ms   min=0s      med=0s       max=1.68s p(90)=0s       p(95)=116.62µs
     http_req_duration..............: avg=848.43ms min=3.09ms  med=708.72ms max=4.98s p(90)=1.74s    p(95)=2.2s    
       { expected_response:true }...: avg=848.43ms min=3.09ms  med=708.72ms max=4.98s p(90)=1.74s    p(95)=2.2s    
     http_req_failed................: 0.00%   ✓ 0          ✗ 5855 
     http_req_receiving.............: avg=322.06ms min=33.17µs med=88.17µs  max=3.81s p(90)=1.15s    p(95)=1.72s   
     http_req_sending...............: avg=22.82ms  min=7.75µs  med=13.68µs  max=2.48s p(90)=141.05µs p(95)=5.03ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s    p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=503.55ms min=2.94ms  med=472.69ms max=2.89s p(90)=935.36ms p(95)=1.05s   
     http_reqs......................: 5855    189.2086/s
     iteration_duration.............: avg=1.55s    min=25.03ms med=1.31s    max=7.77s p(90)=3.1s     p(95)=3.79s   
     iterations.....................: 5835    188.562285/s
     vus............................: 300     min=300      max=300
     vus_max........................: 300     min=300      max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✗ response code was 200
      ↳  99% — ✓ 5352 / ✗ 1
     ✗ no graphql errors
      ↳  99% — ✓ 5352 / ✗ 1
     ✓ valid response structure

     █ setup

     checks.........................: 99.98% ✓ 16056      ✗ 2    
     data_received..................: 471 MB 15 MB/s
     data_sent......................: 6.4 MB 204 kB/s
     http_req_blocked...............: avg=1.17ms   min=1.44µs  med=3.05µs   max=1.51s  p(90)=5.31µs   p(95)=786.21µs
     http_req_connecting............: avg=1.14ms   min=0s      med=0s       max=1.51s  p(90)=0s       p(95)=664.25µs
     http_req_duration..............: avg=1.02s    min=6.96ms  med=753.3ms  max=8.1s   p(90)=2.25s    p(95)=2.73s   
       { expected_response:true }...: avg=1.02s    min=6.96ms  med=753.03ms max=8.1s   p(90)=2.25s    p(95)=2.73s   
     http_req_failed................: 0.01%  ✓ 1          ✗ 5372 
     http_req_receiving.............: avg=435.63ms min=0s      med=91.91µs  max=6.87s  p(90)=1.66s    p(95)=1.97s   
     http_req_sending...............: avg=10.75ms  min=6.53µs  med=14.28µs  max=2.91s  p(90)=166.09µs p(95)=3.66ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s     p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=577.59ms min=6.83ms  med=537.61ms max=1.89s  p(90)=1.04s    p(95)=1.2s    
     http_reqs......................: 5373   172.148522/s
     iteration_duration.............: avg=1.7s     min=28.71ms med=1.36s    max=10.08s p(90)=3.52s    p(95)=4.15s   
     iterations.....................: 5353   171.507731/s
     vus............................: 120    min=120      max=300
     vus_max........................: 300    min=300      max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 15843      ✗ 0    
     data_received..................: 466 MB  15 MB/s
     data_sent......................: 6.3 MB  203 kB/s
     http_req_blocked...............: avg=1.59ms   min=1.48µs  med=3.23µs   max=1.31s  p(90)=5.75µs   p(95)=1.3ms   
     http_req_connecting............: avg=1.46ms   min=0s      med=0s       max=1.31s  p(90)=0s       p(95)=906.78µs
     http_req_duration..............: avg=928.9ms  min=3.08ms  med=584.92ms max=9.04s  p(90)=2.41s    p(95)=2.9s    
       { expected_response:true }...: avg=928.9ms  min=3.08ms  med=584.92ms max=9.04s  p(90)=2.41s    p(95)=2.9s    
     http_req_failed................: 0.00%   ✓ 0          ✗ 5301 
     http_req_receiving.............: avg=463.17ms min=32.97µs med=104.09µs max=7.63s  p(90)=2.01s    p(95)=2.46s   
     http_req_sending...............: avg=20.09ms  min=7.89µs  med=14.86µs  max=2.92s  p(90)=346.76µs p(95)=4.86ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s     p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=445.64ms min=3.02ms  med=344.75ms max=2.46s  p(90)=900.77ms p(95)=1.25s   
     http_reqs......................: 5301    171.36563/s
     iteration_duration.............: avg=1.72s    min=22.34ms med=1.21s    max=10.97s p(90)=3.9s     p(95)=4.95s   
     iterations.....................: 5281    170.719089/s
     vus............................: 300     min=300      max=300
     vus_max........................: 300     min=300      max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 8292      ✗ 0    
     data_received..................: 244 MB  7.3 MB/s
     data_sent......................: 3.3 MB  99 kB/s
     http_req_blocked...............: avg=868.25µs min=1.71µs   med=3.32µs   max=18.56ms p(90)=171.84µs p(95)=9.13ms  
     http_req_connecting............: avg=843.2µs  min=0s       med=0s       max=18.54ms p(90)=113.8µs  p(95)=8.88ms  
     http_req_duration..............: avg=3.34s    min=16.83ms  med=3.01s    max=7.03s   p(90)=5.16s    p(95)=5.62s   
       { expected_response:true }...: avg=3.34s    min=16.83ms  med=3.01s    max=7.03s   p(90)=5.16s    p(95)=5.62s   
     http_req_failed................: 0.00%   ✓ 0         ✗ 2784 
     http_req_receiving.............: avg=68.52ms  min=39.65µs  med=136.35µs max=2.62s   p(90)=133.75ms p(95)=463.52ms
     http_req_sending...............: avg=1.74ms   min=8.32µs   med=16.57µs  max=1.08s   p(90)=243.53µs p(95)=766.95µs
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s      p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=3.27s    min=16.61ms  med=2.96s    max=7.03s   p(90)=5.13s    p(95)=5.57s   
     http_reqs......................: 2784    82.950812/s
     iteration_duration.............: avg=3.45s    min=276.34ms med=3.09s    max=7.06s   p(90)=5.3s     p(95)=5.8s    
     iterations.....................: 2764    82.354901/s
     vus............................: 87      min=87      max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 7779      ✗ 0    
     data_received..................: 230 MB  6.8 MB/s
     data_sent......................: 3.1 MB  92 kB/s
     http_req_blocked...............: avg=854.51µs min=1.83µs   med=3.68µs   max=18.56ms p(90)=1.09ms   p(95)=9.11ms
     http_req_connecting............: avg=818.94µs min=0s       med=0s       max=15.4ms  p(90)=837.64µs p(95)=8.84ms
     http_req_duration..............: avg=3.6s     min=15.53ms  med=1.83s    max=33.08s  p(90)=3.45s    p(95)=22.93s
       { expected_response:true }...: avg=3.6s     min=15.53ms  med=1.83s    max=33.08s  p(90)=3.45s    p(95)=22.93s
     http_req_failed................: 0.00%   ✓ 0         ✗ 2613 
     http_req_receiving.............: avg=354.15µs min=44.26µs  med=107.77µs max=47.73ms p(90)=408.08µs p(95)=1.12ms
     http_req_sending...............: avg=273.94µs min=9.42µs   med=22.02µs  max=75.21ms p(90)=314µs    p(95)=1.4ms 
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s      p(90)=0s       p(95)=0s    
     http_req_waiting...............: avg=3.6s     min=15.42ms  med=1.83s    max=33.08s  p(90)=3.45s    p(95)=22.93s
     http_reqs......................: 2613    77.776542/s
     iteration_duration.............: avg=3.64s    min=205.62ms med=1.86s    max=33.11s  p(90)=3.54s    p(95)=22.96s
     iterations.....................: 2593    77.181237/s
     vus............................: 48      min=48      max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-server`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 7692      ✗ 0    
     data_received..................: 227 MB  6.7 MB/s
     data_sent......................: 3.1 MB  90 kB/s
     http_req_blocked...............: avg=357.04µs min=1.47µs   med=3.51µs   max=10.26ms p(90)=248.22µs p(95)=2.87ms  
     http_req_connecting............: avg=343.93µs min=0s       med=0s       max=10.23ms p(90)=186.85µs p(95)=2.77ms  
     http_req_duration..............: avg=3.68s    min=12.81ms  med=1.48s    max=33.54s  p(90)=3.24s    p(95)=31.92s  
       { expected_response:true }...: avg=3.68s    min=12.81ms  med=1.48s    max=33.54s  p(90)=3.24s    p(95)=31.92s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 2584 
     http_req_receiving.............: avg=229.09µs min=44.85µs  med=111.72µs max=50.41ms p(90)=197.76µs p(95)=333.79µs
     http_req_sending...............: avg=116.76µs min=8.65µs   med=17.53µs  max=11.47ms p(90)=145.99µs p(95)=576.55µs
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s      p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=3.68s    min=12.68ms  med=1.48s    max=33.54s  p(90)=3.24s    p(95)=31.92s  
     http_reqs......................: 2584    75.947176/s
     iteration_duration.............: avg=3.72s    min=201.68ms med=1.5s     max=33.55s  p(90)=3.3s     p(95)=31.96s  
     iterations.....................: 2564    75.35935/s
     vus............................: 6       min=6       max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `mercurius`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 6024      ✗ 0    
     data_received..................: 178 MB  5.5 MB/s
     data_sent......................: 2.4 MB  75 kB/s
     http_req_blocked...............: avg=730.37µs min=1.82µs   med=4.4µs   max=13.23ms  p(90)=1.84ms   p(95)=7.37ms  
     http_req_connecting............: avg=709.02µs min=0s       med=0s      max=12.88ms  p(90)=1.57ms   p(95)=7.27ms  
     http_req_duration..............: avg=4.54s    min=14.23ms  med=4.67s   max=9.72s    p(90)=5.66s    p(95)=7.37s   
       { expected_response:true }...: avg=4.54s    min=14.23ms  med=4.67s   max=9.72s    p(90)=5.66s    p(95)=7.37s   
     http_req_failed................: 0.00%   ✓ 0         ✗ 2028 
     http_req_receiving.............: avg=4.57ms   min=42.51µs  med=111.1µs max=465.18ms p(90)=299.64µs p(95)=762.82µs
     http_req_sending...............: avg=153.37µs min=9.51µs   med=26.29µs max=13.76ms  p(90)=246.56µs p(95)=1.04ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s      max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=4.54s    min=14.1ms   med=4.67s   max=9.72s    p(90)=5.66s    p(95)=7.37s   
     http_reqs......................: 2028    62.893464/s
     iteration_duration.............: avg=4.61s    min=578.76ms med=4.7s    max=9.74s    p(90)=5.68s    p(95)=7.4s    
     iterations.....................: 2008    62.273213/s
     vus............................: 74      min=74      max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview

Copy link

github-actions bot commented Apr 7, 2025

Overview for: federation/constant-vus-subgraphs-delay

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 300 VUs over 30s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
cosmo 183 5650 total, 0 failed avg: 865ms, p95: 2375ms
apollo-router 173 5403 total, 1 failed avg: 905ms, p95: 2498ms ❌ 1 failed requests, 1 non-200 responses, 1 unexpected GraphQL errors
grafbase 173 5383 total, 0 failed avg: 886ms, p95: 2771ms
hive-gateway-bun 79 2653 total, 0 failed avg: 3554ms, p95: 5810ms
apollo-server 76 2570 total, 0 failed avg: 3674ms, p95: 31675ms
hive-gateway 71 2431 total, 0 failed avg: 3883ms, p95: 27065ms
mercurius 64 2066 total, 0 failed avg: 4449ms, p95: 6722ms
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 16890      ✗ 0    
     data_received..................: 496 MB  16 MB/s
     data_sent......................: 6.7 MB  217 kB/s
     http_req_blocked...............: avg=779.88µs min=1.72µs  med=3.42µs   max=1.04s    p(90)=5.62µs   p(95)=389.65µs
     http_req_connecting............: avg=464.49µs min=0s      med=0s       max=764.34ms p(90)=0s       p(95)=247.66µs
     http_req_duration..............: avg=865.27ms min=3.3ms   med=638.44ms max=6.54s    p(90)=1.88s    p(95)=2.37s   
       { expected_response:true }...: avg=865.27ms min=3.3ms   med=638.44ms max=6.54s    p(90)=1.88s    p(95)=2.37s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 5650 
     http_req_receiving.............: avg=389.32ms min=34.31µs med=120.81µs max=5.89s    p(90)=1.39s    p(95)=1.75s   
     http_req_sending...............: avg=22.18ms  min=8.29µs  med=15.37µs  max=3.78s    p(90)=186.5µs  p(95)=13.99ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=453.76ms min=3.23ms  med=429.46ms max=2.71s    p(90)=785.97ms p(95)=936.36ms
     http_reqs......................: 5650    183.043798/s
     iteration_duration.............: avg=1.61s    min=20.95ms med=1.24s    max=9.22s    p(90)=3.48s    p(95)=4.16s   
     iterations.....................: 5630    182.395855/s
     vus............................: 300     min=300      max=300
     vus_max........................: 300     min=300      max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✗ response code was 200
      ↳  99% — ✓ 5382 / ✗ 1
     ✗ no graphql errors
      ↳  99% — ✓ 5382 / ✗ 1
     ✓ valid response structure

     █ setup

     checks.........................: 99.98% ✓ 16146      ✗ 2    
     data_received..................: 474 MB 15 MB/s
     data_sent......................: 6.4 MB 206 kB/s
     http_req_blocked...............: avg=981.93µs min=1.5µs   med=3.11µs   max=1.75s  p(90)=5.53µs   p(95)=825.71µs
     http_req_connecting............: avg=920.63µs min=0s      med=0s       max=1.75s  p(90)=0s       p(95)=647.59µs
     http_req_duration..............: avg=904.59ms min=6.48ms  med=627.64ms max=6.4s   p(90)=2.04s    p(95)=2.49s   
       { expected_response:true }...: avg=904.34ms min=6.48ms  med=627.62ms max=6.4s   p(90)=2.04s    p(95)=2.49s   
     http_req_failed................: 0.01%  ✓ 1          ✗ 5402 
     http_req_receiving.............: avg=390.64ms min=0s      med=97.72µs  max=5.88s  p(90)=1.47s    p(95)=1.97s   
     http_req_sending...............: avg=28.54ms  min=7.94µs  med=14.35µs  max=3.54s  p(90)=365.5µs  p(95)=20.07ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s     p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=485.4ms  min=6.41ms  med=447.82ms max=1.74s  p(90)=826.23ms p(95)=961.99ms
     http_reqs......................: 5403   173.444616/s
     iteration_duration.............: avg=1.68s    min=40.78ms med=1.32s    max=12.34s p(90)=3.58s    p(95)=4.37s   
     iterations.....................: 5383   172.802585/s
     vus............................: 53     min=53       max=300
     vus_max........................: 300    min=300      max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 16089      ✗ 0    
     data_received..................: 473 MB  15 MB/s
     data_sent......................: 6.4 MB  206 kB/s
     http_req_blocked...............: avg=1.04ms   min=1.33µs  med=3.18µs   max=2.06s  p(90)=4.86µs   p(95)=629.46µs
     http_req_connecting............: avg=882.79µs min=0s      med=0s       max=2.06s  p(90)=0s       p(95)=380.47µs
     http_req_duration..............: avg=886.16ms min=3.39ms  med=671.6ms  max=5.34s  p(90)=2.06s    p(95)=2.77s   
       { expected_response:true }...: avg=886.16ms min=3.39ms  med=671.6ms  max=5.34s  p(90)=2.06s    p(95)=2.77s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 5383 
     http_req_receiving.............: avg=390.36ms min=33.29µs med=88.12µs  max=4.3s   p(90)=1.56s    p(95)=1.87s   
     http_req_sending...............: avg=16.37ms  min=8.26µs  med=14.26µs  max=3.75s  p(90)=122.02µs p(95)=833µs   
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s     p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=479.42ms min=3.3ms   med=417.85ms max=1.81s  p(90)=947.51ms p(95)=1.17s   
     http_reqs......................: 5383    173.614786/s
     iteration_duration.............: avg=1.7s     min=24.79ms med=1.32s    max=10.79s p(90)=3.75s    p(95)=4.58s   
     iterations.....................: 5363    172.969737/s
     vus............................: 19      min=19       max=300
     vus_max........................: 300     min=300      max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 7899      ✗ 0    
     data_received..................: 233 MB  7.0 MB/s
     data_sent......................: 3.1 MB  95 kB/s
     http_req_blocked...............: avg=1.17ms   min=1.63µs   med=4.26µs  max=23.01ms p(90)=2.27ms   p(95)=11.04ms
     http_req_connecting............: avg=1.14ms   min=0s       med=0s      max=22.96ms p(90)=819.77µs p(95)=10.87ms
     http_req_duration..............: avg=3.55s    min=16.94ms  med=3.27s   max=8.89s   p(90)=4.85s    p(95)=5.8s   
       { expected_response:true }...: avg=3.55s    min=16.94ms  med=3.27s   max=8.89s   p(90)=4.85s    p(95)=5.8s   
     http_req_failed................: 0.00%   ✓ 0         ✗ 2653 
     http_req_receiving.............: avg=57.11ms  min=44.24µs  med=362.1µs max=1.63s   p(90)=4.4ms    p(95)=471.5ms
     http_req_sending...............: avg=279.87µs min=9µs      med=24.63µs max=21.24ms p(90)=284.08µs p(95)=1.69ms 
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s      max=0s      p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=3.49s    min=16.77ms  med=3.25s   max=8.88s   p(90)=4.78s    p(95)=5.8s   
     http_reqs......................: 2653    79.753333/s
     iteration_duration.............: avg=3.6s     min=237.35ms med=3.3s    max=8.9s    p(90)=4.9s     p(95)=5.84s  
     iterations.....................: 2633    79.152101/s
     vus............................: 34      min=34      max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-server`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 7650      ✗ 0    
     data_received..................: 226 MB  6.7 MB/s
     data_sent......................: 3.1 MB  91 kB/s
     http_req_blocked...............: avg=576.38µs min=1.5µs    med=2.86µs   max=15.08ms  p(90)=434.83µs p(95)=6.03ms  
     http_req_connecting............: avg=556.52µs min=0s       med=0s       max=12.66ms  p(90)=358.69µs p(95)=5.54ms  
     http_req_duration..............: avg=3.67s    min=12.25ms  med=1.43s    max=33.1s    p(90)=2.92s    p(95)=31.67s  
       { expected_response:true }...: avg=3.67s    min=12.25ms  med=1.43s    max=33.1s    p(90)=2.92s    p(95)=31.67s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 2570 
     http_req_receiving.............: avg=306.22µs min=41.91µs  med=108.62µs max=229.38ms p(90)=190.69µs p(95)=280.94µs
     http_req_sending...............: avg=189.2µs  min=8.56µs   med=14.61µs  max=31.24ms  p(90)=149.59µs p(95)=1.18ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=3.67s    min=12.16ms  med=1.43s    max=33.1s    p(90)=2.92s    p(95)=31.67s  
     http_reqs......................: 2570    76.521394/s
     iteration_duration.............: avg=3.71s    min=266.62ms med=1.44s    max=33.1s    p(90)=2.93s    p(95)=31.69s  
     iterations.....................: 2550    75.925897/s
     vus............................: 58      min=58      max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 7233      ✗ 0    
     data_received..................: 214 MB  6.3 MB/s
     data_sent......................: 2.9 MB  86 kB/s
     http_req_blocked...............: avg=1.45ms   min=1.53µs   med=4.26µs   max=76.54ms p(90)=1.5ms    p(95)=10.22ms 
     http_req_connecting............: avg=1.28ms   min=0s       med=0s       max=55.12ms p(90)=1.4ms    p(95)=9.94ms  
     http_req_duration..............: avg=3.88s    min=15.47ms  med=1.92s    max=33.18s  p(90)=4.08s    p(95)=27.06s  
       { expected_response:true }...: avg=3.88s    min=15.47ms  med=1.92s    max=33.18s  p(90)=4.08s    p(95)=27.06s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 2431 
     http_req_receiving.............: avg=352.94µs min=42.03µs  med=112.93µs max=70.06ms p(90)=354.52µs p(95)=853.36µs
     http_req_sending...............: avg=261.87µs min=8.89µs   med=24.23µs  max=16.14ms p(90)=266.28µs p(95)=744.64µs
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s      p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=3.88s    min=15.38ms  med=1.92s    max=33.18s  p(90)=4.08s    p(95)=27.05s  
     http_reqs......................: 2431    71.994271/s
     iteration_duration.............: avg=3.93s    min=213.81ms med=1.95s    max=33.19s  p(90)=4.2s     p(95)=27.35s  
     iterations.....................: 2411    71.401969/s
     vus............................: 65      min=65      max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `mercurius`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 6138      ✗ 0    
     data_received..................: 181 MB  5.7 MB/s
     data_sent......................: 2.5 MB  76 kB/s
     http_req_blocked...............: avg=2.55ms min=1.58µs   med=4.13µs   max=65.1ms   p(90)=7.01ms   p(95)=18.51ms 
     http_req_connecting............: avg=2.46ms min=0s       med=0s       max=60.47ms  p(90)=6.68ms   p(95)=17.48ms 
     http_req_duration..............: avg=4.44s  min=13.56ms  med=4.53s    max=9.09s    p(90)=5.46s    p(95)=6.72s   
       { expected_response:true }...: avg=4.44s  min=13.56ms  med=4.53s    max=9.09s    p(90)=5.46s    p(95)=6.72s   
     http_req_failed................: 0.00%   ✓ 0         ✗ 2066 
     http_req_receiving.............: avg=2.06ms min=39.83µs  med=107.62µs max=274.02ms p(90)=254.24µs p(95)=773.32µs
     http_req_sending...............: avg=1.96ms min=8.95µs   med=24.66µs  max=57.26ms  p(90)=4.3ms    p(95)=14.94ms 
     http_req_tls_handshaking.......: avg=0s     min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=4.44s  min=13.47ms  med=4.53s    max=9.09s    p(90)=5.45s    p(95)=6.7s    
     http_reqs......................: 2066    64.387023/s
     iteration_duration.............: avg=4.51s  min=370.46ms med=4.56s    max=9.1s     p(90)=5.47s    p(95)=6.76s   
     iterations.....................: 2046    63.763722/s
     vus............................: 30      min=30      max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview

Copy link

github-actions bot commented Apr 7, 2025

Overview for: federation/constant-vus-subgraphs-delay-resources

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 500 VUs over 30s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
cosmo 181 5711 total, 0 failed avg: 1240ms, p95: 3242ms
grafbase 165 5206 total, 0 failed avg: 1397ms, p95: 3530ms
apollo-router 158 5026 total, 12 failed avg: 1607ms, p95: 4009ms ❌ 12 failed requests, 12 non-200 responses, 12 unexpected GraphQL errors
hive-gateway-bun 88 3052 total, 0 failed avg: 5201ms, p95: 8938ms
hive-gateway 78 2815 total, 0 failed avg: 5765ms, p95: 33640ms
apollo-server 77 2807 total, 0 failed avg: 5865ms, p95: 34252ms
mercurius 66 2208 total, 0 failed avg: 7074ms, p95: 12145ms
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 17073      ✗ 0    
     data_received..................: 501 MB  16 MB/s
     data_sent......................: 6.8 MB  215 kB/s
     http_req_blocked...............: avg=5.12ms   min=1.42µs  med=3.56µs   max=2.53s  p(90)=13.6µs p(95)=33.88ms
     http_req_connecting............: avg=4.67ms   min=0s      med=0s       max=2.53s  p(90)=0s     p(95)=30.76ms
     http_req_duration..............: avg=1.24s    min=3.22ms  med=1.05s    max=5.45s  p(90)=2.66s  p(95)=3.24s  
       { expected_response:true }...: avg=1.24s    min=3.22ms  med=1.05s    max=5.45s  p(90)=2.66s  p(95)=3.24s  
     http_req_failed................: 0.00%   ✓ 0          ✗ 5711 
     http_req_receiving.............: avg=438.16ms min=34.81µs med=181.86µs max=4.99s  p(90)=1.76s  p(95)=2.14s  
     http_req_sending...............: avg=25.36ms  min=8.11µs  med=15.73µs  max=3.34s  p(90)=7.07ms p(95)=65.15ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s     p(90)=0s     p(95)=0s     
     http_req_waiting...............: avg=776.5ms  min=3.09ms  med=736.97ms max=2.87s  p(90)=1.4s   p(95)=1.72s  
     http_reqs......................: 5711    181.124106/s
     iteration_duration.............: avg=2.67s    min=22.75ms med=2.34s    max=11.46s p(90)=5.22s  p(95)=6.26s  
     iterations.....................: 5691    180.489807/s
     vus............................: 227     min=227      max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 15558      ✗ 0    
     data_received..................: 458 MB  15 MB/s
     data_sent......................: 6.2 MB  197 kB/s
     http_req_blocked...............: avg=5.94ms   min=1.78µs  med=3.78µs   max=2.82s  p(90)=99.51µs p(95)=36.28ms
     http_req_connecting............: avg=5.04ms   min=0s      med=0s       max=1.94s  p(90)=0s      p(95)=35.72ms
     http_req_duration..............: avg=1.39s    min=4.6ms   med=1.22s    max=5.71s  p(90)=2.63s   p(95)=3.52s  
       { expected_response:true }...: avg=1.39s    min=4.6ms   med=1.22s    max=5.71s  p(90)=2.63s   p(95)=3.52s  
     http_req_failed................: 0.00%   ✓ 0          ✗ 5206 
     http_req_receiving.............: avg=353.53ms min=32.93µs med=92.06µs  max=4.05s  p(90)=1.58s   p(95)=1.93s  
     http_req_sending...............: avg=26.97ms  min=7.68µs  med=18.59µs  max=2.7s   p(90)=1.1ms   p(95)=17.32ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s     p(90)=0s      p(95)=0s     
     http_req_waiting...............: avg=1.01s    min=4.51ms  med=948.54ms max=3.76s  p(90)=1.86s   p(95)=2.4s   
     http_reqs......................: 5206    165.65119/s
     iteration_duration.............: avg=2.93s    min=26.28ms med=2.58s    max=15.78s p(90)=5.66s   p(95)=6.55s  
     iterations.....................: 5186    165.014804/s
     vus............................: 317     min=317      max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✗ response code was 200
      ↳  99% — ✓ 4994 / ✗ 12
     ✗ no graphql errors
      ↳  99% — ✓ 4994 / ✗ 12
     ✓ valid response structure

     █ setup

     checks.........................: 99.84% ✓ 14982      ✗ 24   
     data_received..................: 440 MB 14 MB/s
     data_sent......................: 6.0 MB 189 kB/s
     http_req_blocked...............: avg=9.15ms   min=1.64µs  med=3.49µs   max=4.8s   p(90)=667.75µs p(95)=32.59ms
     http_req_connecting............: avg=8.46ms   min=0s      med=0s       max=4.8s   p(90)=327.67µs p(95)=32.11ms
     http_req_duration..............: avg=1.6s     min=7.31ms  med=1.23s    max=10.29s p(90)=3.42s    p(95)=4s     
       { expected_response:true }...: avg=1.6s     min=7.31ms  med=1.23s    max=10.29s p(90)=3.43s    p(95)=4s     
     http_req_failed................: 0.23%  ✓ 12         ✗ 5014 
     http_req_receiving.............: avg=580.17ms min=0s      med=102.03µs max=9.27s  p(90)=2.05s    p(95)=2.82s  
     http_req_sending...............: avg=25.69ms  min=7.9µs   med=16.51µs  max=3.79s  p(90)=1.44ms   p(95)=12.27ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s     p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=1s       min=7.23ms  med=897.56ms max=3.39s  p(90)=1.96s    p(95)=2.3s   
     http_reqs......................: 5026   158.817086/s
     iteration_duration.............: avg=3.05s    min=45.44ms med=2.63s    max=12.67s p(90)=6.08s    p(95)=7s     
     iterations.....................: 5006   158.185104/s
     vus............................: 277    min=277      max=500
     vus_max........................: 500    min=500      max=500

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 9096      ✗ 0    
     data_received..................: 268 MB  7.7 MB/s
     data_sent......................: 3.6 MB  105 kB/s
     http_req_blocked...............: avg=2.05ms   min=1.84µs  med=3.9µs    max=59.21ms  p(90)=9.04ms   p(95)=19.65ms 
     http_req_connecting............: avg=1.97ms   min=0s      med=0s       max=27.14ms  p(90)=8.75ms   p(95)=19.44ms 
     http_req_duration..............: avg=5.2s     min=15.26ms med=4.83s    max=11.89s   p(90)=7.32s    p(95)=8.93s   
       { expected_response:true }...: avg=5.2s     min=15.26ms med=4.83s    max=11.89s   p(90)=7.32s    p(95)=8.93s   
     http_req_failed................: 0.00%   ✓ 0         ✗ 3052 
     http_req_receiving.............: avg=58.02ms  min=37.59µs med=174.63µs max=2.59s    p(90)=5.5ms    p(95)=166.67ms
     http_req_sending...............: avg=841.79µs min=9.73µs  med=22.16µs  max=187.14ms p(90)=692.77µs p(95)=2.86ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=5.14s    min=15.1ms  med=4.8s     max=11.88s   p(90)=7.21s    p(95)=8.93s   
     http_reqs......................: 3052    88.247254/s
     iteration_duration.............: avg=5.29s    min=155.2ms med=4.9s     max=11.92s   p(90)=7.36s    p(95)=9.04s   
     iterations.....................: 3032    87.668963/s
     vus............................: 104     min=104     max=500
     vus_max........................: 500     min=500     max=500

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 8385      ✗ 0    
     data_received..................: 247 MB  6.9 MB/s
     data_sent......................: 3.3 MB  93 kB/s
     http_req_blocked...............: avg=2.66ms  min=2.14µs  med=4.57µs   max=34.18ms  p(90)=13.29ms  p(95)=20.45ms
     http_req_connecting............: avg=2.6ms   min=0s      med=0s       max=28.25ms  p(90)=13.1ms   p(95)=20.28ms
     http_req_duration..............: avg=5.76s   min=15.04ms med=2.27s    max=35.31s   p(90)=24.97s   p(95)=33.64s 
       { expected_response:true }...: avg=5.76s   min=15.04ms med=2.27s    max=35.31s   p(90)=24.97s   p(95)=33.64s 
     http_req_failed................: 0.00%   ✓ 0         ✗ 2815 
     http_req_receiving.............: avg=1.6ms   min=41.51µs med=116.84µs max=337.53ms p(90)=953.35µs p(95)=3.05ms 
     http_req_sending...............: avg=499.9µs min=10.66µs med=27.71µs  max=35.12ms  p(90)=1.4ms    p(95)=3.49ms 
     http_req_tls_handshaking.......: avg=0s      min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=5.76s   min=14.91ms med=2.27s    max=35.31s   p(90)=24.96s   p(95)=33.63s 
     http_reqs......................: 2815    78.478828/s
     iteration_duration.............: avg=5.83s   min=427.5ms med=2.32s    max=35.34s   p(90)=25.19s   p(95)=33.69s 
     iterations.....................: 2795    77.921252/s
     vus............................: 75      min=75      max=500
     vus_max........................: 500     min=500     max=500

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-server`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 8361      ✗ 0    
     data_received..................: 247 MB  6.8 MB/s
     data_sent......................: 3.3 MB  92 kB/s
     http_req_blocked...............: avg=1.87ms   min=1.54µs   med=2.93µs   max=26.06ms p(90)=5.96ms   p(95)=17.28ms
     http_req_connecting............: avg=1.83ms   min=0s       med=0s       max=26.03ms p(90)=5.73ms   p(95)=17.02ms
     http_req_duration..............: avg=5.86s    min=13.26ms  med=1.52s    max=35.82s  p(90)=32.47s   p(95)=34.25s 
       { expected_response:true }...: avg=5.86s    min=13.26ms  med=1.52s    max=35.82s  p(90)=32.47s   p(95)=34.25s 
     http_req_failed................: 0.00%   ✓ 0         ✗ 2807 
     http_req_receiving.............: avg=165.31µs min=41.44µs  med=108.24µs max=8.89ms  p(90)=192.39µs p(95)=277.1µs
     http_req_sending...............: avg=296.36µs min=9.13µs   med=15.92µs  max=16.04ms p(90)=701.26µs p(95)=1.49ms 
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s      p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=5.86s    min=13.18ms  med=1.52s    max=35.82s  p(90)=32.47s   p(95)=34.24s 
     http_reqs......................: 2807    77.213274/s
     iteration_duration.............: avg=5.92s    min=388.38ms med=1.53s    max=35.84s  p(90)=32.55s   p(95)=34.27s 
     iterations.....................: 2787    76.663126/s
     vus............................: 38      min=38      max=500
     vus_max........................: 500     min=500     max=500

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `mercurius`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 6564      ✗ 0    
     data_received..................: 194 MB  5.9 MB/s
     data_sent......................: 2.6 MB  79 kB/s
     http_req_blocked...............: avg=4.02ms   min=1.66µs   med=3.76µs  max=39.37ms p(90)=21.39ms  p(95)=26.68ms 
     http_req_connecting............: avg=3.97ms   min=0s       med=0s      max=39.33ms p(90)=21.18ms  p(95)=26.53ms 
     http_req_duration..............: avg=7.07s    min=13.43ms  med=7.32s   max=14.34s  p(90)=9.48s    p(95)=12.14s  
       { expected_response:true }...: avg=7.07s    min=13.43ms  med=7.32s   max=14.34s  p(90)=9.48s    p(95)=12.14s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 2208 
     http_req_receiving.............: avg=25.68ms  min=40.24µs  med=103.6µs max=1.23s   p(90)=548.75µs p(95)=194.82ms
     http_req_sending...............: avg=473.21µs min=9.73µs   med=22.12µs max=22.74ms p(90)=1.75ms   p(95)=2.94ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s      max=0s      p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=7.04s    min=13.34ms  med=7.32s   max=14.34s  p(90)=9.48s    p(95)=12.14s  
     http_reqs......................: 2208    66.843507/s
     iteration_duration.............: avg=7.18s    min=525.37ms med=7.37s   max=14.35s  p(90)=9.55s    p(95)=12.2s   
     iterations.....................: 2188    66.23804/s
     vus............................: 35      min=35      max=500
     vus_max........................: 500     min=500     max=500

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview

Copy link

github-actions bot commented Apr 7, 2025

Overview for: federation/ramping-vus

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. We are running a heavy load of concurrent VUs to measure response time and other stats, during stress. It measure things like memory usage, CPU usage, response times. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was trying to reach 2000 concurrent VUs over 60s

Comparison

Comparison

Gateway duration(p95)⬇️ RPS Requests Durations Notes
cosmo 5804ms 174 12266 total, 0 failed avg: 2459ms, p95: 5805ms, max: 13209ms, med: 2106ms
grafbase 6512ms 164 11531 total, 0 failed avg: 2818ms, p95: 6513ms, max: 15760ms, med: 2239ms
apollo-router 6680ms 166 11659 total, 440 failed avg: 2624ms, p95: 6680ms, max: 13473ms, med: 2386ms ❌ 440 failed requests, 440 non-200 responses, 440 unexpected GraphQL errors
hive-gateway-bun 28296ms 90 7394 total, 0 failed avg: 11656ms, p95: 28297ms, max: 35681ms, med: 10068ms
mercurius 40464ms 52 4748 total, 0 failed avg: 22654ms, p95: 40465ms, max: 41519ms, med: 23350ms
hive-gateway 53229ms 81 7294 total, 0 failed avg: 13109ms, p95: 53229ms, max: 58545ms, med: 3440ms
apollo-server 60000ms 77 7207 total, 565 failed avg: 12899ms, p95: 60000ms, max: 60017ms, med: 2348ms ❌ 565 failed requests, 565 non-200 responses, 565 unexpected GraphQL errors
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 36738      ✗ 0     
     data_received..................: 1.1 GB  15 MB/s
     data_sent......................: 15 MB   207 kB/s
     http_req_blocked...............: avg=267.07ms min=1.71µs  med=4.11µs  max=9.97s  p(90)=437.73ms p(95)=1.74s   
     http_req_connecting............: avg=264.29ms min=0s      med=0s      max=9.87s  p(90)=412.88ms p(95)=1.74s   
     http_req_duration..............: avg=2.45s    min=3.48ms  med=2.1s    max=13.2s  p(90)=5.26s    p(95)=5.8s    
       { expected_response:true }...: avg=2.45s    min=3.48ms  med=2.1s    max=13.2s  p(90)=5.26s    p(95)=5.8s    
     http_req_failed................: 0.00%   ✓ 0          ✗ 12266 
     http_req_receiving.............: avg=369.13ms min=33.03µs med=89.34µs max=10.17s p(90)=1.1s     p(95)=2.94s   
     http_req_sending...............: avg=115.34ms min=7.89µs  med=21.73µs max=6.65s  p(90)=202.22ms p(95)=743.68ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s     p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=1.97s    min=3.27ms  med=1.52s   max=13.2s  p(90)=4.86s    p(95)=5.35s   
     http_reqs......................: 12266   174.707074/s
     iteration_duration.............: avg=5.67s    min=9.34ms  med=5.08s   max=24.9s  p(90)=11.69s   p(95)=13.95s  
     iterations.....................: 12246   174.42221/s
     vus............................: 5       min=5        max=2000
     vus_max........................: 2000    min=2000     max=2000

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 34533      ✗ 0     
     data_received..................: 1.0 GB  15 MB/s
     data_sent......................: 14 MB   195 kB/s
     http_req_blocked...............: avg=379.45ms min=1.47µs  med=3.85µs  max=10.47s p(90)=915.67ms p(95)=3.54s   
     http_req_connecting............: avg=376.96ms min=0s      med=0s      max=10.47s p(90)=909.38ms p(95)=3.53s   
     http_req_duration..............: avg=2.81s    min=3.16ms  med=2.23s   max=15.76s p(90)=5.94s    p(95)=6.51s   
       { expected_response:true }...: avg=2.81s    min=3.16ms  med=2.23s   max=15.76s p(90)=5.94s    p(95)=6.51s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 11531 
     http_req_receiving.............: avg=366.68ms min=33.86µs med=88.06µs max=7.52s  p(90)=1.34s    p(95)=2.42s   
     http_req_sending...............: avg=155.8ms  min=8.03µs  med=19.08µs max=6.57s  p(90)=285.09ms p(95)=934.62ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s     p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=2.29s    min=3.1ms   med=1.52s   max=15.76s p(90)=5.57s    p(95)=5.97s   
     http_reqs......................: 11531   164.530766/s
     iteration_duration.............: avg=6.28s    min=9ms     med=5.58s   max=26.73s p(90)=12.12s   p(95)=14.87s  
     iterations.....................: 11511   164.245395/s
     vus............................: 5       min=5        max=1957
     vus_max........................: 2000    min=2000     max=2000

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✗ response code was 200
      ↳  96% — ✓ 11199 / ✗ 440
     ✗ no graphql errors
      ↳  96% — ✓ 11199 / ✗ 440
     ✓ valid response structure

     █ setup

     checks.........................: 97.44% ✓ 33597      ✗ 880   
     data_received..................: 984 MB 14 MB/s
     data_sent......................: 14 MB  197 kB/s
     http_req_blocked...............: avg=515.02ms min=1.45µs  med=3.96µs  max=12.25s p(90)=2.45s    p(95)=3.2s  
     http_req_connecting............: avg=433.54ms min=0s      med=0s      max=8.32s  p(90)=2.31s    p(95)=3.08s 
     http_req_duration..............: avg=2.62s    min=6.61ms  med=2.38s   max=13.47s p(90)=5.42s    p(95)=6.68s 
       { expected_response:true }...: avg=2.63s    min=6.61ms  med=2.4s    max=13.47s p(90)=5.44s    p(95)=6.69s 
     http_req_failed................: 3.77%  ✓ 440        ✗ 11219 
     http_req_receiving.............: avg=488.35ms min=0s      med=83.59µs max=9.33s  p(90)=1.62s    p(95)=3.7s  
     http_req_sending...............: avg=211.98ms min=7.99µs  med=21.24µs max=12.76s p(90)=389.19ms p(95)=1.04s 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s     p(90)=0s       p(95)=0s    
     http_req_waiting...............: avg=1.92s    min=6.54ms  med=1.84s   max=10.45s p(90)=3.58s    p(95)=4.05s 
     http_reqs......................: 11659  166.182207/s
     iteration_duration.............: avg=6.09s    min=13.04ms med=5.27s   max=33s    p(90)=12.15s   p(95)=15.01s
     iterations.....................: 11639  165.897136/s
     vus............................: 40     min=40       max=1925
     vus_max........................: 2000   min=2000     max=2000

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 22122     ✗ 0     
     data_received..................: 649 MB  7.9 MB/s
     data_sent......................: 8.8 MB  108 kB/s
     http_req_blocked...............: avg=11.64ms min=1.89µs  med=4.15µs  max=865.35ms p(90)=2.26ms p(95)=66.91ms 
     http_req_connecting............: avg=11.36ms min=0s      med=0s      max=865.28ms p(90)=1.96ms p(95)=66.03ms 
     http_req_duration..............: avg=11.65s  min=16.79ms med=10.06s  max=35.68s   p(90)=24.43s p(95)=28.29s  
       { expected_response:true }...: avg=11.65s  min=16.79ms med=10.06s  max=35.68s   p(90)=24.43s p(95)=28.29s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 7394  
     http_req_receiving.............: avg=97.84ms min=32.85µs med=194.2µs max=7.14s    p(90)=6ms    p(95)=314.61ms
     http_req_sending...............: avg=6.67ms  min=9.22µs  med=25.13µs max=655.51ms p(90)=1.55ms p(95)=35.91ms 
     http_req_tls_handshaking.......: avg=0s      min=0s      med=0s      max=0s       p(90)=0s     p(95)=0s      
     http_req_waiting...............: avg=11.55s  min=16.5ms  med=9.96s   max=35.53s   p(90)=24.39s p(95)=28.16s  
     http_reqs......................: 7394    90.559386/s
     iteration_duration.............: avg=11.89s  min=95.38ms med=10.26s  max=38.62s   p(90)=24.75s p(95)=28.6s   
     iterations.....................: 7374    90.314432/s
     vus............................: 142     min=59      max=1999
     vus_max........................: 2000    min=2000    max=2000

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `mercurius`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 14138     ✗ 0     
     data_received..................: 417 MB  4.6 MB/s
     data_sent......................: 5.6 MB  62 kB/s
     http_req_blocked...............: avg=206.63µs min=1.8µs   med=4.74µs   max=11.82ms  p(90)=456.53µs p(95)=655.17µs
     http_req_connecting............: avg=170.71µs min=0s      med=0s       max=11.75ms  p(90)=380.99µs p(95)=521.11µs
     http_req_duration..............: avg=22.65s   min=13.62ms med=23.34s   max=41.51s   p(90)=39.38s   p(95)=40.46s  
       { expected_response:true }...: avg=22.65s   min=13.62ms med=23.34s   max=41.51s   p(90)=39.38s   p(95)=40.46s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 4748  
     http_req_receiving.............: avg=4.39ms   min=36.74µs med=105.04µs max=910.31ms p(90)=293.76µs p(95)=735.48µs
     http_req_sending...............: avg=63.62µs  min=9.16µs  med=29.76µs  max=21.39ms  p(90)=64.2µs   p(95)=83.42µs 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=22.64s   min=13.51ms med=23.34s   max=41.51s   p(90)=39.37s   p(95)=40.45s  
     http_reqs......................: 4748    52.123505/s
     iteration_duration.............: avg=22.8s    min=89.71ms med=23.34s   max=41.53s   p(90)=39.55s   p(95)=40.59s  
     iterations.....................: 4702    51.618517/s
     vus............................: 166     min=53      max=2000
     vus_max........................: 2000    min=2000    max=2000

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 21822     ✗ 0     
     data_received..................: 641 MB  7.1 MB/s
     data_sent......................: 8.7 MB  96 kB/s
     http_req_blocked...............: avg=930.26µs min=2.03µs   med=4.59µs   max=118.89ms p(90)=516.15µs p(95)=2.05ms  
     http_req_connecting............: avg=893.84µs min=0s       med=0s       max=118.83ms p(90)=436.74µs p(95)=1.89ms  
     http_req_duration..............: avg=13.1s    min=15.74ms  med=3.44s    max=58.54s   p(90)=47.06s   p(95)=53.22s  
       { expected_response:true }...: avg=13.1s    min=15.74ms  med=3.44s    max=58.54s   p(90)=47.06s   p(95)=53.22s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 7294  
     http_req_receiving.............: avg=827.99µs min=41.38µs  med=113.22µs max=354.22ms p(90)=787.43µs p(95)=2.42ms  
     http_req_sending...............: avg=405.96µs min=9.15µs   med=27.88µs  max=71.17ms  p(90)=75.34µs  p(95)=350.07µs
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=13.1s    min=15.49ms  med=3.43s    max=58.54s   p(90)=47.06s   p(95)=53.22s  
     http_reqs......................: 7294    81.11702/s
     iteration_duration.............: avg=13.17s   min=148.47ms med=3.48s    max=58.59s   p(90)=47.11s   p(95)=53.25s  
     iterations.....................: 7274    80.894598/s
     vus............................: 21      min=21      max=2000
     vus_max........................: 2000    min=2000    max=2000

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-server`

K6 Output

     ✗ response code was 200
      ↳  92% — ✓ 6622 / ✗ 565
     ✗ no graphql errors
      ↳  92% — ✓ 6622 / ✗ 565
     ✓ valid response structure

     █ setup

     checks.........................: 94.61% ✓ 19866     ✗ 1130  
     data_received..................: 584 MB 6.3 MB/s
     data_sent......................: 8.6 MB 93 kB/s
     http_req_blocked...............: avg=292.49µs min=1.35µs  med=3.49µs   max=56.3ms  p(90)=326.84µs p(95)=477.69µs
     http_req_connecting............: avg=269.89µs min=0s      med=0s       max=56.24ms p(90)=261.01µs p(95)=396.81µs
     http_req_duration..............: avg=12.89s   min=13.11ms med=2.34s    max=1m0s    p(90)=57.21s   p(95)=1m0s    
       { expected_response:true }...: avg=8.89s    min=13.11ms med=2.26s    max=59.99s  p(90)=37.93s   p(95)=44.51s  
     http_req_failed................: 7.83%  ✓ 565       ✗ 6642  
     http_req_receiving.............: avg=149.7µs  min=0s      med=110.16µs max=17.72ms p(90)=195.84µs p(95)=293.86µs
     http_req_sending...............: avg=118.77µs min=8.44µs  med=18.37µs  max=35.53ms p(90)=58.73µs  p(95)=88.25µs 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s      p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=12.89s   min=12.91ms med=2.34s    max=1m0s    p(90)=57.21s   p(95)=1m0s    
     http_reqs......................: 7207   77.668394/s
     iteration_duration.............: avg=12.94s   min=69.06ms med=2.36s    max=1m0s    p(90)=57.26s   p(95)=1m0s    
     iterations.....................: 7187   77.452858/s
     vus............................: 19     min=19      max=2000
     vus_max........................: 2000   min=2000    max=2000

Performance Overview

Performance Overview

Subgraphs Overview

Subgraphs Overview

HTTP Overview

HTTP Overview

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0 participants