Skip to content
bench

Stream subgraph output while it executes #87

Sign in to view logs

Stream subgraph output while it executes

Stream subgraph output while it executes #87

Triggered via pull request September 17, 2024 01:04
@nfcamposnfcampos
opened #1735
nc/16sep/stream-subgraph-in-progress
Status Success
Total duration 23m 35s
Artifacts –

bench.yml

on: pull_request
benchmark
23m 23s
benchmark
Fit to window
Zoom out
Zoom in

Annotations

1 warning and 2 notices
benchmark
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Benchmark results: libs/langgraph/langgraph/pregel/__init__.py#L1
......................................... fanout_to_subgraph_10x: Mean +- std dev: 63.7 ms +- 6.1 ms ......................................... fanout_to_subgraph_10x_checkpoint: Mean +- std dev: 79.1 ms +- 1.1 ms ......................................... fanout_to_subgraph_100x: Mean +- std dev: 589 ms +- 20 ms ......................................... fanout_to_subgraph_100x_checkpoint: Mean +- std dev: 788 ms +- 18 ms ......................................... react_agent_10x: Mean +- std dev: 39.4 ms +- 0.7 ms ......................................... react_agent_10x_checkpoint: Mean +- std dev: 53.3 ms +- 1.5 ms ......................................... react_agent_100x: Mean +- std dev: 412 ms +- 7 ms ......................................... react_agent_100x_checkpoint: Mean +- std dev: 980 ms +- 11 ms ......................................... wide_state_25x300: Mean +- std dev: 21.2 ms +- 0.2 ms ......................................... wide_state_25x300_checkpoint: Mean +- std dev: 246 ms +- 7 ms ......................................... WARNING: the benchmark result may be unstable * the standard deviation (3.49 ms) is 14% of the mean (25.0 ms) * the maximum (62.9 ms) is 151% greater than the mean (25.0 ms) Try to rerun the benchmark with more runs, values and/or loops. Run 'python -m pyperf system tune' command to reduce the system jitter. Use pyperf stats, pyperf dump and pyperf hist to analyze results. Use --quiet option to hide these warnings. wide_state_15x600: Mean +- std dev: 25.0 ms +- 3.5 ms ......................................... wide_state_15x600_checkpoint: Mean +- std dev: 420 ms +- 13 ms ......................................... wide_state_9x1200: Mean +- std dev: 24.6 ms +- 0.3 ms ......................................... wide_state_9x1200_checkpoint: Mean +- std dev: 285 ms +- 19 ms
Comparison against main: libs/langgraph/langgraph/pregel/__init__.py#L1
+------------------------------------+---------+-----------------------+ | Benchmark | main | changes | +====================================+=========+=======================+ | react_agent_100x_checkpoint | 967 ms | 980 ms: 1.01x slower | +------------------------------------+---------+-----------------------+ | wide_state_25x300_checkpoint | 242 ms | 246 ms: 1.01x slower | +------------------------------------+---------+-----------------------+ | react_agent_10x_checkpoint | 52.4 ms | 53.3 ms: 1.02x slower | +------------------------------------+---------+-----------------------+ | react_agent_10x | 38.7 ms | 39.4 ms: 1.02x slower | +------------------------------------+---------+-----------------------+ | react_agent_100x | 404 ms | 412 ms: 1.02x slower | +------------------------------------+---------+-----------------------+ | wide_state_25x300 | 20.7 ms | 21.2 ms: 1.02x slower | +------------------------------------+---------+-----------------------+ | wide_state_9x1200 | 24.1 ms | 24.6 ms: 1.02x slower | +------------------------------------+---------+-----------------------+ | fanout_to_subgraph_100x_checkpoint | 769 ms | 788 ms: 1.02x slower | +------------------------------------+---------+-----------------------+ | fanout_to_subgraph_100x | 574 ms | 589 ms: 1.03x slower | +------------------------------------+---------+-----------------------+ | fanout_to_subgraph_10x | 61.9 ms | 63.7 ms: 1.03x slower | +------------------------------------+---------+-----------------------+ | wide_state_15x600 | 24.1 ms | 25.0 ms: 1.04x slower | +------------------------------------+---------+-----------------------+ | wide_state_9x1200_checkpoint | 271 ms | 285 ms: 1.05x slower | +------------------------------------+---------+-----------------------+ | Geometric mean | (ref) | 1.02x slower | +------------------------------------+---------+-----------------------+ Benchmark hidden because not significant (2): fanout_to_subgraph_10x_checkpoint, wide_state_15x600_checkpoint