Retry subgraph starting at failing node #81
Triggered via pull request
September 16, 2024 18:38
hinthornw
Status
Success
Total duration
31m 51s
Artifacts
–
Annotations
1 warning and 2 notices
|
benchmark
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
|
|
Benchmark results:
libs/langgraph/langgraph/pregel/retry.py#L1
.........................................
fanout_to_subgraph_10x: Mean +- std dev: 58.2 ms +- 1.0 ms
.........................................
fanout_to_subgraph_10x_checkpoint: Mean +- std dev: 94.4 ms +- 6.1 ms
.........................................
fanout_to_subgraph_100x: Mean +- std dev: 574 ms +- 18 ms
.........................................
fanout_to_subgraph_100x_checkpoint: Mean +- std dev: 931 ms +- 32 ms
.........................................
react_agent_10x: Mean +- std dev: 43.7 ms +- 0.8 ms
.........................................
react_agent_10x_checkpoint: Mean +- std dev: 77.7 ms +- 1.6 ms
.........................................
react_agent_100x: Mean +- std dev: 784 ms +- 10 ms
.........................................
react_agent_100x_checkpoint: Mean +- std dev: 2.75 sec +- 0.10 sec
.........................................
wide_state_25x300: Mean +- std dev: 20.8 ms +- 0.3 ms
.........................................
wide_state_25x300_checkpoint: Mean +- std dev: 578 ms +- 14 ms
.........................................
wide_state_15x600: Mean +- std dev: 24.2 ms +- 0.4 ms
.........................................
wide_state_15x600_checkpoint: Mean +- std dev: 1.04 sec +- 0.03 sec
.........................................
wide_state_9x1200: Mean +- std dev: 24.2 ms +- 0.4 ms
.........................................
wide_state_9x1200_checkpoint: Mean +- std dev: 662 ms +- 17 ms
|
|
Comparison against main:
libs/langgraph/langgraph/pregel/retry.py#L1
+------------------------------------+----------+------------------------+
| Benchmark | main | changes |
+====================================+==========+========================+
| fanout_to_subgraph_100x | 594 ms | 574 ms: 1.03x faster |
+------------------------------------+----------+------------------------+
| fanout_to_subgraph_10x | 59.1 ms | 58.2 ms: 1.02x faster |
+------------------------------------+----------+------------------------+
| react_agent_100x_checkpoint | 2.79 sec | 2.75 sec: 1.01x faster |
+------------------------------------+----------+------------------------+
| wide_state_9x1200 | 24.5 ms | 24.2 ms: 1.01x faster |
+------------------------------------+----------+------------------------+
| react_agent_10x_checkpoint | 78.8 ms | 77.7 ms: 1.01x faster |
+------------------------------------+----------+------------------------+
| react_agent_10x | 44.3 ms | 43.7 ms: 1.01x faster |
+------------------------------------+----------+------------------------+
| wide_state_15x600 | 24.5 ms | 24.2 ms: 1.01x faster |
+------------------------------------+----------+------------------------+
| fanout_to_subgraph_100x_checkpoint | 942 ms | 931 ms: 1.01x faster |
+------------------------------------+----------+------------------------+
| react_agent_100x | 792 ms | 784 ms: 1.01x faster |
+------------------------------------+----------+------------------------+
| wide_state_25x300 | 21.0 ms | 20.8 ms: 1.01x faster |
+------------------------------------+----------+------------------------+
| Geometric mean | (ref) | 1.01x faster |
+------------------------------------+----------+------------------------+
Benchmark hidden because not significant (4): fanout_to_subgraph_10x_checkpoint, wide_state_9x1200_checkpoint, wide_state_15x600_checkpoint, wide_state_25x300_checkpoint
|