Implement serialization with msgpack library #84
Triggered via pull request
September 16, 2024 20:40
nfcampos
Status
Success
Total duration
24m 14s
Artifacts
–
Annotations
1 warning and 2 notices
|
benchmark
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
|
|
Benchmark results:
libs/checkpoint-postgres/langgraph/checkpoint/postgres/base.py#L1
.........................................
WARNING: the benchmark result may be unstable
* the standard deviation (7.75 ms) is 12% of the mean (63.8 ms)
Try to rerun the benchmark with more runs, values and/or loops.
Run 'python -m pyperf system tune' command to reduce the system jitter.
Use pyperf stats, pyperf dump and pyperf hist to analyze results.
Use --quiet option to hide these warnings.
fanout_to_subgraph_10x: Mean +- std dev: 63.8 ms +- 7.8 ms
.........................................
fanout_to_subgraph_10x_checkpoint: Mean +- std dev: 80.2 ms +- 2.4 ms
.........................................
fanout_to_subgraph_100x: Mean +- std dev: 607 ms +- 26 ms
.........................................
fanout_to_subgraph_100x_checkpoint: Mean +- std dev: 841 ms +- 49 ms
.........................................
react_agent_10x: Mean +- std dev: 39.1 ms +- 0.8 ms
.........................................
react_agent_10x_checkpoint: Mean +- std dev: 52.6 ms +- 1.6 ms
.........................................
react_agent_100x: Mean +- std dev: 408 ms +- 7 ms
.........................................
react_agent_100x_checkpoint: Mean +- std dev: 1.01 sec +- 0.02 sec
.........................................
wide_state_25x300: Mean +- std dev: 21.2 ms +- 0.3 ms
.........................................
wide_state_25x300_checkpoint: Mean +- std dev: 247 ms +- 9 ms
.........................................
wide_state_15x600: Mean +- std dev: 24.5 ms +- 0.3 ms
.........................................
wide_state_15x600_checkpoint: Mean +- std dev: 421 ms +- 13 ms
.........................................
wide_state_9x1200: Mean +- std dev: 24.4 ms +- 0.3 ms
.........................................
wide_state_9x1200_checkpoint: Mean +- std dev: 272 ms +- 7 ms
|
|
Comparison against main:
libs/checkpoint-postgres/langgraph/checkpoint/postgres/base.py#L1
+------------------------------------+----------+------------------------+
| Benchmark | main | changes |
+====================================+==========+========================+
| react_agent_100x_checkpoint | 2.58 sec | 1.01 sec: 2.56x faster |
+------------------------------------+----------+------------------------+
| wide_state_15x600_checkpoint | 1.04 sec | 421 ms: 2.47x faster |
+------------------------------------+----------+------------------------+
| wide_state_9x1200_checkpoint | 655 ms | 272 ms: 2.41x faster |
+------------------------------------+----------+------------------------+
| wide_state_25x300_checkpoint | 573 ms | 247 ms: 2.31x faster |
+------------------------------------+----------+------------------------+
| react_agent_10x_checkpoint | 73.9 ms | 52.6 ms: 1.41x faster |
+------------------------------------+----------+------------------------+
| fanout_to_subgraph_10x_checkpoint | 90.1 ms | 80.2 ms: 1.12x faster |
+------------------------------------+----------+------------------------+
| fanout_to_subgraph_100x_checkpoint | 897 ms | 841 ms: 1.07x faster |
+------------------------------------+----------+------------------------+
| wide_state_25x300 | 21.1 ms | 21.2 ms: 1.00x slower |
+------------------------------------+----------+------------------------+
| react_agent_100x | 406 ms | 408 ms: 1.01x slower |
+------------------------------------+----------+------------------------+
| wide_state_15x600 | 24.4 ms | 24.5 ms: 1.01x slower |
+------------------------------------+----------+------------------------+
| fanout_to_subgraph_100x | 575 ms | 607 ms: 1.06x slower |
+------------------------------------+----------+------------------------+
| Geometric mean | (ref) | 1.33x faster |
+------------------------------------+----------+------------------------+
Benchmark hidden because not significant (3): wide_state_9x1200, react_agent_10x, fanout_to_subgraph_10x
|