[Test] Improve coverage of ChessEnv.all_actions

[kurtamohler]

Stack from ghstack (oldest at bottom):

• -> [Test] Improve coverage of ChessEnv.all_actions #2849

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2849

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 8 New Failures, 3 Unrelated Failures

As of commit f33a064 with merge base efe9389 ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  Process completed with exit code 1.
• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Process completed with exit code 1.
• Generate documentation / build-docs (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 1ae17cf2c2f7fe08b41fb4c07ffe3900c5a2ba1481ac353e26f0732f153baf37 /exec failed with exit code 2
• Habitat Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f9667313d898b369b99d5e0e14c2c5ec3fa0b681b3fefc678983e0fd861f318 /exec failed with exit code 1
• SOTA Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 41cb9519573be6adee68a09eb9abee85b2b5aab3ec2c7c8426418139102ce2aa /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh)
  test/test_env.py::TestNonTensorEnv::test_parallel[False-True]
• Unit-tests on Linux / tests-cpu (3.12) / linux-job (gh)
  test/test_env.py::TestNonTensorEnv::test_parallel[False-True]
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh)
  RuntimeError: Command docker exec -t 1d33c1a5d3dcaf0834b779349434758a5b14ebf46879d7223a1f1037361f6833 /exec failed with exit code 1

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• LLM Tests on Linux / unittests (3.9, 12.8) / linux-job (gh) (trunk failure)
  ##[error]The operation was canceled.
• Unit-tests on Linux / tests-optdeps (3.11, 12.8) / linux-job (gh) (trunk failure)
  test/test_cost.py::TestPPO4LLMs::test_hf[False]
• Unit-tests on Windows / unittests-cpu (3.10, windows.4xlarge, cpu) / windows-job (gh) (trunk failure)
  test/test_transforms.py::TestTimer::test_transform_env

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_simple	0.6303s	0.5423s	1.8440 Ops/s	1.8766 Ops/s	$\color{#d91a1a}-1.74\%$
test_transformed	1.1722s	1.0907s	0.9169 Ops/s	0.9517 Ops/s	$\color{#d91a1a}-3.66\%$
test_serial	1.5706s	1.5687s	0.6375 Ops/s	0.6448 Ops/s	$\color{#d91a1a}-1.13\%$
test_parallel	1.4005s	1.3176s	0.7589 Ops/s	0.7492 Ops/s	$\color{#35bf28}+1.31\%$
test_step_mdp_speed[True-True-True-True-True]	0.1923ms	29.5824μs	33.8039 KOps/s	33.0250 KOps/s	$\color{#35bf28}+2.36\%$
test_step_mdp_speed[True-True-True-True-False]	44.7940μs	17.8182μs	56.1225 KOps/s	55.6621 KOps/s	$\color{#35bf28}+0.83\%$
test_step_mdp_speed[True-True-True-False-True]	85.5620μs	17.1096μs	58.4466 KOps/s	57.0433 KOps/s	$\color{#35bf28}+2.46\%$
test_step_mdp_speed[True-True-True-False-False]	33.2630μs	9.9652μs	100.3496 KOps/s	96.6687 KOps/s	$\color{#35bf28}+3.81\%$
test_step_mdp_speed[True-True-False-True-True]	70.7640μs	32.1035μs	31.1493 KOps/s	30.8670 KOps/s	$\color{#35bf28}+0.91\%$
test_step_mdp_speed[True-True-False-True-False]	49.9840μs	19.6306μs	50.9410 KOps/s	50.6803 KOps/s	$\color{#35bf28}+0.51\%$
test_step_mdp_speed[True-True-False-False-True]	0.6429ms	18.9685μs	52.7190 KOps/s	52.2037 KOps/s	$\color{#35bf28}+0.99\%$
test_step_mdp_speed[True-True-False-False-False]	68.6670μs	11.7482μs	85.1197 KOps/s	83.3823 KOps/s	$\color{#35bf28}+2.08\%$
test_step_mdp_speed[True-False-True-True-True]	65.8640μs	33.7119μs	29.6631 KOps/s	28.8531 KOps/s	$\color{#35bf28}+2.81\%$
test_step_mdp_speed[True-False-True-True-False]	58.0400μs	21.3264μs	46.8903 KOps/s	46.3344 KOps/s	$\color{#35bf28}+1.20\%$
test_step_mdp_speed[True-False-True-False-True]	55.5450μs	18.8208μs	53.1328 KOps/s	52.3444 KOps/s	$\color{#35bf28}+1.51\%$
test_step_mdp_speed[True-False-True-False-False]	50.2040μs	11.8302μs	84.5292 KOps/s	82.5561 KOps/s	$\color{#35bf28}+2.39\%$
test_step_mdp_speed[True-False-False-True-True]	83.9390μs	35.3803μs	28.2644 KOps/s	27.4778 KOps/s	$\color{#35bf28}+2.86\%$
test_step_mdp_speed[True-False-False-True-False]	73.0010μs	23.0275μs	43.4264 KOps/s	42.5446 KOps/s	$\color{#35bf28}+2.07\%$
test_step_mdp_speed[True-False-False-False-True]	66.6840μs	20.5418μs	48.6813 KOps/s	47.7854 KOps/s	$\color{#35bf28}+1.87\%$
test_step_mdp_speed[True-False-False-False-False]	45.2150μs	13.5052μs	74.0458 KOps/s	71.8680 KOps/s	$\color{#35bf28}+3.03\%$
test_step_mdp_speed[False-True-True-True-True]	73.5090μs	33.4993μs	29.8514 KOps/s	29.0387 KOps/s	$\color{#35bf28}+2.80\%$
test_step_mdp_speed[False-True-True-True-False]	71.5790μs	21.3186μs	46.9074 KOps/s	45.4844 KOps/s	$\color{#35bf28}+3.13\%$
test_step_mdp_speed[False-True-True-False-True]	45.5660μs	21.5356μs	46.4348 KOps/s	46.4151 KOps/s	$\color{#35bf28}+0.04\%$
test_step_mdp_speed[False-True-True-False-False]	39.4350μs	13.3377μs	74.9753 KOps/s	74.7132 KOps/s	$\color{#35bf28}+0.35\%$
test_step_mdp_speed[False-True-False-True-True]	77.7370μs	35.2709μs	28.3520 KOps/s	27.9939 KOps/s	$\color{#35bf28}+1.28\%$
test_step_mdp_speed[False-True-False-True-False]	2.6999ms	23.2995μs	42.9193 KOps/s	42.8988 KOps/s	$\color{#35bf28}+0.05\%$
test_step_mdp_speed[False-True-False-False-True]	67.4370μs	23.3994μs	42.7361 KOps/s	43.0601 KOps/s	$\color{#d91a1a}-0.75\%$
test_step_mdp_speed[False-True-False-False-False]	46.2270μs	15.0748μs	66.3359 KOps/s	66.3900 KOps/s	$\color{#d91a1a}-0.08\%$
test_step_mdp_speed[False-False-True-True-True]	0.7228ms	37.2493μs	26.8461 KOps/s	26.4021 KOps/s	$\color{#35bf28}+1.68\%$
test_step_mdp_speed[False-False-True-True-False]	61.4860μs	25.4755μs	39.2534 KOps/s	39.5985 KOps/s	$\color{#d91a1a}-0.87\%$
test_step_mdp_speed[False-False-True-False-True]	56.2660μs	23.5598μs	42.4452 KOps/s	42.4832 KOps/s	$\color{#d91a1a}-0.09\%$
test_step_mdp_speed[False-False-True-False-False]	41.9890μs	15.0109μs	66.6184 KOps/s	65.8885 KOps/s	$\color{#35bf28}+1.11\%$
test_step_mdp_speed[False-False-False-True-True]	74.8210μs	38.6901μs	25.8464 KOps/s	25.4019 KOps/s	$\color{#35bf28}+1.75\%$
test_step_mdp_speed[False-False-False-True-False]	56.8480μs	26.5472μs	37.6687 KOps/s	36.9708 KOps/s	$\color{#35bf28}+1.89\%$
test_step_mdp_speed[False-False-False-False-True]	56.5160μs	24.7350μs	40.4285 KOps/s	39.6499 KOps/s	$\color{#35bf28}+1.96\%$
test_step_mdp_speed[False-False-False-False-False]	51.4170μs	16.6238μs	60.1548 KOps/s	59.8465 KOps/s	$\color{#35bf28}+0.52\%$
test_values[generalized_advantage_estimate-True-True]	10.3447ms	10.0459ms	99.5432 Ops/s	102.2812 Ops/s	$\color{#d91a1a}-2.68\%$
test_values[vec_generalized_advantage_estimate-True-True]	28.5287ms	26.8176ms	37.2890 Ops/s	41.4017 Ops/s	$\textbf{\color{#d91a1a}-9.93\%}$
test_values[td0_return_estimate-False-False]	0.2245ms	0.1751ms	5.7126 KOps/s	5.6234 KOps/s	$\color{#35bf28}+1.59\%$
test_values[td1_return_estimate-False-False]	24.7183ms	24.2127ms	41.3007 Ops/s	42.5004 Ops/s	$\color{#d91a1a}-2.82\%$
test_values[vec_td1_return_estimate-False-False]	31.7534ms	26.9691ms	37.0794 Ops/s	41.3079 Ops/s	$\textbf{\color{#d91a1a}-10.24\%}$
test_values[td_lambda_return_estimate-True-False]	37.4360ms	35.1277ms	28.4675 Ops/s	29.3921 Ops/s	$\color{#d91a1a}-3.15\%$
test_values[vec_td_lambda_return_estimate-True-False]	28.6484ms	26.8710ms	37.2148 Ops/s	41.1385 Ops/s	$\textbf{\color{#d91a1a}-9.54\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.8819ms	8.6388ms	115.7570 Ops/s	118.6751 Ops/s	$\color{#d91a1a}-2.46\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	3.3967ms	1.9902ms	502.4589 Ops/s	507.7623 Ops/s	$\color{#d91a1a}-1.04\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4474ms	0.3712ms	2.6942 KOps/s	2.7090 KOps/s	$\color{#d91a1a}-0.55\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	47.0872ms	45.9069ms	21.7832 Ops/s	25.5293 Ops/s	$\textbf{\color{#d91a1a}-14.67\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	4.4495ms	3.4557ms	289.3749 Ops/s	289.9859 Ops/s	$\color{#d91a1a}-0.21\%$
test_dqn_speed[False-None]	5.8890ms	1.4211ms	703.6609 Ops/s	698.5857 Ops/s	$\color{#35bf28}+0.73\%$
test_dqn_speed[False-backward]	2.5806ms	1.9357ms	516.6067 Ops/s	526.0678 Ops/s	$\color{#d91a1a}-1.80\%$
test_dqn_speed[True-None]	0.8860ms	0.5547ms	1.8029 KOps/s	1.7771 KOps/s	$\color{#35bf28}+1.45\%$
test_dqn_speed[True-backward]	1.0039ms	0.9653ms	1.0359 KOps/s	1.0157 KOps/s	$\color{#35bf28}+1.99\%$
test_dqn_speed[reduce-overhead-None]	0.6554ms	0.5511ms	1.8146 KOps/s	1.7617 KOps/s	$\color{#35bf28}+3.00\%$
test_dqn_speed[reduce-overhead-backward]	1.0273ms	0.9648ms	1.0365 KOps/s	1.0215 KOps/s	$\color{#35bf28}+1.46\%$
test_ddpg_speed[False-None]	3.8389ms	2.9139ms	343.1873 Ops/s	347.4311 Ops/s	$\color{#d91a1a}-1.22\%$
test_ddpg_speed[False-backward]	4.1758ms	4.0519ms	246.8002 Ops/s	247.1234 Ops/s	$\color{#d91a1a}-0.13\%$
test_ddpg_speed[True-None]	1.9010ms	1.4278ms	700.3893 Ops/s	687.8850 Ops/s	$\color{#35bf28}+1.82\%$
test_ddpg_speed[True-backward]	3.0080ms	2.3701ms	421.9294 Ops/s	424.2564 Ops/s	$\color{#d91a1a}-0.55\%$
test_ddpg_speed[reduce-overhead-None]	2.1160ms	1.4288ms	699.8896 Ops/s	674.2311 Ops/s	$\color{#35bf28}+3.81\%$
test_ddpg_speed[reduce-overhead-backward]	2.7718ms	2.3640ms	423.0169 Ops/s	427.8405 Ops/s	$\color{#d91a1a}-1.13\%$
test_sac_speed[False-None]	8.4182ms	8.0410ms	124.3633 Ops/s	124.1005 Ops/s	$\color{#35bf28}+0.21\%$
test_sac_speed[False-backward]	11.8042ms	10.8447ms	92.2107 Ops/s	92.9998 Ops/s	$\color{#d91a1a}-0.85\%$
test_sac_speed[True-None]	5.8838ms	2.5626ms	390.2346 Ops/s	385.5430 Ops/s	$\color{#35bf28}+1.22\%$
test_sac_speed[True-backward]	4.5671ms	4.2212ms	236.9020 Ops/s	235.8551 Ops/s	$\color{#35bf28}+0.44\%$
test_sac_speed[reduce-overhead-None]	3.0637ms	2.5466ms	392.6840 Ops/s	386.6109 Ops/s	$\color{#35bf28}+1.57\%$
test_sac_speed[reduce-overhead-backward]	4.5381ms	4.2110ms	237.4723 Ops/s	232.0220 Ops/s	$\color{#35bf28}+2.35\%$
test_redq_speed[False-None]	13.9400ms	13.0452ms	76.6565 Ops/s	76.0231 Ops/s	$\color{#35bf28}+0.83\%$
test_redq_speed[False-backward]	27.4018ms	22.8656ms	43.7339 Ops/s	44.4376 Ops/s	$\color{#d91a1a}-1.58\%$
test_redq_speed[True-None]	7.4181ms	6.7012ms	149.2270 Ops/s	149.8680 Ops/s	$\color{#d91a1a}-0.43\%$
test_redq_speed[True-backward]	15.5107ms	14.2341ms	70.2540 Ops/s	70.8842 Ops/s	$\color{#d91a1a}-0.89\%$
test_redq_speed[reduce-overhead-None]	7.5842ms	6.7950ms	147.1679 Ops/s	146.9770 Ops/s	$\color{#35bf28}+0.13\%$
test_redq_speed[reduce-overhead-backward]	14.5003ms	14.0892ms	70.9762 Ops/s	70.6176 Ops/s	$\color{#35bf28}+0.51\%$
test_redq_deprec_speed[False-None]	13.6754ms	12.8364ms	77.9033 Ops/s	77.4865 Ops/s	$\color{#35bf28}+0.54\%$
test_redq_deprec_speed[False-backward]	20.3184ms	18.6193ms	53.7077 Ops/s	53.9975 Ops/s	$\color{#d91a1a}-0.54\%$
test_redq_deprec_speed[True-None]	5.9468ms	5.2099ms	191.9419 Ops/s	193.1371 Ops/s	$\color{#d91a1a}-0.62\%$
test_redq_deprec_speed[True-backward]	10.7083ms	10.0156ms	99.8439 Ops/s	100.7879 Ops/s	$\color{#d91a1a}-0.94\%$
test_redq_deprec_speed[reduce-overhead-None]	5.8414ms	5.1938ms	192.5368 Ops/s	193.8237 Ops/s	$\color{#d91a1a}-0.66\%$
test_redq_deprec_speed[reduce-overhead-backward]	10.4059ms	9.9783ms	100.2172 Ops/s	100.6713 Ops/s	$\color{#d91a1a}-0.45\%$
test_td3_speed[False-None]	9.0781ms	8.1243ms	123.0875 Ops/s	123.8672 Ops/s	$\color{#d91a1a}-0.63\%$
test_td3_speed[False-backward]	11.6845ms	10.5834ms	94.4875 Ops/s	95.8735 Ops/s	$\color{#d91a1a}-1.45\%$
test_td3_speed[True-None]	2.5709ms	2.2733ms	439.8896 Ops/s	442.2713 Ops/s	$\color{#d91a1a}-0.54\%$
test_td3_speed[True-backward]	4.0402ms	3.9203ms	255.0800 Ops/s	252.3586 Ops/s	$\color{#35bf28}+1.08\%$
test_td3_speed[reduce-overhead-None]	2.5007ms	2.2749ms	439.5826 Ops/s	440.6871 Ops/s	$\color{#d91a1a}-0.25\%$
test_td3_speed[reduce-overhead-backward]	4.2206ms	3.9348ms	254.1412 Ops/s	230.2695 Ops/s	$\textbf{\color{#35bf28}+10.37\%}$
test_cql_speed[False-None]	39.8843ms	36.7216ms	27.2319 Ops/s	27.3236 Ops/s	$\color{#d91a1a}-0.34\%$
test_cql_speed[False-backward]	51.3540ms	46.9428ms	21.3025 Ops/s	21.3508 Ops/s	$\color{#d91a1a}-0.23\%$
test_cql_speed[True-None]	22.8036ms	22.2632ms	44.9172 Ops/s	44.7357 Ops/s	$\color{#35bf28}+0.41\%$
test_cql_speed[True-backward]	30.9521ms	29.3211ms	34.1051 Ops/s	33.9924 Ops/s	$\color{#35bf28}+0.33\%$
test_cql_speed[reduce-overhead-None]	23.3370ms	22.1911ms	45.0631 Ops/s	45.1090 Ops/s	$\color{#d91a1a}-0.10\%$
test_cql_speed[reduce-overhead-backward]	30.2420ms	28.9841ms	34.5017 Ops/s	34.3345 Ops/s	$\color{#35bf28}+0.49\%$
test_a2c_speed[False-None]	8.2014ms	7.1882ms	139.1165 Ops/s	138.3554 Ops/s	$\color{#35bf28}+0.55\%$
test_a2c_speed[False-backward]	16.8172ms	14.3370ms	69.7495 Ops/s	70.9048 Ops/s	$\color{#d91a1a}-1.63\%$
test_a2c_speed[True-None]	5.4293ms	4.6535ms	214.8925 Ops/s	212.9639 Ops/s	$\color{#35bf28}+0.91\%$
test_a2c_speed[True-backward]	11.5867ms	11.0378ms	90.5976 Ops/s	90.1270 Ops/s	$\color{#35bf28}+0.52\%$
test_a2c_speed[reduce-overhead-None]	5.0085ms	4.6234ms	216.2889 Ops/s	205.3507 Ops/s	$\textbf{\color{#35bf28}+5.33\%}$
test_a2c_speed[reduce-overhead-backward]	13.9096ms	11.5202ms	86.8039 Ops/s	90.3117 Ops/s	$\color{#d91a1a}-3.88\%$
test_ppo_speed[False-None]	8.1196ms	7.5247ms	132.8952 Ops/s	132.9467 Ops/s	$\color{#d91a1a}-0.04\%$
test_ppo_speed[False-backward]	16.1673ms	14.9343ms	66.9600 Ops/s	66.0653 Ops/s	$\color{#35bf28}+1.35\%$
test_ppo_speed[True-None]	5.9166ms	5.0504ms	198.0029 Ops/s	197.4223 Ops/s	$\color{#35bf28}+0.29\%$
test_ppo_speed[True-backward]	11.6947ms	10.9268ms	91.5178 Ops/s	90.8353 Ops/s	$\color{#35bf28}+0.75\%$
test_ppo_speed[reduce-overhead-None]	5.6833ms	5.0431ms	198.2918 Ops/s	196.3530 Ops/s	$\color{#35bf28}+0.99\%$
test_ppo_speed[reduce-overhead-backward]	11.4086ms	10.9816ms	91.0613 Ops/s	89.7056 Ops/s	$\color{#35bf28}+1.51\%$
test_reinforce_speed[False-None]	7.9417ms	6.5838ms	151.8870 Ops/s	150.5846 Ops/s	$\color{#35bf28}+0.86\%$
test_reinforce_speed[False-backward]	10.6052ms	9.8628ms	101.3907 Ops/s	101.1421 Ops/s	$\color{#35bf28}+0.25\%$
test_reinforce_speed[True-None]	4.9658ms	4.0259ms	248.3921 Ops/s	243.6387 Ops/s	$\color{#35bf28}+1.95\%$
test_reinforce_speed[True-backward]	10.3984ms	9.9032ms	100.9775 Ops/s	99.7613 Ops/s	$\color{#35bf28}+1.22\%$
test_reinforce_speed[reduce-overhead-None]	4.4183ms	4.0151ms	249.0567 Ops/s	244.5094 Ops/s	$\color{#35bf28}+1.86\%$
test_reinforce_speed[reduce-overhead-backward]	10.3486ms	9.8948ms	101.0636 Ops/s	99.8206 Ops/s	$\color{#35bf28}+1.25\%$
test_iql_speed[False-None]	42.0389ms	34.3558ms	29.1072 Ops/s	29.4837 Ops/s	$\color{#d91a1a}-1.28\%$
test_iql_speed[False-backward]	47.2797ms	45.6653ms	21.8985 Ops/s	21.7978 Ops/s	$\color{#35bf28}+0.46\%$
test_iql_speed[True-None]	16.3976ms	15.5950ms	64.1231 Ops/s	62.7618 Ops/s	$\color{#35bf28}+2.17\%$
test_iql_speed[True-backward]	28.6780ms	27.4714ms	36.4015 Ops/s	37.0942 Ops/s	$\color{#d91a1a}-1.87\%$
test_iql_speed[reduce-overhead-None]	17.4382ms	15.7695ms	63.4136 Ops/s	63.4938 Ops/s	$\color{#d91a1a}-0.13\%$
test_iql_speed[reduce-overhead-backward]	28.4538ms	26.9214ms	37.1452 Ops/s	35.1584 Ops/s	$\textbf{\color{#35bf28}+5.65\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.3283ms	4.8215ms	207.4062 Ops/s	202.4883 Ops/s	$\color{#35bf28}+2.43\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7660ms	0.5123ms	1.9521 KOps/s	1.8971 KOps/s	$\color{#35bf28}+2.90\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.9013ms	0.4912ms	2.0359 KOps/s	2.0007 KOps/s	$\color{#35bf28}+1.76\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	7.2047ms	4.6345ms	215.7719 Ops/s	215.7618 Ops/s	$+0.00\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	2.0980ms	0.5052ms	1.9795 KOps/s	1.9409 KOps/s	$\color{#35bf28}+1.99\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7787ms	0.4787ms	2.0890 KOps/s	2.0261 KOps/s	$\color{#35bf28}+3.11\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.4348ms	1.6626ms	601.4668 Ops/s	605.9687 Ops/s	$\color{#d91a1a}-0.74\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.3785ms	1.5798ms	633.0096 Ops/s	631.7644 Ops/s	$\color{#35bf28}+0.20\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	7.3392ms	4.7211ms	211.8129 Ops/s	207.7444 Ops/s	$\color{#35bf28}+1.96\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	2.4140ms	0.6528ms	1.5319 KOps/s	1.4939 KOps/s	$\color{#35bf28}+2.54\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	1.0338ms	0.6325ms	1.5811 KOps/s	1.5742 KOps/s	$\color{#35bf28}+0.43\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.0669ms	4.5533ms	219.6200 Ops/s	214.3632 Ops/s	$\color{#35bf28}+2.45\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.8975ms	0.5161ms	1.9375 KOps/s	1.9276 KOps/s	$\color{#35bf28}+0.51\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.9149ms	0.5008ms	1.9968 KOps/s	2.0022 KOps/s	$\color{#d91a1a}-0.27\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	4.9145ms	4.5710ms	218.7690 Ops/s	217.1765 Ops/s	$\color{#35bf28}+0.73\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.1097ms	0.5095ms	1.9628 KOps/s	1.9833 KOps/s	$\color{#d91a1a}-1.04\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7926ms	0.4789ms	2.0881 KOps/s	1.9956 KOps/s	$\color{#35bf28}+4.64\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	4.8974ms	4.6637ms	214.4228 Ops/s	210.1951 Ops/s	$\color{#35bf28}+2.01\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.2276ms	0.6488ms	1.5414 KOps/s	1.5377 KOps/s	$\color{#35bf28}+0.24\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	1.0019ms	0.6316ms	1.5834 KOps/s	1.5568 KOps/s	$\color{#35bf28}+1.71\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.7455s	18.9977ms	52.6380 Ops/s	244.0627 Ops/s	$\textbf{\color{#d91a1a}-78.43\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	8.4995ms	2.3741ms	421.2070 Ops/s	425.9880 Ops/s	$\color{#d91a1a}-1.12\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	4.8307ms	1.3627ms	733.8473 Ops/s	744.9534 Ops/s	$\color{#d91a1a}-1.49\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	5.5395ms	4.2710ms	234.1391 Ops/s	23.7608 Ops/s	$\textbf{\color{#35bf28}+885.40\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	8.4299ms	2.3561ms	424.4295 Ops/s	416.6679 Ops/s	$\color{#35bf28}+1.86\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	4.6463ms	1.3582ms	736.2927 Ops/s	795.2952 Ops/s	$\textbf{\color{#d91a1a}-7.42\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	6.0034ms	4.3995ms	227.2982 Ops/s	220.6034 Ops/s	$\color{#35bf28}+3.03\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	6.0301ms	2.4509ms	408.0069 Ops/s	410.1764 Ops/s	$\color{#d91a1a}-0.53\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.4451ms	1.5522ms	644.2471 Ops/s	662.6983 Ops/s	$\color{#d91a1a}-2.78\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	56.5022ms	49.2697ms	20.2965 Ops/s	19.0439 Ops/s	$\textbf{\color{#35bf28}+6.58\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	15.6074ms	14.1058ms	70.8926 Ops/s	70.2478 Ops/s	$\color{#35bf28}+0.92\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	57.4623ms	49.3911ms	20.2466 Ops/s	19.4667 Ops/s	$\color{#35bf28}+4.01\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	15.3953ms	14.3646ms	69.6155 Ops/s	69.3810 Ops/s	$\color{#35bf28}+0.34\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	58.7235ms	50.1277ms	19.9491 Ops/s	19.7935 Ops/s	$\color{#35bf28}+0.79\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	17.4614ms	15.7813ms	63.3663 Ops/s	63.3841 Ops/s	$\color{#d91a1a}-0.03\%$

[vmoens]

LGTM thanks!