Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AttributeError: 'SFSolver' object has no attribute 'add_grad_from_state' #348

Closed
mk2001233 opened this issue Dec 27, 2024 · 1 comment · May be fixed by #355
Closed

AttributeError: 'SFSolver' object has no attribute 'add_grad_from_state' #348

mk2001233 opened this issue Dec 27, 2024 · 1 comment · May be fixed by #355

Comments

@mk2001233
Copy link

python examples/differentiable_push.py -v

[Genesis] [03:47:05] [INFO] ╭─────────────────────────────────────────────────────────────────────────────────────╮
[Genesis] [03:47:05] [INFO] │┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉ Genesis ┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉│
[Genesis] [03:47:05] [INFO] ╰─────────────────────────────────────────────────────────────────────────────────────╯
[Genesis] [03:47:05] [INFO] Running on [NVIDIA GeForce RTX 3090 Ti] with backend gs.cuda. Device memory: 23.58 GB.
[Genesis] [03:47:06] [DEBUG] [Taichi] version 1.7.2, llvm 15.0.4, commit 0131dce9, linux, python 3.11.10
[Genesis] [03:47:06] [DEBUG] [Taichi] Starting on arch=cuda
[Genesis] [03:47:06] [INFO] 🚀 Genesis initialized. 🔖 version: 0.2.0, 🌱 seed: 0, 📏 precision: '32', 🐛 debug: False, 🎨 theme: 'dark'.
[Genesis] [03:47:06] [INFO] Scene <94a9386> created.
[Genesis] [03:47:06] [INFO] Adding <gs.RigidEntity>. idx: 0, uid: , morph: <gs.morphs.URDF(file='/opt/conda/lib/python3.11/site-packages/genesis/assets/urdf/plane/plane.urdf')>, material: <gs.materials.Rigid>.
[Genesis] [03:47:06] [DEBUG] Preprocessed .gsd file found in cache for geom idx 0.
[Genesis] [03:47:06] [INFO] Adding <gs.ToolEntity>. idx: 1, uid: <688a0a0>, morph: <gs.morphs.Mesh(file='/opt/conda/lib/python3.11/site-packages/genesis/assets/meshes/stirrer.obj')>, material: <gs.materials.Tool>.
[Genesis] [03:47:06] [DEBUG] Processing mesh: /opt/conda/lib/python3.11/site-packages/genesis/assets/meshes/stirrer.obj.
[Genesis] [03:47:07] [INFO] Adding <gs.MPMEntity>. idx: 2, uid: <066489c>, morph: <gs.morphs.Box>, material: <gs.materials.MPM.Elastic>.
[Genesis] [03:47:07] [DEBUG] Sampled particles (.ptc) found in cache.
[Genesis] [03:47:07] [INFO] Sampled 4,127 particles.
[Genesis] [03:47:07] [INFO] Adding <gs.MPMEntity>. idx: 3, uid: , morph: <gs.morphs.Mesh(file='/opt/conda/lib/python3.11/site-packages/genesis/assets/meshes/duck.obj')>, material: <gs.materials.MPM.Elastic>.
[Genesis] [03:47:07] [DEBUG] Sampled particles (.ptc) found in cache.
[Genesis] [03:47:07] [INFO] Sampled 2,564 particles.
[Genesis] [03:47:07] [INFO] Building scene <94a9386>...
[Genesis] [03:47:10] [INFO] Compiling simulation kernels...
[Genesis] [03:47:15] [DEBUG] Forward: Saved checkpoint for global substep 0 to 9. Now starts from substep 10.
[Genesis] [03:47:16] [INFO] Building visualizer...
[Genesis] [03:47:16] [INFO] Viewer created. Resolution: 720×540, max_FPS: 60.
[Genesis] [03:47:16] [INFO] Resetting Scene <94a9386>.
forward
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[Genesis] [03:47:16] [WARNING] Manually setting particle velocities. This is not recommended and could break gradient flow.
[Genesis] [03:47:16] [WARNING] Manually setting particle positions. This is not recommended and could break gradient flow. This also resets particle stress and velocities.
/opt/conda/lib/python3.11/site-packages/torch/_tensor.py:1512: UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the .grad field to be populated for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations. (Triggered internally at aten/src/ATen/core/TensorBody.h:489.)
ret = func(*args, **kwargs)
[Genesis] [03:47:16] [DEBUG] Forward: Saved checkpoint for global substep 0 to 9. Now starts from substep 10.
[Genesis] [03:47:16] [INFO] Running at 172.84 FPS.
[Genesis] [03:47:16] [DEBUG] Forward: Saved checkpoint for global substep 10 to 19. Now starts from substep 20.
[Genesis] [03:47:17] [INFO] Running at 157.97 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 20 to 29. Now starts from substep 30.
[Genesis] [03:47:17] [INFO] Running at 146.15 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 30 to 39. Now starts from substep 40.
[Genesis] [03:47:17] [INFO] Running at 136.40 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 40 to 49. Now starts from substep 50.
[Genesis] [03:47:17] [INFO] Running at 128.25 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 50 to 59. Now starts from substep 60.
[Genesis] [03:47:17] [INFO] Running at 121.43 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 60 to 69. Now starts from substep 70.
[Genesis] [03:47:17] [INFO] Running at 115.60 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 70 to 79. Now starts from substep 80.
[Genesis] [03:47:17] [INFO] Running at 110.52 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 80 to 89. Now starts from substep 90.
[Genesis] [03:47:17] [INFO] Running at 105.99 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 90 to 99. Now starts from substep 100.
[Genesis] [03:47:17] [INFO] Running at 102.11 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 100 to 109. Now starts from substep 110.
[Genesis] [03:47:17] [INFO] Running at 98.69 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 110 to 119. Now starts from substep 120.
[Genesis] [03:47:17] [INFO] Running at 95.62 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 120 to 129. Now starts from substep 130.
[Genesis] [03:47:17] [INFO] Running at 92.91 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 130 to 139. Now starts from substep 140.
[Genesis] [03:47:17] [INFO] Running at 90.44 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 140 to 149. Now starts from substep 150.
[Genesis] [03:47:17] [INFO] Running at 88.25 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 150 to 159. Now starts from substep 160.
[Genesis] [03:47:17] [INFO] Running at 86.24 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 160 to 169. Now starts from substep 170.
[Genesis] [03:47:17] [INFO] Running at 84.42 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 170 to 179. Now starts from substep 180.
[Genesis] [03:47:17] [INFO] Running at 82.75 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 180 to 189. Now starts from substep 190.
[Genesis] [03:47:17] [INFO] Running at 81.24 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 190 to 199. Now starts from substep 200.
[Genesis] [03:47:17] [INFO] Running at 79.84 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 200 to 209. Now starts from substep 210.
[Genesis] [03:47:17] [INFO] Running at 78.60 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 210 to 219. Now starts from substep 220.
[Genesis] [03:47:17] [INFO] Running at 77.41 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 220 to 229. Now starts from substep 230.
[Genesis] [03:47:17] [INFO] Running at 76.30 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 230 to 239. Now starts from substep 240.
[Genesis] [03:47:17] [INFO] Running at 75.31 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 240 to 249. Now starts from substep 250.
[Genesis] [03:47:17] [INFO] Running at 74.38 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 250 to 259. Now starts from substep 260.
[Genesis] [03:47:17] [INFO] Running at 73.50 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 260 to 269. Now starts from substep 270.
[Genesis] [03:47:17] [INFO] Running at 63.71 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 270 to 279. Now starts from substep 280.
[Genesis] [03:47:17] [INFO] Running at 63.52 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 280 to 289. Now starts from substep 290.
[Genesis] [03:47:17] [INFO] Running at 63.34 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 290 to 299. Now starts from substep 300.
[Genesis] [03:47:17] [INFO] Running at 63.18 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 300 to 309. Now starts from substep 310.
[Genesis] [03:47:17] [INFO] Running at 63.03 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 310 to 319. Now starts from substep 320.
[Genesis] [03:47:17] [INFO] Running at 62.88 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 320 to 329. Now starts from substep 330.
[Genesis] [03:47:17] [INFO] Running at 62.73 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 330 to 339. Now starts from substep 340.
[Genesis] [03:47:17] [INFO] Running at 62.61 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 340 to 349. Now starts from substep 350.
[Genesis] [03:47:17] [INFO] Running at 62.49 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 350 to 359. Now starts from substep 360.
[Genesis] [03:47:17] [INFO] Running at 62.36 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 360 to 369. Now starts from substep 370.
[Genesis] [03:47:17] [INFO] Running at 62.25 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 370 to 379. Now starts from substep 380.
[Genesis] [03:47:17] [INFO] Running at 62.16 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 380 to 389. Now starts from substep 390.
[Genesis] [03:47:17] [INFO] Running at 62.07 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 390 to 399. Now starts from substep 400.
[Genesis] [03:47:17] [INFO] Running at 61.97 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 400 to 409. Now starts from substep 410.
[Genesis] [03:47:17] [INFO] Running at 61.88 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 410 to 419. Now starts from substep 420.
[Genesis] [03:47:17] [INFO] Running at 61.79 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 420 to 429. Now starts from substep 430.
[Genesis] [03:47:17] [INFO] Running at 61.71 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 430 to 439. Now starts from substep 440.
[Genesis] [03:47:17] [INFO] Running at 61.63 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 440 to 449. Now starts from substep 450.
[Genesis] [03:47:17] [INFO] Running at 61.54 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 450 to 459. Now starts from substep 460.
[Genesis] [03:47:17] [INFO] Running at 61.53 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 460 to 469. Now starts from substep 470.
[Genesis] [03:47:17] [INFO] Running at 61.46 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 470 to 479. Now starts from substep 480.
[Genesis] [03:47:17] [INFO] Running at 61.39 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 480 to 489. Now starts from substep 490.
[Genesis] [03:47:17] [INFO] Running at 61.34 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 490 to 499. Now starts from substep 500.
[Genesis] [03:47:17] [INFO] Running at 61.28 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 500 to 509. Now starts from substep 510.
[Genesis] [03:47:17] [INFO] Running at 61.23 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 510 to 519. Now starts from substep 520.
[Genesis] [03:47:17] [INFO] Running at 61.17 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 520 to 529. Now starts from substep 530.
[Genesis] [03:47:17] [INFO] Running at 61.12 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 530 to 539. Now starts from substep 540.
[Genesis] [03:47:17] [INFO] Running at 61.07 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 540 to 549. Now starts from substep 550.
[Genesis] [03:47:17] [INFO] Running at 61.02 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 550 to 559. Now starts from substep 560.
[Genesis] [03:47:17] [INFO] Running at 60.98 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 560 to 569. Now starts from substep 570.
[Genesis] [03:47:17] [INFO] Running at 60.92 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 570 to 579. Now starts from substep 580.
[Genesis] [03:47:17] [INFO] Running at 60.89 FPS.
[Genesis] [03:47:17] [DEBUG] Forward: Saved checkpoint for global substep 580 to 589. Now starts from substep 590.
[Genesis] [03:47:18] [INFO] Running at 60.85 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 590 to 599. Now starts from substep 600.
[Genesis] [03:47:18] [INFO] Running at 60.81 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 600 to 609. Now starts from substep 610.
[Genesis] [03:47:18] [INFO] Running at 60.78 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 610 to 619. Now starts from substep 620.
[Genesis] [03:47:18] [INFO] Running at 60.75 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 620 to 629. Now starts from substep 630.
[Genesis] [03:47:18] [INFO] Running at 60.72 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 630 to 639. Now starts from substep 640.
[Genesis] [03:47:18] [INFO] Running at 60.69 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 640 to 649. Now starts from substep 650.
[Genesis] [03:47:18] [INFO] Running at 60.66 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 650 to 659. Now starts from substep 660.
[Genesis] [03:47:18] [INFO] Running at 60.63 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 660 to 669. Now starts from substep 670.
[Genesis] [03:47:18] [INFO] Running at 60.61 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 670 to 679. Now starts from substep 680.
[Genesis] [03:47:18] [INFO] Running at 60.59 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 680 to 689. Now starts from substep 690.
[Genesis] [03:47:18] [INFO] Running at 60.58 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 690 to 699. Now starts from substep 700.
[Genesis] [03:47:18] [INFO] Running at 60.56 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 700 to 709. Now starts from substep 710.
[Genesis] [03:47:18] [INFO] Running at 60.54 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 710 to 719. Now starts from substep 720.
[Genesis] [03:47:18] [INFO] Running at 60.52 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 720 to 729. Now starts from substep 730.
[Genesis] [03:47:18] [INFO] Running at 60.50 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 730 to 739. Now starts from substep 740.
[Genesis] [03:47:18] [INFO] Running at 60.48 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 740 to 749. Now starts from substep 750.
[Genesis] [03:47:18] [INFO] Running at 60.47 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 750 to 759. Now starts from substep 760.
[Genesis] [03:47:18] [INFO] Running at 60.45 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 760 to 769. Now starts from substep 770.
[Genesis] [03:47:18] [INFO] Running at 60.36 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 770 to 779. Now starts from substep 780.
[Genesis] [03:47:18] [INFO] Running at 60.35 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 780 to 789. Now starts from substep 790.
[Genesis] [03:47:18] [INFO] Running at 60.35 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 790 to 799. Now starts from substep 800.
[Genesis] [03:47:18] [INFO] Running at 60.34 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 800 to 809. Now starts from substep 810.
[Genesis] [03:47:18] [INFO] Running at 60.36 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 810 to 819. Now starts from substep 820.
[Genesis] [03:47:18] [INFO] Running at 60.35 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 820 to 829. Now starts from substep 830.
[Genesis] [03:47:18] [INFO] Running at 60.35 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 830 to 839. Now starts from substep 840.
[Genesis] [03:47:18] [INFO] Running at 60.32 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 840 to 849. Now starts from substep 850.
[Genesis] [03:47:18] [INFO] Running at 60.32 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 850 to 859. Now starts from substep 860.
[Genesis] [03:47:18] [INFO] Running at 60.31 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 860 to 869. Now starts from substep 870.
[Genesis] [03:47:18] [INFO] Running at 60.30 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 870 to 879. Now starts from substep 880.
[Genesis] [03:47:18] [INFO] Running at 60.29 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 880 to 889. Now starts from substep 890.
[Genesis] [03:47:18] [INFO] Running at 60.27 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 890 to 899. Now starts from substep 900.
[Genesis] [03:47:18] [INFO] Running at 60.23 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 900 to 909. Now starts from substep 910.
[Genesis] [03:47:18] [INFO] Running at 60.25 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 910 to 919. Now starts from substep 920.
[Genesis] [03:47:18] [INFO] Running at 60.25 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 920 to 929. Now starts from substep 930.
[Genesis] [03:47:18] [INFO] Running at 60.24 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 930 to 939. Now starts from substep 940.
[Genesis] [03:47:18] [INFO] Running at 60.24 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 940 to 949. Now starts from substep 950.
[Genesis] [03:47:18] [INFO] Running at 60.24 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 950 to 959. Now starts from substep 960.
[Genesis] [03:47:18] [INFO] Running at 60.22 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 960 to 969. Now starts from substep 970.
[Genesis] [03:47:18] [INFO] Running at 60.22 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 970 to 979. Now starts from substep 980.
[Genesis] [03:47:18] [INFO] Running at 60.22 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 980 to 989. Now starts from substep 990.
[Genesis] [03:47:18] [INFO] Running at 60.19 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 990 to 999. Now starts from substep 1000.
[Genesis] [03:47:18] [INFO] Running at 60.19 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1000 to 1009. Now starts from substep 1010.
[Genesis] [03:47:18] [INFO] Running at 60.19 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1010 to 1019. Now starts from substep 1020.
[Genesis] [03:47:18] [INFO] Running at 60.18 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1020 to 1029. Now starts from substep 1030.
[Genesis] [03:47:18] [INFO] Running at 60.19 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1030 to 1039. Now starts from substep 1040.
[Genesis] [03:47:18] [INFO] Running at 60.19 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1040 to 1049. Now starts from substep 1050.
[Genesis] [03:47:18] [INFO] Running at 60.18 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1050 to 1059. Now starts from substep 1060.
[Genesis] [03:47:18] [INFO] Running at 60.20 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1060 to 1069. Now starts from substep 1070.
[Genesis] [03:47:18] [INFO] Running at 60.20 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1070 to 1079. Now starts from substep 1080.
[Genesis] [03:47:18] [INFO] Running at 60.19 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1080 to 1089. Now starts from substep 1090.
[Genesis] [03:47:18] [INFO] Running at 60.18 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1090 to 1099. Now starts from substep 1100.
[Genesis] [03:47:18] [INFO] Running at 60.18 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1100 to 1109. Now starts from substep 1110.
[Genesis] [03:47:18] [INFO] Running at 60.17 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1110 to 1119. Now starts from substep 1120.
[Genesis] [03:47:18] [INFO] Running at 60.14 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1120 to 1129. Now starts from substep 1130.
[Genesis] [03:47:18] [INFO] Running at 60.14 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1130 to 1139. Now starts from substep 1140.
[Genesis] [03:47:18] [INFO] Running at 60.14 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1140 to 1149. Now starts from substep 1150.
[Genesis] [03:47:18] [INFO] Running at 60.14 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1150 to 1159. Now starts from substep 1160.
[Genesis] [03:47:18] [INFO] Running at 60.14 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1160 to 1169. Now starts from substep 1170.
[Genesis] [03:47:18] [INFO] Running at 60.14 FPS.
[Genesis] [03:47:18] [DEBUG] Forward: Saved checkpoint for global substep 1170 to 1179. Now starts from substep 1180.
[Genesis] [03:47:18] [INFO] Running at 60.14 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1180 to 1189. Now starts from substep 1190.
[Genesis] [03:47:19] [INFO] Running at 60.15 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1190 to 1199. Now starts from substep 1200.
[Genesis] [03:47:19] [INFO] Running at 60.15 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1200 to 1209. Now starts from substep 1210.
[Genesis] [03:47:19] [INFO] Running at 60.15 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1210 to 1219. Now starts from substep 1220.
[Genesis] [03:47:19] [INFO] Running at 59.97 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1220 to 1229. Now starts from substep 1230.
[Genesis] [03:47:19] [INFO] Running at 59.98 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1230 to 1239. Now starts from substep 1240.
[Genesis] [03:47:19] [INFO] Running at 59.78 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1240 to 1249. Now starts from substep 1250.
[Genesis] [03:47:19] [INFO] Running at 59.79 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1250 to 1259. Now starts from substep 1260.
[Genesis] [03:47:19] [INFO] Running at 59.81 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1260 to 1269. Now starts from substep 1270.
[Genesis] [03:47:19] [INFO] Running at 59.83 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1270 to 1279. Now starts from substep 1280.
[Genesis] [03:47:19] [INFO] Running at 59.83 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1280 to 1289. Now starts from substep 1290.
[Genesis] [03:47:19] [INFO] Running at 59.85 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1290 to 1299. Now starts from substep 1300.
[Genesis] [03:47:19] [INFO] Running at 59.86 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1300 to 1309. Now starts from substep 1310.
[Genesis] [03:47:19] [INFO] Running at 59.87 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1310 to 1319. Now starts from substep 1320.
[Genesis] [03:47:19] [INFO] Running at 59.91 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1320 to 1329. Now starts from substep 1330.
[Genesis] [03:47:19] [INFO] Running at 59.91 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1330 to 1339. Now starts from substep 1340.
[Genesis] [03:47:19] [INFO] Running at 59.89 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1340 to 1349. Now starts from substep 1350.
[Genesis] [03:47:19] [INFO] Running at 59.91 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1350 to 1359. Now starts from substep 1360.
[Genesis] [03:47:19] [INFO] Running at 59.92 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1360 to 1369. Now starts from substep 1370.
[Genesis] [03:47:19] [INFO] Running at 59.94 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1370 to 1379. Now starts from substep 1380.
[Genesis] [03:47:19] [INFO] Running at 59.95 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1380 to 1389. Now starts from substep 1390.
[Genesis] [03:47:19] [INFO] Running at 59.97 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1390 to 1399. Now starts from substep 1400.
[Genesis] [03:47:19] [INFO] Running at 59.98 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1400 to 1409. Now starts from substep 1410.
[Genesis] [03:47:19] [INFO] Running at 59.99 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1410 to 1419. Now starts from substep 1420.
[Genesis] [03:47:19] [INFO] Running at 60.00 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1420 to 1429. Now starts from substep 1430.
[Genesis] [03:47:19] [INFO] Running at 60.02 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1430 to 1439. Now starts from substep 1440.
[Genesis] [03:47:19] [INFO] Running at 60.02 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1440 to 1449. Now starts from substep 1450.
[Genesis] [03:47:19] [INFO] Running at 60.03 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1450 to 1459. Now starts from substep 1460.
[Genesis] [03:47:19] [INFO] Running at 60.03 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1460 to 1469. Now starts from substep 1470.
[Genesis] [03:47:19] [INFO] Running at 60.03 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1470 to 1479. Now starts from substep 1480.
[Genesis] [03:47:19] [INFO] Running at 60.04 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1480 to 1489. Now starts from substep 1490.
[Genesis] [03:47:19] [INFO] Running at 60.07 FPS.
[Genesis] [03:47:19] [DEBUG] Forward: Saved checkpoint for global substep 1490 to 1499. Now starts from substep 1500.
[Genesis] [03:47:19] [INFO] Running at 60.07 FPS.
backward
[Genesis] [03:47:20] [DEBUG] Backward: Loaded checkpoint for global substep 1490 to 1499. Now starts from substep 1490.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1480 to 1489. Now starts from substep 1480.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1470 to 1479. Now starts from substep 1470.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1460 to 1469. Now starts from substep 1460.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1450 to 1459. Now starts from substep 1450.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1440 to 1449. Now starts from substep 1440.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1430 to 1439. Now starts from substep 1430.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1420 to 1429. Now starts from substep 1420.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1410 to 1419. Now starts from substep 1410.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1400 to 1409. Now starts from substep 1400.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1390 to 1399. Now starts from substep 1390.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1380 to 1389. Now starts from substep 1380.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1370 to 1379. Now starts from substep 1370.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1360 to 1369. Now starts from substep 1360.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1350 to 1359. Now starts from substep 1350.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1340 to 1349. Now starts from substep 1340.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1330 to 1339. Now starts from substep 1330.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1320 to 1329. Now starts from substep 1320.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1310 to 1319. Now starts from substep 1310.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1300 to 1309. Now starts from substep 1300.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1290 to 1299. Now starts from substep 1290.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1280 to 1289. Now starts from substep 1280.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1270 to 1279. Now starts from substep 1270.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1260 to 1269. Now starts from substep 1260.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1250 to 1259. Now starts from substep 1250.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1240 to 1249. Now starts from substep 1240.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1230 to 1239. Now starts from substep 1230.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1220 to 1229. Now starts from substep 1220.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1210 to 1219. Now starts from substep 1210.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1200 to 1209. Now starts from substep 1200.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1190 to 1199. Now starts from substep 1190.
[Genesis] [03:47:23] [DEBUG] Backward: Loaded checkpoint for global substep 1180 to 1189. Now starts from substep 1180.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1170 to 1179. Now starts from substep 1170.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1160 to 1169. Now starts from substep 1160.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1150 to 1159. Now starts from substep 1150.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1140 to 1149. Now starts from substep 1140.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1130 to 1139. Now starts from substep 1130.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1120 to 1129. Now starts from substep 1120.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1110 to 1119. Now starts from substep 1110.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1100 to 1109. Now starts from substep 1100.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1090 to 1099. Now starts from substep 1090.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1080 to 1089. Now starts from substep 1080.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1070 to 1079. Now starts from substep 1070.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1060 to 1069. Now starts from substep 1060.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1050 to 1059. Now starts from substep 1050.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1040 to 1049. Now starts from substep 1040.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1030 to 1039. Now starts from substep 1030.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1020 to 1029. Now starts from substep 1020.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1010 to 1019. Now starts from substep 1010.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 1000 to 1009. Now starts from substep 1000.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 990 to 999. Now starts from substep 990.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 980 to 989. Now starts from substep 980.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 970 to 979. Now starts from substep 970.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 960 to 969. Now starts from substep 960.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 950 to 959. Now starts from substep 950.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 940 to 949. Now starts from substep 940.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 930 to 939. Now starts from substep 930.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 920 to 929. Now starts from substep 920.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 910 to 919. Now starts from substep 910.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 900 to 909. Now starts from substep 900.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 890 to 899. Now starts from substep 890.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 880 to 889. Now starts from substep 880.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 870 to 879. Now starts from substep 870.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 860 to 869. Now starts from substep 860.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 850 to 859. Now starts from substep 850.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 840 to 849. Now starts from substep 840.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 830 to 839. Now starts from substep 830.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 820 to 829. Now starts from substep 820.
[Genesis] [03:47:24] [DEBUG] Backward: Loaded checkpoint for global substep 810 to 819. Now starts from substep 810.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 800 to 809. Now starts from substep 800.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 790 to 799. Now starts from substep 790.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 780 to 789. Now starts from substep 780.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 770 to 779. Now starts from substep 770.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 760 to 769. Now starts from substep 760.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 750 to 759. Now starts from substep 750.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 740 to 749. Now starts from substep 740.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 730 to 739. Now starts from substep 730.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 720 to 729. Now starts from substep 720.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 710 to 719. Now starts from substep 710.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 700 to 709. Now starts from substep 700.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 690 to 699. Now starts from substep 690.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 680 to 689. Now starts from substep 680.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 670 to 679. Now starts from substep 670.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 660 to 669. Now starts from substep 660.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 650 to 659. Now starts from substep 650.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 640 to 649. Now starts from substep 640.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 630 to 639. Now starts from substep 630.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 620 to 629. Now starts from substep 620.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 610 to 619. Now starts from substep 610.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 600 to 609. Now starts from substep 600.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 590 to 599. Now starts from substep 590.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 580 to 589. Now starts from substep 580.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 570 to 579. Now starts from substep 570.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 560 to 569. Now starts from substep 560.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 550 to 559. Now starts from substep 550.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 540 to 549. Now starts from substep 540.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 530 to 539. Now starts from substep 530.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 520 to 529. Now starts from substep 520.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 510 to 519. Now starts from substep 510.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 500 to 509. Now starts from substep 500.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 490 to 499. Now starts from substep 490.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 480 to 489. Now starts from substep 480.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 470 to 479. Now starts from substep 470.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 460 to 469. Now starts from substep 460.
[Genesis] [03:47:25] [DEBUG] Backward: Loaded checkpoint for global substep 450 to 459. Now starts from substep 450.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 440 to 449. Now starts from substep 440.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 430 to 439. Now starts from substep 430.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 420 to 429. Now starts from substep 420.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 410 to 419. Now starts from substep 410.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 400 to 409. Now starts from substep 400.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 390 to 399. Now starts from substep 390.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 380 to 389. Now starts from substep 380.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 370 to 379. Now starts from substep 370.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 360 to 369. Now starts from substep 360.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 350 to 359. Now starts from substep 350.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 340 to 349. Now starts from substep 340.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 330 to 339. Now starts from substep 330.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 320 to 329. Now starts from substep 320.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 310 to 319. Now starts from substep 310.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 300 to 309. Now starts from substep 300.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 290 to 299. Now starts from substep 290.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 280 to 289. Now starts from substep 280.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 270 to 279. Now starts from substep 270.
[Genesis] [03:47:26] [DEBUG] Backward: Loaded checkpoint for global substep 260 to 269. Now starts from substep 260.
Traceback (most recent call last):
File "/workspace/genesis/examples/differentiable_push.py", line 152, in
main()
File "/workspace/genesis/examples/differentiable_push.py", line 137, in main
loss.backward() # this lets gradient flow all the way back to tensor input
^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/genesis/grad/tensor.py", line 75, in backward
self.scene._backward()
File "/opt/conda/lib/python3.11/site-packages/genesis/engine/scene.py", line 869, in _backward
self._step_grad()
File "/opt/conda/lib/python3.11/site-packages/genesis/engine/scene.py", line 691, in _step_grad
self._sim.collect_output_grads()
File "/opt/conda/lib/python3.11/site-packages/genesis/engine/simulator.py", line 315, in collect_output_grads
self.add_grad_from_state(state)
File "/opt/conda/lib/python3.11/site-packages/genesis/engine/simulator.py", line 304, in add_grad_from_state
solver.add_grad_from_state(solver_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/taichi/lang/kernel_impl.py", line 1221, in _getattr
x = super(cls, self).getattribute(item)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'SFSolver' object has no attribute 'add_grad_from_state'
[Genesis] [03:47:26] [INFO] 💤 Exiting Genesis and caching compiled kernels...
@zhenjia-xu
Copy link
Collaborator

Thanks for pointing it out. This bug has been fixed in #355, which will be merged to the main after review.

@zhenjia-xu zhenjia-xu linked a pull request Dec 27, 2024 that will close this issue
[FEATURE] add gradient API placeholder for SF solver #355
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[FEATURE] add gradient API placeholder for SF solver Genesis-Embodied-AI/Genesis
3 participants
@zhenjia-xu @mk2001233 @ziyanx02