Disable empirical normalizer updates on resume training #30
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nikitardn
reviewed
Jun 21, 2024
rsl_rl/runners/on_policy_runner.py
Outdated
| else: | ||
| until = 1.0e8 | ||
| self.obs_normalizer = EmpiricalNormalization(shape=[num_obs], until=until).to(self.device) | ||
| self.critic_obs_normalizer = EmpiricalNormalization(shape=[num_critic_obs], until=until).to(self.device) |
There was a problem hiding this comment.
I think the correct way to solve the problem is to save and load the internal count of EmpiricalNormalization. The current solution breaks if you resume training before 1.0e8 steps
There was a problem hiding this comment.
https://github.com/tasdep/rsl_rl-1/blob/f2b721aae7c33a34155a0db6135a559b75513334/rsl_rl/runners/on_policy_runner.py#L257-L262
what is saved_dict[infos] intended for? Or should I just add another term to the saved_dict?
tasdep
force-pushed
the
dev/resume_training
branch
from
2175df4
to
48af1c1
Compare
…oaded with the module
tasdep
force-pushed
the
dev/resume_training
branch
from
48af1c1
to
40e669d
Compare
|
@nikitardn should we merge this? |
Mayankm96
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This completely disables the updates if "resume" is in the agent_cfg and True.
Should we check how many iterations/updates were done previously and only disable if over a threshold?