feat: add SPS for trainer #129
Conversation
|
Looks good to me! |
| @@ -511,7 +511,15 @@ def run_experiment(_config: DictConfig) -> float: | |||
| logger.log({"timestep": t}, t, eval_step, LogEvent.MISC) | |||
| if ep_completed: # only log episode metrics if an episode was completed in the rollout. | |||
| logger.log(episode_metrics, t, eval_step, LogEvent.ACT) | |||
There was a problem hiding this comment.
Isn't a little bit ambiguous to call that ACTOR steps_per_seconds as it is capturing the "learn" function ?
There was a problem hiding this comment.
I'm not sure i fully understand. In the anakin systems, the "Actors" generate the data within the "learn" function. The amount of data produced / elapsed time. The only real issue is that the elapsed time includes the learn step so the actual actor steps per second would be faster if you only timed that specific chunk of code.
There was a problem hiding this comment.
Unless by saying capturing the learn function, you mean this issue:
The only real issue is that the elapsed time includes the learn step so the actual actor steps per second would be faster if you only timed that specific chunk of code.
There was a problem hiding this comment.
That was my point, the actor sps include the learning (which include backprop + inferences e.g. within the _actor_loss_fn or the _critic_loss_fn) It makes me think that the actual "acting" time is mostly composed of this learning component than anything else
There was a problem hiding this comment.
Yeah you are right. This is actually something necessary to fix. Its going to be annoying since it'll need to be fixed in every single file. Can you make an issue and then we can chat about it there.
There was a problem hiding this comment.
I would be happy to help you fix it, we just need to investigate how to do it for one file
There was a problem hiding this comment.
awesome. Thanks so much.
What?
Add a steps per second metric to the trainer.
Why?
Useful to know how many optimiser steps are being performed per second.
How?
Multiply the number of updates per eval by the algorithms epoch number of epoch*num minibatch number.