KeyError: 'action'

[hnekoeiq]

Hi, I tried launching r2d1 on atari using atari_r2d1_async_alt and I get this error:

call string:
 taskset -c 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22 /home/mila/n/nekoeiha/.conda/envs/rlpyt/bin/python /home/mila/n/nekoeiha/MILA/rlpyt/rlpyt/experiments/scripts/atari/dqn/launch/pabti/../../train/atari_r2d1_async_alt.py 0slt_24cpu_4gpu_0hto_1ass_2sgr_1alt /home/mila/n/nekoeiha/MILA/rlpyt/data/local/20200824/161821/atari_r2d1_async_alt/gravitar 0 async_alt_pabti
Unable to import tensorboard SummaryWriter, proceeding without.
using seed 131
Traceback (most recent call last):
  File "/home/mila/n/nekoeiha/MILA/rlpyt/rlpyt/experiments/scripts/atari/dqn/launch/pabti/../../train/atari_r2d1_async_alt.py", line 48, in <module>
    build_and_train(*sys.argv[1:])
  File "/home/mila/n/nekoeiha/MILA/rlpyt/rlpyt/experiments/scripts/atari/dqn/launch/pabti/../../train/atari_r2d1_async_alt.py", line 44, in build_and_train
    runner.train()
  File "/home/mila/n/nekoeiha/MILA/rlpyt/rlpyt/runners/async_rl.py", line 87, in train
    throttle_itr, delta_throttle_itr = self.startup()
  File "/home/mila/n/nekoeiha/MILA/rlpyt/rlpyt/runners/async_rl.py", line 147, in startup
    seed=self.seed,
  File "/home/mila/n/nekoeiha/MILA/rlpyt/rlpyt/samplers/async_/alternating_sampler.py", line 24, in async_initialize
    return super().async_initialize(agent, *args, **kwargs)
  File "/home/mila/n/nekoeiha/MILA/rlpyt/rlpyt/samplers/async_/base.py", line 34, in async_initialize
    subprocess=True)  # Would like subprocess=True, but might hang?
  File "/home/mila/n/nekoeiha/MILA/rlpyt/rlpyt/samplers/buffer.py", line 29, in build_samples_buffer
    all_action = buffer_from_example(examples["action"], (T + 1, B), agent_shared)
  File "<string>", line 2, in __getitem__
  File "/home/mila/n/nekoeiha/.conda/envs/rlpyt/lib/python3.7/multiprocessing/managers.py", line 834, in _callmethod
    raise convert_to_error(kind, result)
KeyError: 'action'

I'll be grateful if you could help me figure it out.

[astooke]

Yes, sorry this is an uninformative error I sometimes run into. It usually means there is something wrong inside the subprocess which is generating the example action, observation, etc... One thing to try to get more information would be to set subprocess=False here:

  File "/home/mila/n/nekoeiha/MILA/rlpyt/rlpyt/samplers/async_/base.py", line 34, in async_initialize
    subprocess=True)  # Would like subprocess=True, but might hang?

but it might still be hard to follow since the async sampler opens in its own subprocess.

Let me know if that helps?

[csingh27]

I am stuck with this error @astooke ! Any suggestions ?

[tsuchishinka]

Yes, sorry this is an uninformative error I sometimes run into. It usually means there is something wrong inside the subprocess which is generating the example action, observation, etc... One thing to try to get more information would be to set subprocess=False here:

  File "/home/mila/n/nekoeiha/MILA/rlpyt/rlpyt/samplers/async_/base.py", line 34, in async_initialize
    subprocess=True)  # Would like subprocess=True, but might hang?

but it might still be hard to follow since the async sampler opens in its own subprocess.

Let me know if that helps?

I encountered a similar error. When I modify 'subprocess=False', it shows as follows:

[ZiwenZhuang]

This should be the early termination of the subprocess which is running the function get_example_outputs during sampler initialization. I'm not sure why the subprocess must be True during the initialization.

But, every time KeyError: 'action' happens when the interaction between agent (at CPU mode) and the environment goes wrong and stops the subprocess. It is either the env instance encountered any error or the agent did not support non-batchwise agent input in agent.step() implementation.

:)