Watcher.claim_jobs failing with a KeyError

[mbthornton-lbl]

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/global/homes/n/nmdcda/nmdc_automation/dev/nmdc_automation/nmdc_automation/run_process/run_workflows.py", line 110, in <module>
    cli()
  File "/global/cfs/cdirs/m3408/nmdc_automation/dev/nmdc_automation/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/cfs/cdirs/m3408/nmdc_automation/dev/nmdc_automation/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/global/cfs/cdirs/m3408/nmdc_automation/dev/nmdc_automation/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/cfs/cdirs/m3408/nmdc_automation/dev/nmdc_automation/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/cfs/cdirs/m3408/nmdc_automation/dev/nmdc_automation/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/cfs/cdirs/m3408/nmdc_automation/dev/nmdc_automation/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/cfs/cdirs/m3408/nmdc_automation/dev/nmdc_automation/.venv/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/n/nmdcda/nmdc_automation/dev/nmdc_automation/nmdc_automation/run_process/run_workflows.py", line 98, in daemon
    watcher.watch()
  File "/global/homes/n/nmdcda/nmdc_automation/dev/nmdc_automation/nmdc_automation/workflow_automation/watch_nmdc.py", line 360, in watch
    self.cycle()
  File "/global/homes/n/nmdcda/nmdc_automation/dev/nmdc_automation/nmdc_automation/workflow_automation/watch_nmdc.py", line 320, in cycle
    self.claim_jobs(unclaimed_jobs)
  File "/global/homes/n/nmdcda/nmdc_automation/dev/nmdc_automation/nmdc_automation/workflow_automation/watch_nmdc.py", line 370, in claim_jobs
    opid = claim["detail"]["id"]
           ~~~~~^^^^^^^^^^
KeyError: 'detail'

[mbthornton-lbl]

Test results on Dev show that this error is not occurring.

Log snippet showing successfully claimed job:

2024-11-25 15:49:08,772 INFO: Found 1 unclaimed jobs.
2024-11-25 15:49:08,772 INFO: Claiming job nmdc:c2b7c884-ab78-11ef-8298-3e652b5abb3d
2024-11-25 15:49:08,775 DEBUG: Starting new HTTPS connection (1): api-dev.microbiomedata.org:443
2024-11-25 15:49:09,388 DEBUG: https://api-dev.microbiomedata.org:443 "POST /token HTTP/11" 200 None
2024-11-25 15:49:09,391 DEBUG: Starting new HTTPS connection (1): api-dev.microbiomedata.org:443
2024-11-25 15:49:14,907 DEBUG: https://api-dev.microbiomedata.org:443 "POST /jobs/nmdc:c2b7c884-ab78-11ef-8298-3e652b5abb3d:claim HTTP/11" 200 None