Update irodsbackend connection handling

[mikkonie]

We have two problems with the current way of handling disconnections in iRODS.

• The structure of the API is illogical
  • Some methods require the connection while others do not
  • We have to use conn=False to avoid opening unnecessary connections
• The cleanup() command in python-irodsclient does not work as expected
  • cleanup() has to be manually called, otherwise iRODS connections are left hanging
  • If called in IrodsAPI.__del__(), this fails for some reason
  • This causes Postgres server connections filling up sodar-docker-compose#32 and possibly iRODS disconnect() exceptions in production #1542.
  • In cubi-tk, this issue has been previously solved by using context managers

Refactoring the entire backend and its usages to always call sessions as a context manager, or otherwise envoke cleanup() before exiting, is a huge task. Also, this will require us opening and closing a lot more connections than we do now, which may affect performance. The latter is hard to measure before implementing and deploying this improvement.

I don't really see a better way to handle this, so will probably just have to bite the bullet and get it done with.

Tasks

• Update session initialization to allow for context manager
• Update usage
• Optimize the amount of connections opened where applicable
• Update tests
• Update docs as needed

[mikkonie]

Done. Some niceties I left out to get a fix out for #1542 ASAP, will add separate issues for them.