WIP: intermediary file format specification

[gessulat]

@sambenfredj 's pull requests introduces streaming at several places of the workflow but those intermediary file formats are not specified and documented yet. In addition, switching to a binary format such as partitioned pyarrow datasets would speed up IO.

Schemas will be defined here after @wfondrie 's switch to polars.

Tasks

• document where we need intermediary files
• document how the files relate to input files, to each other, and to output files (e.g. how should they be joined?)
• specify columns and their datatype and potential indeces on columns