Design clear caching protocol

[delgadom]

This should be implemented in the docs/examples as a proposed API.

The cache protocol must clearly define:

• possible types of cache configurations
• how these configurations are specified (config file, python, CLI)
• what is cached on write for each cache configuration
• what is cached on read for each cache configuration
• any modifications that can be made to the defaults using method flags/CLI args

In addition, it should clearly articulate how users are expected to interact with the cache in each of our main uses cases (big servers, e.g. Sacagawea, and personal/OSDC machines) including cache management.

[delgadom]

@jgerardsimcock FYI I'm going to brainstorm ideas here:
https://docs.google.com/document/d/13Q9UB8oOrgHo0Tr8CQFt5vJ5VdbEoSoL9TTePeD6eYA/edit?usp=sharing

[delgadom]

@jgerardsimcock how do you feel about the proposal in the google doc? should this get closed and move to developing tests?

[delgadom]

The api design in cache_policies is, I think, good.

As for work on this issue, there is some work in cache_by_default that may be informative, but this branch should by and large be discarded. All of the work is geared toward the previous system. Most of these changes should be done through an overhaul of data_file, whereby caching is done in data_file based on the value of a cache_policy parameter (string or object? not sure) that is passed into file operation calls. This will make this module much more clear and testable.