Site configuration needs to be rethought for clients

[cmickeyb]

Files like site.psh are really designed for client consumption and are not necessary on the services. However, PDO does not have a very good method for clients to add information about sites. Currently the assumption is that the client edits site.psh to include "interesting" endpoints. Some of that is necessary. The storage, enclave and provisioning service groups are based on a client's preferences. However, the endpoints and information about those endpoints is determined by the services.

It would be nice to have a "real" database for information about all services and service groups. Maybe each site could export a TOML file that contains information about the endpoints that could be incorporated into that database. So a "site.toml" file that is built from the configuration information in the service configuration files.

[cmickeyb]

Proposed flow to address this issue:

• Remove site.psh... it includes updates to the eservice database and sets up the service groups for easier specification
• Replace the json-file based eservice database with a shelve based database for all of the service endpoints; this might be a little tricky because the eservice database is also a cache of active service clients
• Add an "import" function that can take the site.toml that is created by the pdo-configure-services script; this would import into the service database information about all of the endpoints at a particular host/site. The site.toml file becomes the means through which endpoints can be advertised & shared
• Add a load/save pair of functions for the eservice, pservice and sservice commands in the pdo-shell. The functions would operate on toml files that could also be edited by hand. this would separate building groups (a very client thing) from exporting information about a site (a very services thing)

Sync with @prakashngit on this.

[cmickeyb]

This was addressed in #481