Add cultivars site metadata #71
Conversation
|
recent changes : a new method brapi_get takes the endpoint and parameters, getting the BRAPI_URL and BRAPI_VERSION from either predefined values in the class or environment variables. this method also paginates through results in cases where there is more than one page. other methods for layouts, germplasm and observationunits changed to use this method |
new method for getting exp id adding brapi endpoints adding methods for brapi calls brapi get studies, brapi get studies/germplasm added removing todo since experiment_id added to sites json method for getting map of site_ids and cultivar info per experiment id. using brapi to get map of cultivars by site_id, this is added to cached sites json file and then added to site metadata in lemnatec fixing unicode u adding site names, this will help since brapi endpoint that gives treatments has studyDbId, but not siteid - it has sitename adding treatments to cached sites file treatments added to metadata some plots do not have cultivar or treatment data available. in these cases the site entry will have 'no info available' as the value for cultivar and treatments all dict entries now strings check for keys replaces try/except, ending W or E removed these changes work with an issue and pull request created in brapi. pagination did not work properly for the observationunits endpoint using observationUnitName and not location_abbrevation using 'definition' instead of 'treatment_description' to match actual key in bety db adding page size to call to brapi observationunits using one method brapi_get to make all requests still need to fix getting all the observationunits iterate through results for observation units page by page one method brapi_get handles the brapi url, and version method also paginates through results new brapi.py class - brapi methods moved there v1 added to endpoints, not read from environment variable no need to remove ending W or E from plot names
tcnichol
force-pushed
the
add-cultivars-site-metadata
branch
from
c431f5a to
7619b44
Compare
|
Should work now. |
|
I fixed the issue with urlparse (it was a version issue but tested with Docker locally.) Scaled brapi down and up, so the key for location_abbrevation is now observationUnitName, so it matches what key is expected by the terrautils.brapi |
|
Another warning - get_sites takes quite a long time because it has to paginate through all the observationunits results. Since we are caching files this might not matter much in production. Will follow up with @max-zilla about optimizing this Monday. |
|
Changes should be taken into account. The only issue is that this is very slow, some optimization might be needed. |
This pull request is dependent on this pull request in brapi
terraref/brapi#31
It resolves this issue
terraref/computing-pipeline#573
The following changes have been made.
In the class betydb new methods are added to make calls to brapi studies, layouts, germplasm and observationunits endpoints.
betydb.get_sites now includes info on the sites, cultivars and treatments for each site.
in lemnatec._get_sites this new metadata for cultivar and treatments is added to the site metadata.