Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: more robust GitHub actions caching #9

Closed
alexsavulescu opened this issue Mar 7, 2022 · 3 comments
Closed

CI: more robust GitHub actions caching #9

alexsavulescu opened this issue Mar 7, 2022 · 3 comments
Labels
CI enhancement New feature or request question Further information is requested

Comments

@alexsavulescu
Copy link
Member

alexsavulescu commented Mar 7, 2022

Currently we only rely on a simple keyword + date for caching.
The idea is to somehow make use of ModelDB API so that we know when to retrieve (updated or new) models? Otherwise add a manual trigger?

@alexsavulescu alexsavulescu added CI enhancement New feature or request question Further information is requested labels Mar 7, 2022
@olupton
Copy link
Collaborator

olupton commented Jun 17, 2022

Based on https://neuron.yale.edu/ftp/neuron/2019northwestern/python-2019-10-18.html it may be possible to use queries like https://senselab.med.yale.edu/_site/webapi/object.json/12631?woatts=23,24,25,26,27,28,112,113,114,155,249,299,471,488,502,582 to get the last-modified date of a model without downloading its content. It's not immediately clear how to query many models at once.

@alexsavulescu
Copy link
Member Author

Ideally we'd get a unique API call with latest update times (like we have for getting all NEURON accession numbers https://github.com/neuronsimulator/nrn-modeldb-ci/blob/master/modeldb/config.py#L6 ) but I remember I couldn't find any.

@olupton
Copy link
Collaborator

olupton commented Jul 18, 2023

This was addressed by #88.
Now in each run we make two API calls, one for the full list of models and the second for the full list of their modification times.
The cache metadata also includes the modification time, and this is used to only re-download new or updated models.

@olupton olupton closed this as completed Jul 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI enhancement New feature or request question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants