Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check that dataset being pulled is the most recent #14

Open
2 of 3 tasks
ciarag01 opened this issue Feb 8, 2023 · 1 comment
Open
2 of 3 tasks

Check that dataset being pulled is the most recent #14

ciarag01 opened this issue Feb 8, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@ciarag01
Copy link

ciarag01 commented Feb 8, 2023

As discussed with @rmccreath :

I use the GP Practice Contact Details and List Sizes dataset, currently using get_resource and dataset ID but this needs the ID to be updated whenever the dataset is which is easy to miss. Looking at get_dataset instead, it does seem that setting max_resources to 1 will give the most recent file every time but this isn't obvious from the documentation or the data returns. Possible fixes:

I've mentioned GP Practice Contact Details and List Sizes dataset because that's how this came up for me but this will likely be useful for any dataset that gets updated regularly.

@rmccreath rmccreath added the enhancement New feature or request label Feb 8, 2023
@Moohan
Copy link
Member

Moohan commented Feb 8, 2023

I had exactly this use case in mind when I first built get_dataset(). I don't think the package does any explicit sorting of the IDs / datasets so we're just relying on the API returning them in age order (which I'm pretty certain it does).

I think we should confirm that the API will always list datasets in age order (someone who know the backend will have to look this up).

There's probably not scope to sort the dataset IDs as, to me they just seem like random strings?

Pending the checks on the API my preference would be to update the documentation to make it clear that it returns them in age order. We could modify the param to take "all" or "latest" (default) as well as a specific number, which might also help with clarity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants