Skip to content
This repository has been archived by the owner on Aug 31, 2023. It is now read-only.

Retrieve BQ metadata asynchronously to reduce GAE costs #25

Open
marcin-kolda opened this issue Jan 8, 2018 · 1 comment
Open

Retrieve BQ metadata asynchronously to reduce GAE costs #25

marcin-kolda opened this issue Jan 8, 2018 · 1 comment

Comments

@marcin-kolda
Copy link
Contributor

We could group retrieving table metadata into larger tasks (i.e. 10 API calls per task) and then retrieve them asynchronously. This should reduce GAE costs.
@marcin-kolda
Copy link
Contributor Author

Alternatively we could also retrieve BQ metadata in batches, as BQ supports batch requests. See also Python Client docs.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@marcin-kolda