Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Github Authentication required to manage API rate limiting #153

Open
lepsalex opened this issue Jan 26, 2021 · 0 comments
Open

Github Authentication required to manage API rate limiting #153

lepsalex opened this issue Jan 26, 2021 · 0 comments
Labels
major High Priority

Comments

@lepsalex
Copy link
Contributor

lepsalex commented Jan 26, 2021

When starting many workflows at once we can hit the GH rate limiter (we are no authenticated as a user currently) and this results in a 422 Unprocessable Entity

Response:

<ClientResponse(https://wes.rdpc.cancercollaboratory.org/api/v1/runs) []>
<CIMultiDictProxy('Server': 'openresty/1.15.8.2', 'Date': 'Tue, 26 Jan 2021 22:40:54 GMT', 'Content-Type': 'application/json', 'Content-Length': '142', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=15724800; includeSubDomains', 'Cache-Control': 'no-cache, no-store, max-age=0, must-revalidate', 'Pragma': 'no-cache', 'Expires': '0', ragma': 'no-cache', 'Expires': '0', 'X-Content-Type-Options': 'nosniff', 'X-Frame-Options': 'DENY', 'X-XSS-Protection': '1 ; mode=block', 'Referrer-Policy': 'no-referrer')>

Log from Management:

2021-01-26 22:40:54.960 ERROR 1 --- [flow-service-98] o.i.a.w.service.NextflowService          : nextflow runtime exception
nextflow.exception.AbortOperationException: API rate limit exceeded -- Provide your GitHub user name and password to get a higher rate limit
at nextflow.scm.RepositoryProvider.checkResponse(RepositoryProvider.groovy:170) ~[nextflow-20.07.1.jar!/:na]
at nextflow.scm.RepositoryProvider.invoke(RepositoryProvider.groovy:126) ~[nextflow-20.07.1.jar!/:na]
at nextflow.scm.RepositoryProvider.memoizedMethodPriv$invokeAndParseResponseString(RepositoryProvider.groovy:209) ~[nextflow-20.07.1.jar!/:na]
at jdk.internal.reflect.GeneratedMethodAccessor299.invoke(Unknown Source) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:na]
...

Nextflow has a way of doing this built-in, just need to add the config options to management: https://www.nextflow.io/docs/latest/sharing.html#github-credentials

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
major High Priority
Projects
None yet
Development

No branches or pull requests

1 participant