Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

/user requests fail with "504 Server Error: Gateway Timeout" for users who have submitted many past jobs #1312

Open
asjohnston-asf opened this issue Nov 3, 2022 · 1 comment
Labels
bug Something isn't working Jira Bug Create a Jira Bug for this issue

Comments

@asjohnston-asf
Copy link
Member

asjohnston-asf commented Nov 3, 2022

Jira: https://asfdaac.atlassian.net/browse/TOOL-2041

The /user endpoint returns a list of all job names ever submitted by the logged in user. For users that have submitted many (hundreds of thousands) of past jobs, the query for past job names can take longer than the 30 second API Gateway Integration timeout, resulting in a HTTP 504 Gateway Timeout error being returned to the user.

The troublesome query is get_names_for_user() invoked by the /user handler at https://github.com/ASFHyP3/hyp3/blob/develop/apps/api/src/hyp3_api/handlers.py#L84

Here is an example traceback generated for this error when calling hyp3_sdk.hyp3.my_info() as reported by @cirrusasf :

Traceback (most recent call last):
  File "/home/jiangzhu/apps/anaconda3/envs/its-live-operations/lib/python3.10/site-packages/hyp3_sdk/exceptions.py", line 29, in _raise_for_hyp3_status
    response.raise_for_status()
  File "/home/jiangzhu/apps/anaconda3/envs/its-live-operations/lib/python3.10/site-packages/requests/models.py", line 1022, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 504 Server Error: Gateway Timeout for url: https://hyp3-its-live.asf.alaska.edu/user

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jiangzhu/apps/anaconda3/envs/its-live-operations/lib/python3.10/site-packages/hyp3_sdk/hyp3.py", line 410, in my_info
    _raise_for_hyp3_status(response)
  File "/home/jiangzhu/apps/anaconda3/envs/its-live-operations/lib/python3.10/site-packages/hyp3_sdk/exceptions.py", line 33, in _raise_for_hyp3_status
    raise ServerError
hyp3_sdk.exceptions.ServerError
@jtherrmann
Copy link
Contributor

Proposed implementation:

  • HyP3 automatically adds new job names to user record when jobs are submitted
  • We've updated the user records for all HyP3 deployments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Jira Bug Create a Jira Bug for this issue
Projects
None yet
Development

No branches or pull requests

2 participants