Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Increase the sparkNetworkTimeout default value for the affiliation matching algorithm #1501

Open
marekhorst opened this issue Nov 27, 2024 · 0 comments
Open

Increase the sparkNetworkTimeout default value for the affiliation matching algorithm #1501

marekhorst opened this issue Nov 27, 2024 · 0 comments
Assignees
@marekhorst
Labels
functionality: affiliations

Comments

@marekhorst
Copy link
Member

Currently sparkNetworkTimeout value specified for the affiliation matching algorithm is defined as 5m.

This seems to be insufficient, spark job on the current graph fails with:

org.apache.spark.rpc.RpcTimeoutException: Futures timed out after [300 seconds]. This timeout is controlled by spark.network.timeout

This 5m value is lower than the default spark.network.timeout value set for an infrastructure (600s) so we should align them and increase the sparkNetworkTimeout to 600s.
@marekhorst marekhorst self-assigned this Nov 27, 2024
marekhorst added a commit that referenced this issue Nov 27, 2024
@marekhorst
Closes #1501: Increase the sparkNetworkTimeout default value for the …
197159e 
…affiliation matching algorithm

Setting `sparkNetworkTimeout` value to `600s`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@marekhorst marekhorst

Labels
functionality: affiliations
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@marekhorst