Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

solr_dist can make the dmci worker to SIGKILL when extracting featureType from OPeNDAP link, if thredds or lustre is not responsing. #240

Open
magnarem opened this issue Oct 7, 2024 · 1 comment
Assignees

Comments

@magnarem
Copy link
Contributor

magnarem commented Oct 7, 2024

If thredds.met.no or the NFS lustre Gateways, are having issues,
The part of the solr_dist (solrindexer) that tries to extract featureType from the netcdf-file using the OPeNDAP url, will hang, and then the gunicorn dmci app will timeout and get SIGKILLED, and the solr document processed will not be indexed.

A solution of this is to tell the netCDF4-python (netcdf c-library) to have a timeout short than gunicorn. (Default gunicorn is 30s).

The solution according to the netCDF4-python mailinglist, is to add a .dodsrcin the home of the dmci app.

So the solution will be to add a .dodsrcfile with the contents: HTTP.TIMEOUT=20 and place it in the root folder of this repository.
Then the opendap request should timeout before the gunicorn process times, out. solr_dist will print a timeout error, but the document will still be indexed with this solution.

@mortenwh
Copy link
Collaborator

mortenwh commented Oct 7, 2024

@shamlymajeed - can you take a look at this, also in relation to #211?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants