Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

getDTthreads MIN(OMP_NUM_THREADS, OMP_THREAD_LIMIT) #3390

Merged
merged 3 commits into from
Feb 11, 2019
Merged

getDTthreads MIN(OMP_NUM_THREADS, OMP_THREAD_LIMIT) #3390

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9af61f1
Added omp_get_thread_limit() to getDTthreads
mattdowle Feb 11, 2019
6b2f4ac
news item tweak
mattdowle Feb 11, 2019
8ef4e4f
coverage
mattdowle Feb 11, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions NEWS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@

6. v1.12.0 did not compile on Solaris 10 using Oracle Developer Studio 12.6, [#3285](https://github.com/Rdatatable/data.table/issues/3285). Many thanks to Prof Ripley for providing and testing a patch. For future reference and other package developers, a `const` variable should not be passed to OpenMP's `num_threads()` directive otherwise `left operand must be modifiable lvalue` occurs.

7. `getDTthreads()` respected the `OMP_NUM_THREADS` environment variable but not `OMP_THREAD_LIMIT`, [#3300](https://github.com/Rdatatable/data.table/issues/3300). There are two very similar OpenMP functions: `omp_get_max_threads()` and `omp_get_thread_limit()`. It now calls both and chooses the minimum. Note that these environment variables should be set before the R session starts. Using the R command `Sys.setenv()` to set them is too late because the OpenMP runtime is already running by then; use `setDTthreads()` instead.


### Changes in v1.12.0 (13 Jan 2019)

Expand Down
8 changes: 5 additions & 3 deletions src/openmp-utils.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,17 +16,19 @@
* avoid the deadlock/hang (#1745 and #1727) and return to prior state afterwards.
*/

static int DTthreads = 0; // Never read directly, hence static; always go via getDTthreads().
static int DTthreads = 0; // Never read directly, hence static; always go via getDTthreads().
static bool RestoreAfterFork = true; // see #2885 in v1.12.0

int getDTthreads() {
#ifdef _OPENMP
int ans = omp_get_max_threads();
int ans = MIN(omp_get_max_threads(), omp_get_thread_limit()); // #3300
if (ans>1024) {
warning("omp_get_max_threads() has returned %d. This is unreasonably large. Applying hard limit of 1024. Please check OpenMP environment variables and other packages using OpenMP to see where this very large number has come from. Try getDTthreads(verbose=TRUE).", ans);
// # nocov start
warning("MIN(omp_get_max_threads(), omp_get_thread_limit()) has returned %d. This is unreasonably large. Applying hard limit of 1024. Please check OpenMP environment variables and other packages using OpenMP to see where this very large number has come from. Try getDTthreads(verbose=TRUE).", ans);
// to catch INT_MAX for example, which may be the case if user or another package has called omp_set_num_threads(omp_get_thread_limit())
// 1024 is a reasonable hard limit based on a comfortable margin above the most number of CPUs in one server I have heard about
ans=1024;
// # nocov end
}
if (DTthreads>0 && DTthreads<ans) ans = DTthreads;
if (ans<1) ans=1;
Expand Down