Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

torch crashes in R-4.4.2 for MacOS #1213

Open
gilbertocamara opened this issue Nov 23, 2024 · 4 comments
Open

torch crashes in R-4.4.2 for MacOS #1213

gilbertocamara opened this issue Nov 23, 2024 · 4 comments

Comments

@gilbertocamara
Copy link

Dear @dfalbel as you probably already known, torch is crashing in R-4.4.2 for MacOS. The problem is caused because R-4.4 for MacOS now includes an OpenMP library as a common run-time support for the packages that use it. Meanwhile, torch is also loading its version of OpenMP. Since other R packages in version R-4.4 will now load the OpenMP, a clash of library versions happens, R aborts with the following error message;

OMP: Error #15: Initializing libiomp5.dylib, but found libomp.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.

I asked @s-u (Simon Urbanek, one of the maintainers of R for MacOS) and he provided the following advice:

====
"Any OpenMP-enabled package will trigger the bug. As the error you sent indicated the problem is the incompatibility of the run-time used by torch. If you use any R package with OpenMP support (which loads the official OpenMP run-time - in above example it is data.table) and then run torch afterwards, torch will fail, because it loads its private OpenMP run-time which is incompatible. This is precisely why we provide a common run-time and torch fails to use it so I would recommend filing a bug report with the torch package. It is easily fixed with something like

system(
 paste("ln","-sfn",shQuote(file.path(R.home(),"lib","libomp.dylib")),
 shQuote(system.file("lib","libiomp5.dylib",package="torch"))))

to point torch to the common run-time which guarantees compatibility between packages. After that fix torch and the above example works again.

The current torch setup essentially means that no one may use OpenMP as it will always clash with its private library (since no one else can use it). Even though it may not bite many people now given the sequence of packages involved, if we decided to enable OpenMP in CRAN build of R then torch won't work at all until this is fixed.

Would you be able to fix this problem? It affects potentially all MacOS torch users that upgrade to the latest R version.

@dfalbel
Copy link
Member

dfalbel commented Nov 23, 2024

Hi @gilbertocamara ,

This should be fixed in the current dev version of torch as we explicitly build libtorch without openMP.
Would you be able to try it and see if it works for you?

You can install a pre-built binary with:

options(timeout = 600) 
kind <- "cpu-intel"
version <- "0.13.0.9001"
options(repos = c(
  torch = sprintf("https://torch-cdn.mlverse.org/packages/%s/%s/", kind, version),
  CRAN = "https://cloud.r-project.org" # or any other from which you want to install the other R dependencies.
))
install.packages("torch", type = "binary")

or from github with remotes::install_github("mlverse/torch")

@gilbertocamara
Copy link
Author

Dear @dfalbel many thanks! I downloaded torch using remotes::install_github, compiled the package, and then ran torch::install_torch(). I got an error while trying to download the lantern library, as follows:

Error in `download_file()`:
✖ Unable to download from
  <https://torch-cdn.mlverse.org/binaries/refs/heads/cran/v0.13.0/latest/lantern-0.13.0.9001+cpu+x86_64-Darwin.zip>
ℹ Please verify that the URL is not blocked by your firewall. See also
  <https://torch.mlverse.org/docs/articles/installation.html#file-based-download>
Caused by error in `utils::download.file()`:
! cannot open URL 'https://torch-cdn.mlverse.org/binaries/refs/heads/cran/v0.13.0/latest/lantern-0.13.0.9001+cpu+x86_64-Darwin.zip'

@gilbertocamara
Copy link
Author

Dear @dfalbel the first option you propose (binary installation) works fine. Many thanks! Do you have an expected date to send it to CRAN?

@dfalbel
Copy link
Member

dfalbel commented Nov 25, 2024

Glad it worked out!

I plan to submit it soon, probably some time next week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants