[R-package] FATAL Error when run train() #4045
Comments
jameslamb
changed the title
|
Hi, thanks for using To help you out, we need more information. Could you please provide the following?
From your screenshot it looks like you MIGHT be facing the same issue as #4007 , but it's hard to know from what you've provided. |
|
what commands did you use to install {lightgbm}? what version of {lightgbm} are you on? what operating system are you using? can you provide a full reproducible example (code that I could try running to see if I get the same error)? # GITHUB INSTALLATIONS ----
remotes::install_github("curso-r/treesnip", dependencies = TRUE, force = TRUE)
devtools::install_github('catboost/catboost', subdir = 'catboost/R-package')
# LIBRARIES ----
# Machine Learning Packages
library(lightgbm)
library(catboost)
library(xgboost)
# Tidymodels
library(treesnip)
library(tidymodels)
# Core
library(tidyverse)
# LIGHTGBM BASIC USAGE ----
data(agaricus.train, package='lightgbm')
data(agaricus.test, package='lightgbm')
train <- agaricus.train
dtrain <- lgb.Dataset(train$data, label = train$label)
params = list(
objective = "regression" , metric = "l2"
)
model <- lgb.train(params = params,data = dtrain) |
|
Thanks! Hope you don't mind, but I added formatting like this so the code is formatted as code:
I will take a look at this in the coming days. You should subscribe to #4007 in case that fix ends up being the same as this issue. |
Merged
|
Dear developers, I'm having a similar case, receiving a Fatal error and Rstudio session aborts. my session info: lightgbm version =3.1.1 A reproducible example When the last line is executed : Also, it did use to work until a few days ago not sure what has changed. Any advice \ idea is appriciated |
|
Thanks very much. At this time we don't have an update to offer. The development effort in this project is very focused on the next release (v3.2.0, #3872) right now and since this bug seems to exist on 3.1.1 it isn't blocking 3.2.0. I'll revisit this in a few days and see what I can find. |
|
@jameslamb : Just as info: This example reproduces the hard crash on Windows and lgb 3.1.1 (if built from CRAN). Loading dplyr after lightgbm solves the issue. The crash happens in these lines called in The input in lgb.call seems the same (with and without crash), so I am not sure what is going on. Update 1I have rebuilt lightgbm 3.2.0 and 3.1.1 on Windows and could not reproduce the problem. Currently I think it is a problem of the CRAN version. Update 2The CRAN version yields the error no matter if installing the zip or from source. Update 3Same behaviour with LGB 3.2 on CRAN. |
|
@mayer79 in terminal then in R Tried the following code then but it still crashed Did you experience different when installing from the repo? |
|
@tonyk7440 : Good finding. What goes wrong in I actually built the project with (just as a test). |
|
Sorry I haven't had the capacity to look at this yet. Windows is not my main day-to-day operating system to I need to really find some dedicated time to do Windows work. @tonyk7440 it would be helpful if you could share the full logs generated when you installed using a package built from That's really important, because if you can reproduce this error when building from source then the problem has to be more than "something is specifically wrong with binaries installed from CRAN". |
|
Ok this is interesting! I tried three different ways of installing and ran the R code in #4045 (comment) as a test to see if it runs or crashes.
So it does seem like there is something going wrong in the
|
|
..continued due to comment max length, only caught the end of the second half of the output of the command below due to it getting cutoff in the console `install.packages("lightgbm_3.2.0.99.tar.gz", repos = NULL, type = "source")` output
|
|
Thanks for that. For future times when you need to post a lot of data, GitHub does support attaching files to comments. I'll try to look into this in the next few days. Keep in mind that
|
|
I typed up an investigation as I went tonight and the post got kind of long. Long story short...I THINK I found the problem. I believe This could explain the behavior where just Linking to #3016, which I think is related. Ok I started testing tonight. On Windows 10, R 4.0.5, I'm able to reproduce this with an even smaller repro, that doesn't require library(tibble)
library(lightgbm)
data(agaricus.train, package='lightgbm')
train <- agaricus.train
dtrain <- lgb.Dataset(train$data, label = train$label)
dtrain$construct()If you comment out Sometimes after
I was able to trace the crash to this call: LightGBM/R-package/R/lgb.Dataset.R Lines 226 to 237 in d517ba1 The is a call to an R wrapper around a function that creates a LightGBM dataset from a sparse (CSC) matrix. I tried printing each of those passed-in parameter values, and saw one that surprised me. I see some things that look possibly-relevant. It looks like routine registration in library(lightgbm)
library(tibble)
library(xgboost)
dll_info <- getLoadedDLLs()[c("lightgbm", "tibble", "xgboost")]
dll_infoNotice that unlike those other libraries,
I can see with LightGBM/R-package/src/lightgbm_R.cpp Line 723 in d517ba1 getDLLRegisteredRoutines(dll_info$lightgbm[["path"]])
Compare that with, for example, the results from getDLLRegisteredRoutines(dll_info$xgboost[["path"]])I think this line in "Writing R Extensions", specifically https://cran.r-project.org/doc/manuals/r-release/R-exts.html#dyn_002eload-and-dyn_002eunload, is relevant.
And this one from https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Registering-native-routines
I think that something at LightGBM/R-package/src/lightgbm_R.cpp Line 723 in d517ba1 |
|
I believe I've found the root cause and have opened #4155 with a fix. You can test it with the following. git clone --recursive [email protected]:microsoft/LightGBM.git
cd LightGBM
git fetch origin fix/r-registration
git checkout fix/r-registration
sh build-cran-package.sh
R CMD INSTALL lightgbm_3.2.0.99.tar.gzThen you can try this sample code or any of the other examples mentioned in this thread. library(tibble)
library(lightgbm)
data(agaricus.train, package='lightgbm')
train <- agaricus.train
dtrain <- lgb.Dataset(train$data, label = train$label)
dtrain$construct()@mayer79 @tonyk7440 @TalWac @nmnbkhr thanks for all your help with reproducible examples and testing information! Sorry it took so long to get to this. I hope #4155 will fix this and any other instability you've seen with |
|
@jameslamb Awesome! I can confirm that your as well as "my" example works using r-registration branch. |
|
I've tentatively closed this by merging #4155 because, at least in my testing, I believe that that PR will have resolved this issue. For anyone following along, there are some major changes coming to the R package over the next month, to try to reduce the risk of issues like this. If you want to stay up to date on what is happening, you can subscribe to the following:
The next release of |
|
This issue has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Description
getting Fatal Error when i run lgb.train()
Reproducible example
model <- lgb.train(params = params,data = dtrain)
Environment info
The text was updated successfully, but these errors were encountered: