Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[R-package] lightgbm removed from CRAN #4713

Closed
jameslamb opened this issue Oct 25, 2021 · 29 comments
Closed

[R-package] lightgbm removed from CRAN #4713

jameslamb opened this issue Oct 25, 2021 · 29 comments

Comments

@jameslamb
Copy link
Collaborator

Description

As of this morning, {lightgbm} has been removed from CRAN.

https://cran.r-project.org/web/packages/lightgbm/index.html

Package ‘lightgbm’ was removed from the CRAN repository.

Formerly available versions can be obtained from the archive.

Archived on 2021-10-25 as check problems were not corrected in time.

A summary of the most recent check results can be obtained from the check results archive.

Please use the canonical form https://CRAN.R-project.org/package=lightgbm to link to this page.

Additional Comments

"check problems" most likely refers to #4674 and #4680.

@jameslamb
Copy link
Collaborator Author

This is going to be VERY disruptive for users of the R package 😬

@shiyu1994 did you receive any emails from CRAN in the last few weeks asking us to re-submit? If so can you please post them here in their entirety (with dates and times) so we can understand why the package has been removed?

I can open a v3.3.1 release pull request today, but the last time the package was taken down by CRAN, we were told we could not re-submit for a month 😬

@llrs
Copy link

llrs commented Oct 25, 2021

Hi! Sorry to hear that the package was removed from CRAN. I'm in no way to speak for CRAN but in case it helps I've heard that you can re-submit your package with more frequency than once a month if the changes are to fix problems found on CRAN checks. Hope it can be made available soon!

@shiyu1994
Copy link
Collaborator

After searching my email, I found this was sent on Oct 12th,
cran
It seems that today is the deadline for fixing the problem. I'm sorry about missing that email. I'll be careful next time.

[???? [email protected] ?????????? http://aka.ms/LearnAboutSenderIdentification?????????????]

Dear maintainer,

Please see the problems shown on
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcran.r-project.org%2Fweb%2Fchecks%2Fcheck_results_lightgbm.html&data=04%7C01%7Cyushi2%40microsoft.com%7Ce45fee760332456a232608d98cd54d41%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637695670177785045%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=R%2Bt3eXZ7gA92SstlqOTv5yBIgQ8JL%2FluW8n1qLt7Yh4%3D&reserved=0.

Please correct before 2021-10-25 to safely retain your package on CRAN.

Do remember to look at the 'Additional issues'.

The CRAN Team

@shiyu1994
Copy link
Collaborator

I should have noticed the email and remind the team to fix the problems before today. Sorry about making such a mistake.

@jameslamb
Copy link
Collaborator Author

Thanks for the quick response @shiyu1994 . It's ok, these things happen!

I'll open up a PR for a new release today.

@mayer79
Copy link
Contributor

mayer79 commented Oct 26, 2021

@jameslamb : I was quick enough to test the 3.3 version on CRAN and it worked very well, fantastic work! No hard crashs as with the old version. I fear those CRAN e-mails as they sometimes tend to even end up in the spam folder... As @llrs wrote, I don't think there is a problem to immediately resubmit a fixed version.

This is the "additional issues" link that is mentioned by Brian:
https://www.stats.ox.ac.uk/pub/bdr/memtests/gcc-ASAN/lightgbm/

@jameslamb
Copy link
Collaborator Author

I don't think there is a problem to immediately resubmit a fixed version

Thanks, for sure! I wanted to wait to see what the exact email from CRAN said, since in the past they have explicitly told us not to resubmit for a fixed amount of time. Since @shiyu1994 hasn't received any such emails saying there is a limit on when we can resubmit, I hope we'll be able to submit today if @shiyu1994 and @StrikerRUS have time to review #4715 and the PRs linked there.

This is the "additional issues" link that is mentioned by Brian:

Yep, thanks! You might be interested to see...I have an open PR to replicate CRAN's gcc-ASAN, gcc-UBSAN, clang-ASAN, and clang-UBSAN tests in this project's continuous integration: #4678

Thanks to the amazing work by @wch and collaborators on https://github.com/wch/r-debug.

@AdrianAntico
Copy link

I just noticed this issue as well. I tried to install from github and received the following error message, in case this helps in any way...

> devtools::install_github('microsoft/LightGBM/R-package', upgrade = FALSE, dependencies = FALSE, force = TRUE)
Downloading GitHub repo microsoft/LightGBM@HEAD
"C:\PROGRA~1\Git\cmd\git.exe" clone --depth 1 --no-hardlinks --recurse-submodules https://github.com/boostorg/compute C:\Users\Bizon\AppData\Local\Temp\Rtmpemuek9\remotes525c4b387b14/microsoft-LightGBM-adef94f/R-package/../external_libs/compute
"C:\PROGRA~1\Git\cmd\git.exe" clone --depth 1 --no-hardlinks --recurse-submodules https://gitlab.com/libeigen/eigen.git C:\Users\Bizon\AppData\Local\Temp\Rtmpemuek9\remotes525c4b387b14/microsoft-LightGBM-adef94f/R-package/../external_libs/eigen
"C:\PROGRA~1\Git\cmd\git.exe" clone --depth 1 --no-hardlinks --recurse-submodules https://github.com/fmtlib/fmt.git C:\Users\Bizon\AppData\Local\Temp\Rtmpemuek9\remotes525c4b387b14/microsoft-LightGBM-adef94f/R-package/../external_libs/fmt
"C:\PROGRA~1\Git\cmd\git.exe" clone --depth 1 --no-hardlinks --recurse-submodules https://github.com/lemire/fast_double_parser.git C:\Users\Bizon\AppData\Local\Temp\Rtmpemuek9\remotes525c4b387b14/microsoft-LightGBM-adef94f/R-package/../external_libs/fast_double_parser
√  checking for file 'C:\Users\Bizon\AppData\Local\Temp\Rtmpemuek9\remotes525c4b387b14\microsoft-LightGBM-adef94f\R-package/DESCRIPTION' ...
-  preparing 'lightgbm': (560ms)
E  checking DESCRIPTION meta-information ... 
   Malformed package version.
   
   See section 'The DESCRIPTION file' in the 'Writing R Extensions'
   manual.
Error: Failed to install 'lightgbm' from GitHub:
  System command 'Rcmd.exe' failed, exit status: 1, stdout + stderr:
E> * checking for file 'C:\Users\Bizon\AppData\Local\Temp\Rtmpemuek9\remotes525c4b387b14\microsoft-LightGBM-adef94f\R-package/DESCRIPTION' ... OK
E> * preparing 'lightgbm':
E> * checking DESCRIPTION meta-information ... ERROR
E> Malformed package version.
E> 
E> See section 'The DESCRIPTION file' in the 'Writing R Extensions'
E> manual.

@jameslamb
Copy link
Collaborator Author

@AdrianAntico you cannot use devtools::install_github() / remotes::install_github() to install {lightgbm}.

The file structure in source control here is not a valid, self-contained R package. For example, the core C/C++ code in the project is stored in src/ at the top-level of the repo, and then copied at build time into the relevant directories when building the R and Python packages.

See https://github.com/microsoft/LightGBM/tree/master/R-package#contents for a list of available options for installing the R package.

We will try to restore the R package to CRAN as soon as possible.

@AdrianAntico
Copy link

Ah, makes sense

@leecommamichael
Copy link

This hurts a lot. Why is there a 1 month wait for resubmission? That makes zero sense.

@llrs
Copy link

llrs commented Oct 28, 2021

@leecommamichael For fixes there isn't a 1 month wait for resubmitting. That period is just if there are updates that do not fix the package on issues detected by CRAN checks. Check this message from one of the CRAN volunteer:

Then policies you are quoting have two reasons:

  1. we take a look what is going on if a submission comes in too quickly
    as this is typically suspicious. Hence a quick submission creates manual
    work in any case,

  2. each version of a package is archived, so when people submit
    frequently, CRAN is growing quickly.

As CRAN is a repository for releases software, we expect that releases
are well tested and hence do not happen too frequently.

In case you nevertheless found a serious bug, please submit a fixed
version and explain why this was needed in the submission comments.

@leecommamichael
Copy link

@leecommamichael For fixes there isn't a 1 month wait for resubmitting. That period is just if there are updates that do not fix the package on issues detected by CRAN checks. Check this message from one of the CRAN volunteer:

Then policies you are quoting have two reasons:

  1. we take a look what is going on if a submission comes in too quickly
    as this is typically suspicious. Hence a quick submission creates manual
    work in any case,
  2. each version of a package is archived, so when people submit
    frequently, CRAN is growing quickly.

As CRAN is a repository for releases software, we expect that releases
are well tested and hence do not happen too frequently.
In case you nevertheless found a serious bug, please submit a fixed
version and explain why this was needed in the submission comments.

I appreciate the detail, Lluís. I assumed from earlier discussion, and the fact that it has been many days without the package in CRAN, that we'd be waiting a full month.

@shiyu1994
Copy link
Collaborator

We've uploaded R package for the new release 3.3.1 to CRAN. And we are waiting for a manual check.

@jameslamb
Copy link
Collaborator Author

Anyone wanting to track the progress of {lightgbm}'s latest submission to CRAN is encouraged to use https://lockedata.github.io/cransays/articles/dashboard.html, maintained by @maelle and others.

At the moment it looks like {lightgbm} has been placed in the same manual-review queue as completely new packages.

image

There are 77 total incoming submissions to CRAN being evaluated right now. I don't know how to turn that into an estimate of how long it will take for the package to be restored.

Those who really really cannot wait can install directly from the most recent artifact published to GitHub (but this will mean a build from source, as we don't prepare pre-compiled binaries like CRAN does).

install.packages("https://github.com/microsoft/LightGBM/releases/download/v3.3.1/lightgbm-3.3.1-r-cran.tar.gz")

I got that URL from https://github.com/microsoft/LightGBM/releases/tag/v3.3.1.

@llrs
Copy link

llrs commented Oct 28, 2021

Yes, once packages are archived they are considered as new and must go under the manual inspection again. I made some analysis of time to approval of new packages here, expect 1 week but it is highly variable so it could be just 1 or 2 days.

@mikemahoney218
Copy link
Contributor

Those who really really cannot wait can install directly from the most recent artifact published to GitHub (but this will mean a build from source, as we don't prepare pre-compiled binaries like CRAN does).

install.packages("https://github.com/microsoft/LightGBM/releases/download/v3.3.1/lightgbm-3.3.1-r-cran.tar.gz")

If the goal is to install any version of lightgbm (and not specifically the new 3.3.1 version), a second option would be to install from the MRAN snapshot from the day before archival:

install.packages(
  "lightgbm",
  repos = "https://cran.microsoft.com/snapshot/2014-10-24/"
  )

I have no idea how critical the sanitizer checks actually are (so if using 3.3.0 would be actively discouraged), but this installation method takes advantage of the binary packages.

@Javdat
Copy link

Javdat commented Oct 28, 2021

Those who really really cannot wait can install directly from the most recent artifact published to GitHub (but this will mean a build from source, as we don't prepare pre-compiled binaries like CRAN does).

install.packages("https://github.com/microsoft/LightGBM/releases/download/v3.3.1/lightgbm-3.3.1-r-cran.tar.gz")

I got that URL from https://github.com/microsoft/LightGBM/releases/tag/v3.3.1.

Note: you might need to specify repos=NULL otherwise R might search for package with that URL name and fail:

install.packages(
  "https://github.com/microsoft/LightGBM/releases/download/v3.3.1/lightgbm-3.3.1-r-cran.tar.gz",
  repos = NULL
)

@StrikerRUS
Copy link
Collaborator

At the moment it looks like {lightgbm} has been placed in the same manual-review queue as completely new packages.

Our status has been changed to pending which means

pending: the CRAN maintainers are waiting for an action on your side. You should check your emails!

image

@shiyu1994 Please check your e-mail.

@shiyu1994
Copy link
Collaborator

shiyu1994 commented Oct 29, 2021

The last email I got from CRAN is this, which is sent 3 hours after I confirmed the submission.
cran
It seems there's no action being asked to take? Following is the log file and the link the the check results.

[00details.log[24792].log]
(https://github.com/microsoft/LightGBM/files/7438441/_00details.log_.24792.log)

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwin-builder.r-project.org%2Fincoming_pretest%2Flightgbm_3.3.1_20211028_035536%2F&data=04%7C01%7Cyushi2%40microsoft.com%7C501cb39749f94b20d82b08d999cb3880%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637709920840925512%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=12OpqFovY8ovbw22DcWv4jlydHaERb8p5KMC7DFhUzk%3D&reserved=0

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwin-builder.r-project.org%2Fincoming_pretest%2Flightgbm_3.3.1_20211028_035536%2FWindows%2F00check.log&data=04%7C01%7Cyushi2%40microsoft.com%7C501cb39749f94b20d82b08d999cb3880%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637709920840925512%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=Jtu9GNWqkC8aeUqBKJ%2BMfo%2FQ%2F2Lg7zI%2BhDKn041zIIs%3D&reserved=0

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwin-builder.r-project.org%2Fincoming_pretest%2Flightgbm_3.3.1_20211028_035536%2FDebian%2F00check.log&data=04%7C01%7Cyushi2%40microsoft.com%7C501cb39749f94b20d82b08d999cb3880%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637709920840925512%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=quuN4L48V3XeIMCtrk2bXTsQJAR3Avo7VvVVuZbtRGc%3D&reserved=0

@jameslamb
Copy link
Collaborator Author

thanks @shiyu1994 ! That's a good sign...the two jobs referenced in those links had no issues.

@StrikerRUS
Copy link
Collaborator

Looks like v3.3.1 release is starting to roll out at the CRAN 🤞

https://cran.r-project.org/web/checks/check_results_lightgbm.html
image

@shiyu1994
Copy link
Collaborator

Great to hear that!

@jameslamb
Copy link
Collaborator Author

A few more checks (linux, mac, solaris) have passed, and clang-ASAN and gcc-ASAN issues have been removed from the package's checks page.

image

https://cran.r-project.org/web/checks/check_results_lightgbm.html

I don't know if it's possible to see any information for the "Additional Checks" (like gcc-ASAN) that says "CRAN ran this and it passed"...in my experience, you only get to learn that those checks ran if they fail.

At the moment, according to https://cran.r-project.org/web/packages/lightgbm/index.html, there are v3.3.1 binaries available for armg64 mac environments, and otherwise the package can be installed from source.

image

Still a lot of checks to go, but at least now it is once again possible to get {lightgbm} from CRAN with install.packages("lightgbm", repos = "https://cran.r-project.org")!

@jameslamb
Copy link
Collaborator Author

According to https://cran.r-project.org/web/packages/lightgbm/index.html, binaries for Mac are now available.

image

No checks have been run yet on Windows, I think related to some expected maintenance on CRAN's Windows cluster today. From r-pkg-devel mailing list (https://stat.ethz.ch/pipermail/r-package-devel/2021q4/007498.html):

Dear all,

winbuilder service and CRAN incoming checks (submission is possible)
will be offline tomorrow (Nov 1) from 10 am CET to not later than 6pm.

Some CRAN hardware in Dortmund will be physically migrated to a new
location.

Best,
Uwe Ligges
(CRAN team)

@jameslamb
Copy link
Collaborator Author

jameslamb commented Nov 2, 2021

Checks going well so far. Windows binaries for R 4.2.x 4.1.x are now available.

Comparing https://cran.r-project.org/web/checks/check_results_lightgbm.html to https://cran.r-project.org/web/checks/check_results_xgboost.html, I think the following checks remain:

  • r-devel-linux-x86_64-debian-clang
  • r-devel-windows-x86_64
  • r-devel-windows-x86_64-gcc10-UCRT
  • r-patched-linux-x86_64
  • r-release-linux-x86_64
  • r-oldrel-windows-ix86+x86_64

As well as any of CRAN's "Additional Checks" (link), whose status we can't see.

@mayer79
Copy link
Contributor

mayer79 commented Nov 2, 2021

Excellent, thank you so much!

@jameslamb
Copy link
Collaborator Author

According to https://cran.r-project.org/web/checks/check_results_lightgbm.html, {lightgbm} 3.3.1 is now passing all of CRAN's checks, and CRAN has finished preparing Windows and Mac binaries.

Thanks to everyone who helped with this issue, and apologies for the disruption!

@github-actions
Copy link

This issue has been automatically locked since there has not been any recent activity since it was closed.
To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues
including a reference to this.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 16, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

9 participants