From 8e5659dbf60e416067d434336e535230af1de25e Mon Sep 17 00:00:00 2001 From: James Lamb Date: Sun, 7 Nov 2021 09:18:44 -0600 Subject: [PATCH 1/5] [docs][R-package] update cran-comments for v3.3.1 release (#4738) * [docs] [R-package] update cran-comments for v3.3.1 release * Update R-package/cran-comments.md Co-authored-by: Nikita Titov * update cran-comments.md now that v3.3.1 is accepted Co-authored-by: Nikita Titov --- R-package/cran-comments.md | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/R-package/cran-comments.md b/R-package/cran-comments.md index 8d9672e9d5a7..1d354cfa3777 100644 --- a/R-package/cran-comments.md +++ b/R-package/cran-comments.md @@ -1,11 +1,31 @@ # CRAN Submission History -## v3.3.1 - Submission 1 - (TBD) +## v3.3.1 - Submission 1 - (October 27, 2021) ### CRAN response +Accepted to CRAN on October 30, 2021. + +CRAN completed its checks and preparation of binaries on November 6, 2021. + ### Maintainer Notes +Submitted v3.3.1 to CRAN, with the following fixes for the issues that caused CRAN to reject v3.3.0 and archive the package: + +* https://github.com/microsoft/LightGBM/pull/4673 +* https://github.com/microsoft/LightGBM/pull/4714 + +Submitted with the following comment: + +> This submission contains {lightgbm} 3.3.1. +> Per CRAN's policies, I am submitting on it on behalf of the project's maintainer (Yu Shi), with his permission (https://github.com/microsoft/LightGBM/pull/4715#issuecomment-952537783). + +> {lightgbm} was removed from CRAN on October 25, 2021 due to issues detected in the gcc-ASAN and clang-ASAN checks. To the best of our knowledge, we believe this release fixes those issues. We have introduced automated testing that we believe faithfully reproduces CRAN's tests with sanitizers (https://github.com/microsoft/LightGBM/pull/4678). + +> Thank you very much for your time and consideration. + +Progress on the submission was tracked in https://github.com/microsoft/LightGBM/issues/4713. + ## v3.3.0 - Submission 1 - (October 8, 2021) ### CRAN response From 4e3fa2ebad17594b1667ff7513ea5150110f7bad Mon Sep 17 00:00:00 2001 From: shiyu1994 Date: Wed, 5 Jan 2022 11:05:44 +0800 Subject: [PATCH 2/5] apply patch for R4.2 on Windows (#4923) --- src/network/socket_wrapper.hpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/network/socket_wrapper.hpp b/src/network/socket_wrapper.hpp index caa12c279133..e4d63c434fc4 100644 --- a/src/network/socket_wrapper.hpp +++ b/src/network/socket_wrapper.hpp @@ -60,6 +60,9 @@ const int INVALID_SOCKET = -1; #endif #ifdef _WIN32 +#ifndef _UCRT +// Recent MinGW has inet_pton, which then causes compiler error in +// combination with this replacement. #ifndef _MSC_VER // not using visual studio in windows inline int inet_pton(int af, const char *src, void *dst) { @@ -86,6 +89,7 @@ inline int inet_pton(int af, const char *src, void *dst) { } #endif #endif +#endif #define MALLOC(x) HeapAlloc(GetProcessHeap(), 0, (x)) #define FREE(x) HeapFree(GetProcessHeap(), 0, (x)) From 1c3e8238efebb3b10227ebfe8a850ed383e4e621 Mon Sep 17 00:00:00 2001 From: Nikita Titov Date: Thu, 6 Jan 2022 17:35:06 +0300 Subject: [PATCH 3/5] release 3.3.2 --- .appveyor.yml | 2 +- R-package/cran-comments.md | 115 +++++++++++++++++++++++++++++++++++++ VERSION.txt | 2 +- 3 files changed, 117 insertions(+), 2 deletions(-) diff --git a/.appveyor.yml b/.appveyor.yml index d1e28319f19e..85ffe5ceddfb 100644 --- a/.appveyor.yml +++ b/.appveyor.yml @@ -1,4 +1,4 @@ -version: 3.3.1.{build} +version: 3.3.2.{build} image: Visual Studio 2015 platform: x64 diff --git a/R-package/cran-comments.md b/R-package/cran-comments.md index 1d354cfa3777..8b04d748149c 100644 --- a/R-package/cran-comments.md +++ b/R-package/cran-comments.md @@ -1,5 +1,120 @@ # CRAN Submission History +## v3.3.2 - Submission 1 - (TBD) + +### CRAN response + +### Maintainer Notes + +In this submission, we uploaded a patch that CRAN stuff provided us via e-mail. The full text of the e-mail from CRAN: + +```text +Dear maintainers, + +This concerns the CRAN packages + +Cairo cepreader gpboost httpuv ipaddress lightgbm proj4 prophet +RcppCWB RcppParallel RDieHarder re2 redux rgeolocate RGtk2 tth +udunits2 unrtf + +maintained by one of you: + +Andreas Blaette andreas.blaette@uni-due.de: RcppCWB +David Hall david.hall.physics@gmail.com: ipaddress +Dirk Eddelbuettel edd@debian.org: RDieHarder +Fabio Sigrist fabiosigrist@gmail.com: gpboost +Friedrich Leisch Friedrich.Leisch@R-project.org: tth +Girish Palya girishji@gmail.com: re2 +James Hiebert hiebert@uvic.ca: udunits2 +Jari Oksanen jhoksane@gmail.com: cepreader +Kevin Ushey kevin@rstudio.com: RcppParallel +ORPHANED: RGtk2 +Os Keyes ironholds@gmail.com: rgeolocate +Rich FitzJohn rich.fitzjohn@gmail.com: redux +Sean Taylor sjtz@pm.me: prophet +Simon Urbanek simon.urbanek@r-project.org: proj4 +Simon Urbanek Simon.Urbanek@r-project.org: Cairo +Winston Chang winston@rstudio.com: httpuv +Yu Shi yushi2@microsoft.com: lightgbm + +your packages need to be updated for R-devel/R 4.2 to work on Windows, +following the recent switch to UCRT and Rtools42. + +Sorry for the group message, please feel free to respond individually +regarding your package or ask specifically about what needs to be fixed. + +I've created patches for you, so please review them and fix your packages: + +https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsvn.r-project.org%2FR-dev-web%2Ftrunk%2FWindowsBuilds%2Fwinutf8%2Fucrt3%2Fr_packages%2Fpatches%2FCRAN%2F&data=04%7C01%7Cyushi2%40microsoft.com%7C8e6c353d1a8842c81eeb08d9bef5d835%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637750786169848244%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=rFGf7Y4Dvo6g1kzV%2BeAJDLGm1TUtzQsLsavElTw6H1U%3D&reserved=0 + +You can apply them as follows + +tar xfz package_1.0.0.tar.gz + +wget +https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsvn.r-project.org%2FR-dev-web%2Ftrunk%2FWindowsBuilds%2Fwinutf8%2Fucrt3%2Fr_packages%2Fpatches%2FCRAN%2Fpackage.diff&data=04%7C01%7Cyushi2%40microsoft.com%7C8e6c353d1a8842c81eeb08d9bef5d835%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637750786169848244%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=iyTjhoqvzj3IbQ8HGCZeh1IQl34FAGpIdVyZWkzNvO0%3D&reserved=0 + +patch --binary < package.diff + +These patches are currently automatically applied by R-devel on Windows +at installation time, which makes most of your packages pass their +checks (as OK or NOTE), but please check your results carefully and +carefully review the patches. Usually these changes were because of +newer GCC or newer MinGW in the toolchain, but some for other reasons, +and some of them will definitely have to be improved so that the package +keeps building also for older versions of R using Rtools40. We have only +been testing the patches with UCRT (and Rtools42) on Windows. + +For more information, please see + +https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.r-project.org%2FBlog%2Fpublic%2F2021%2F12%2F07%2Fupcoming-changes-in-r-4.2-on-windows%2F&data=04%7C01%7Cyushi2%40microsoft.com%7C8e6c353d1a8842c81eeb08d9bef5d835%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637750786169848244%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=SY77zgtbDbHvTxTgPLOoe%2Fw5OZDhXvJoxpVOoEaKoYo%3D&reserved=0 +https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.r-project.org%2FWindowsBuilds%2Fwinutf8%2Fucrt3%2Fhowto.html&data=04%7C01%7Cyushi2%40microsoft.com%7C8e6c353d1a8842c81eeb08d9bef5d835%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637750786169848244%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=dlVJ4nhQlmDPd56bHoVsWZuRfrUUorvOWxoUTmVDM%2Bg%3D&reserved=0 + +Once you add your patches/fix the issues, your package will probably +show a warning during R CMD check (as patching would be attempted to be +applied again). That's ok, at that point please let me know and I will +remove my patch from the repository of automatically applied patches. + +If you end up just applying the patch as is, there is probably no need +testing on your end, but you can do so using Winbuilder, r-hub, github +actions (e.g. https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkalibera%2Fucrt3&data=04%7C01%7Cyushi2%40microsoft.com%7C8e6c353d1a8842c81eeb08d9bef5d835%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637750786169848244%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=msqoPzqDStlAUn%2Bb6gGevwFPD%2FaNL5dTxiNud2Sqzy8%3D&reserved=0). + +If you wanted to test locally on your Windows machine and do not have a +UCRT version of R-devel yet, please uninstall your old version of +R-devel, delete the old library used with that, install a new UCRT +version of R-devel , and install Rtools42. You can keep Rtools40 +installed if you need it with R 4.1 or earlier. + +Currently, the new R-devel can be downloaded from +https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.r-project.org%2Fnosvn%2Fwinutf8%2Fucrt3%2Fweb%2Frdevel.html&data=04%7C01%7Cyushi2%40microsoft.com%7C8e6c353d1a8842c81eeb08d9bef5d835%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637750786169848244%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0hCwONzLmcW0GIXNqiOZQEIuhNA%2BjHhQvXsofs8J98o%3D&reserved=0 + +And Rtools42 from +https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.r-project.org%2Fnosvn%2Fwinutf8%2Fucrt3%2Fweb%2Frtools.html&data=04%7C01%7Cyushi2%40microsoft.com%7C8e6c353d1a8842c81eeb08d9bef5d835%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637750786169848244%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=WLWLbOyQKbaYz8gkfKz2sqoGknjIOtl1aGAhUF%2Bpylg%3D&reserved=0 + +If you end up testing locally, you can use R_INSTALL_TIME_PATCHES +environment variable to disable the automated patching, see the "howto" +document above. That way you could also see what the original issue was +causing. + +If you wanted to find libraries to link for yourself, e.g. in a newer +version of your package, please look for "Using findLinkingOrder with +Rtools42 (tiff package example)" in the "howto" document above. I +created the patches for you manually before we finished this script, so +you may be able to create a shorter version using it, but - it's +probably not worth the effort. + +If you wanted to try in a virtual machine, but did not have a license, +you can use also an automated setup of a free trial VM from +https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.r-project.org%2FBlog%2Fpublic%2F2021%2F03%2F18%2Fvirtual-windows-machine-for-checking-r-packages&data=04%7C01%7Cyushi2%40microsoft.com%7C8e6c353d1a8842c81eeb08d9bef5d835%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637750786169848244%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=aFFQYuC9CoBwBiLgZHi8N3yUnSiHu5Xtdqb2YBiMIHQ%3D&reserved=0 + +(but that needs a very good and un-metered network connection to install) + +Please let us know if you have any questions. + +Thanks, +Tomas & Uwe +``` + ## v3.3.1 - Submission 1 - (October 27, 2021) ### CRAN response diff --git a/VERSION.txt b/VERSION.txt index bea438e9ade7..477254331794 100644 --- a/VERSION.txt +++ b/VERSION.txt @@ -1 +1 @@ -3.3.1 +3.3.2 From 324d2a144816171338cb1f2cc967fd0e0dd65384 Mon Sep 17 00:00:00 2001 From: GitHub Actions Bot Date: Thu, 6 Jan 2022 14:40:57 +0000 Subject: [PATCH 4/5] Auto-update configure --- R-package/configure | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/R-package/configure b/R-package/configure index 9f9e36e4312b..79470dc515aa 100755 --- a/R-package/configure +++ b/R-package/configure @@ -1,6 +1,6 @@ #! /bin/sh # Guess values for system-dependent variables and create Makefiles. -# Generated by GNU Autoconf 2.69 for lightgbm 3.3.1. +# Generated by GNU Autoconf 2.69 for lightgbm 3.3.2. # # # Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc. @@ -576,8 +576,8 @@ MAKEFLAGS= # Identity of this package. PACKAGE_NAME='lightgbm' PACKAGE_TARNAME='lightgbm' -PACKAGE_VERSION='3.3.1' -PACKAGE_STRING='lightgbm 3.3.1' +PACKAGE_VERSION='3.3.2' +PACKAGE_STRING='lightgbm 3.3.2' PACKAGE_BUGREPORT='' PACKAGE_URL='' @@ -1182,7 +1182,7 @@ if test "$ac_init_help" = "long"; then # Omit some internal or obsolete options to make the list less imposing. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF -\`configure' configures lightgbm 3.3.1 to adapt to many kinds of systems. +\`configure' configures lightgbm 3.3.2 to adapt to many kinds of systems. Usage: $0 [OPTION]... [VAR=VALUE]... @@ -1244,7 +1244,7 @@ fi if test -n "$ac_init_help"; then case $ac_init_help in - short | recursive ) echo "Configuration of lightgbm 3.3.1:";; + short | recursive ) echo "Configuration of lightgbm 3.3.2:";; esac cat <<\_ACEOF @@ -1311,7 +1311,7 @@ fi test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF -lightgbm configure 3.3.1 +lightgbm configure 3.3.2 generated by GNU Autoconf 2.69 Copyright (C) 2012 Free Software Foundation, Inc. @@ -1328,7 +1328,7 @@ cat >config.log <<_ACEOF This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. -It was created by lightgbm $as_me 3.3.1, which was +It was created by lightgbm $as_me 3.3.2, which was generated by GNU Autoconf 2.69. Invocation command line was $ $0 $@ @@ -2398,7 +2398,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" -This file was extended by lightgbm $as_me 3.3.1, which was +This file was extended by lightgbm $as_me 3.3.2, which was generated by GNU Autoconf 2.69. Invocation command line was CONFIG_FILES = $CONFIG_FILES @@ -2451,7 +2451,7 @@ _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_version="\\ -lightgbm config.status 3.3.1 +lightgbm config.status 3.3.2 configured by $0, generated by GNU Autoconf 2.69, with options \\"\$ac_cs_config\\" From dce7e58b020bc14b69eefc31546c366971ecb2d9 Mon Sep 17 00:00:00 2001 From: Nikita Titov Date: Fri, 5 Nov 2021 04:17:47 +0300 Subject: [PATCH 5/5] [ci] Temporary pin dask version at CI (#4770) * pin Dask version at CI * Update .vsts-ci.yml * Update .vsts-ci.yml * workaround for Python 3.6 * Update test.sh --- .ci/test.sh | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/.ci/test.sh b/.ci/test.sh index 047395797951..00b2a3a38356 100755 --- a/.ci/test.sh +++ b/.ci/test.sh @@ -114,7 +114,14 @@ if [[ $TASK == "swig" ]]; then exit 0 fi -conda install -q -y -n $CONDA_ENV cloudpickle dask distributed joblib matplotlib numpy pandas psutil pytest scikit-learn scipy +# temporary fix for https://github.com/microsoft/LightGBM/issues/4769 +if [[ $PYTHON_VERSION == "3.6" ]]; then + DASK_DEPENDENCIES="dask distributed" +else + DASK_DEPENDENCIES="dask=2021.9.1 distributed=2021.9.1" +fi + +conda install -q -y -n $CONDA_ENV cloudpickle ${DASK_DEPENDENCIES} joblib matplotlib numpy pandas psutil pytest scikit-learn scipy pip install graphviz # python-graphviz from Anaconda is not allowed to be installed with Python 3.9 if [[ $OS_NAME == "macos" ]] && [[ $COMPILER == "clang" ]]; then