Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] fix some shellcheck warnings in package-building scripts #6641

Merged
merged 36 commits into from
Oct 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
9854b8a
Shellcheck compliance improvement (commit squashing)
Aug 26, 2024
e8db3e4
rolledback some sh files to address PR review
Aug 27, 2024
043141a
Addressing comments on PR review
Aug 27, 2024
1bb9bcc
Merge branch 'microsoft:master' into master
vnherdeiro Aug 27, 2024
526e3f9
Update docs/build-docs.sh
jameslamb Aug 28, 2024
36b3da2
checking gh login
Aug 28, 2024
e3801fa
Merge branch 'master' into master
jameslamb Aug 30, 2024
35b74a8
fixing cmakelint
vnherdeiro Aug 30, 2024
3bf6164
Merge branch 'master' of github.com:vnherdeiro/LightGBM
vnherdeiro Aug 30, 2024
eba9f6b
another fix to failing ci
vnherdeiro Aug 30, 2024
e7b13d9
fixing linting ignoring files
vnherdeiro Aug 31, 2024
1dfee74
Merge branch 'master' into master
jameslamb Sep 2, 2024
8aee3db
Merge branch 'microsoft:master' into master
vnherdeiro Sep 4, 2024
b36ddf2
Fixing shellcheck issues on build-python and build-cran-package scripts
vnherdeiro Sep 4, 2024
0dc8fe4
Update build-cran-package.sh
vnherdeiro Sep 4, 2024
19958b9
Update build-cran-package.sh
vnherdeiro Sep 4, 2024
6f04ce0
Update build-cran-package.sh
vnherdeiro Sep 4, 2024
357bbc7
Adressing PR comments
vnherdeiro Sep 4, 2024
a88adf3
Merge branch 'shellcheck_fixes' of github.com:vnherdeiro/LightGBM int…
vnherdeiro Sep 4, 2024
67a784d
fixing CI
vnherdeiro Sep 4, 2024
b0ef7d2
Update build-python.sh
vnherdeiro Sep 7, 2024
96cd009
Update build-python.sh
vnherdeiro Sep 7, 2024
05602a7
Update build-python.sh
vnherdeiro Sep 7, 2024
23ad962
Update build-python.sh
vnherdeiro Sep 7, 2024
884e79e
Update build-python.sh
vnherdeiro Sep 7, 2024
8f7bbb4
fixing quotemark typo
vnherdeiro Sep 7, 2024
35a698a
fix CI
vnherdeiro Sep 7, 2024
2920b60
fix CI
vnherdeiro Sep 7, 2024
8f0af7a
triggering new CI (scikit-learn dev has changed)
vnherdeiro Sep 14, 2024
7fcef2e
adressing PR comments
vnherdeiro Sep 15, 2024
6673348
Merge branch 'master' into shellcheck_fixes
jameslamb Oct 7, 2024
372dd3c
Merge branch 'master' into shellcheck_fixes
jameslamb Oct 9, 2024
ac4c128
Merge branch 'master' into shellcheck_fixes
vnherdeiro Oct 10, 2024
e7cd689
Merge branch 'master' into shellcheck_fixes
vnherdeiro Oct 14, 2024
4235cdc
Merge branch 'master' into shellcheck_fixes
vnherdeiro Oct 16, 2024
5221cda
ignore warnings about BUILD_ARGS and PIP_INSTALL_ARGS
jameslamb Oct 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 9 additions & 10 deletions build-cran-package.sh
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
# # skip vignette building
# sh build-cran-package.sh --no-build-vignettes

set -e -E -u
set -e -u

# Default values of arguments
BUILD_VIGNETTES=true
Expand Down Expand Up @@ -63,7 +63,7 @@ CURRENT_DATE=$(date +'%Y-%m-%d')

# R packages cannot have versions like 3.0.0rc1, but
# 3.0.0-1 is acceptable
LGB_VERSION=$(cat VERSION.txt | sed "s/rc/-/g")
LGB_VERSION=$(head -1 ./VERSION.txt | sed "s/rc/-/g")

# move relevant files
cp -R R-package/* "${TEMP_R_DIR}"
Expand Down Expand Up @@ -91,10 +91,10 @@ mkdir -p "${EIGEN_R_DIR}"

modules="Cholesky Core Dense Eigenvalues Geometry Householder Jacobi LU QR SVD"
for eigen_module in ${modules}; do
cp external_libs/eigen/Eigen/${eigen_module} "${EIGEN_R_DIR}/${eigen_module}"
if [ ${eigen_module} != "Dense" ]; then
cp "external_libs/eigen/Eigen/${eigen_module}" "${EIGEN_R_DIR}/${eigen_module}"
if [ "${eigen_module}" != "Dense" ]; then
mkdir -p "${EIGEN_R_DIR}/src/${eigen_module}/"
cp -R external_libs/eigen/Eigen/src/${eigen_module}/* "${EIGEN_R_DIR}/src/${eigen_module}/"
cp -R "external_libs/eigen/Eigen/src/${eigen_module}"/* "${EIGEN_R_DIR}/src/${eigen_module}/"
fi
done

Expand Down Expand Up @@ -130,6 +130,7 @@ cd "${TEMP_R_DIR}"
# Rtools35 (used with R 3.6 on Windows) doesn't support C++17
LGB_CXX_STD="C++17"
using_windows_and_r3=$(
# shellcheck disable=SC2016
Rscript -e 'cat(.Platform$OS.type == "windows" && R.version[["major"]] < 4)'
)
if test "${using_windows_and_r3}" = "TRUE"; then
Expand All @@ -142,7 +143,7 @@ cd "${TEMP_R_DIR}"
# not allow you to use compiler flag '-Wno-unknown-pragmas' or
# pragmas that suppress warnings.
echo "Removing unknown pragmas in headers"
for file in $(find . -name '*.h' -o -name '*.hpp' -o -name '*.cpp'); do
find . \( -name '*.h' -o -name '*.hpp' -o -name '*.cpp' \) -exec \
sed \
-i.bak \
-e 's/^.*#pragma clang diagnostic.*$//' \
Expand All @@ -151,9 +152,7 @@ cd "${TEMP_R_DIR}"
-e 's/^.*#pragma region.*$//' \
-e 's/^.*#pragma endregion.*$//' \
-e 's/^.*#pragma warning.*$//' \
"${file}"
done
find . -name '*.h.bak' -o -name '*.hpp.bak' -o -name '*.cpp.bak' -exec rm {} \;
{} +

# 'processx' is listed as a 'Suggests' dependency in DESCRIPTION
# because it is used in install.libs.R, a file that is not
Expand All @@ -164,7 +163,7 @@ cd "${TEMP_R_DIR}"
DESCRIPTION

echo "Cleaning sed backup files"
rm *.bak
find . -name '*.bak' -exec rm {} \;

cd "${ORIG_WD}"

Expand Down
36 changes: 20 additions & 16 deletions build-python.sh
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@
# Install into user-specific instead of global site-packages directory.
# Only used with 'install' command.

set -e -E -u
set -e -u

echo "building lightgbm"

Expand Down Expand Up @@ -91,42 +91,42 @@ while [ $# -gt 0 ]; do
# customized library paths #
############################
--boost-dir|--boost-dir=*)
if [[ "$1" != *=* ]];
if echo "$1" | grep -q '^*=*$';
vnherdeiro marked this conversation as resolved.
Show resolved Hide resolved
then shift;
fi
BOOST_DIR="${1#*=}"
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.define.Boost_DIR='${BOOST_DIR}'"
;;
--boost-include-dir|--boost-include-dir=*)
if [[ "$1" != *=* ]];
if echo "$1" | grep -q '^*=*$';
then shift;
fi
BOOST_INCLUDE_DIR="${1#*=}"
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.define.Boost_INCLUDE_DIR='${BOOST_INCLUDE_DIR}'"
;;
--boost-librarydir|--boost-librarydir=*)
if [[ "$1" != *=* ]];
if echo "$1" | grep -q '^*=*$';
then shift;
fi
BOOST_LIBRARY_DIR="${1#*=}"
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.define.BOOST_LIBRARYDIR='${BOOST_LIBRARY_DIR}'"
;;
--boost-root|--boost-root=*)
if [[ "$1" != *=* ]];
if echo "$1" | grep -q '^*=*$';
then shift;
fi
BOOST_ROOT="${1#*=}"
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.define.Boost_ROOT='${BOOST_ROOT}'"
;;
--opencl-include-dir|--opencl-include-dir=*)
if [[ "$1" != *=* ]];
if echo "$1" | grep -q '^*=*$';
then shift;
fi
OPENCL_INCLUDE_DIR="${1#*=}"
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.define.OpenCL_INCLUDE_DIR='${OPENCL_INCLUDE_DIR}'"
;;
--opencl-library|--opencl-library=*)
if [[ "$1" != *=* ]];
if echo "$1" | grep -q '^*=*$';
then shift;
fi
OPENCL_LIBRARY="${1#*=}"
Expand Down Expand Up @@ -254,14 +254,14 @@ create_isolated_source_dir() {
modules="Cholesky Core Dense Eigenvalues Geometry Householder Jacobi LU QR SVD"
for eigen_module in ${modules}; do
cp \
external_libs/eigen/Eigen/${eigen_module} \
./lightgbm-python/external_libs/eigen/Eigen/${eigen_module}
if [ ${eigen_module} != "Dense" ]; then
mkdir -p ./lightgbm-python/external_libs/eigen/Eigen/src/${eigen_module}/
"external_libs/eigen/Eigen/${eigen_module}" \
"./lightgbm-python/external_libs/eigen/Eigen/${eigen_module}"
if [ "${eigen_module}" != "Dense" ]; then
mkdir -p "./lightgbm-python/external_libs/eigen/Eigen/src/${eigen_module}/"
cp \
-R \
external_libs/eigen/Eigen/src/${eigen_module}/* \
./lightgbm-python/external_libs/eigen/Eigen/src/${eigen_module}/
"external_libs/eigen/Eigen/src/${eigen_module}"/* \
"./lightgbm-python/external_libs/eigen/Eigen/src/${eigen_module}/"
fi
done

Expand Down Expand Up @@ -308,6 +308,7 @@ if test "${INSTALL}" = true; then
# use regular-old setuptools for these builds, to avoid
# trying to recompile the shared library
sed -i.bak -e '/start:build-system/,/end:build-system/d' pyproject.toml
# shellcheck disable=SC2129
echo '[build-system]' >> ./pyproject.toml
echo 'requires = ["setuptools"]' >> ./pyproject.toml
echo 'build-backend = "setuptools.build_meta"' >> ./pyproject.toml
Expand Down Expand Up @@ -339,6 +340,7 @@ fi
if test "${BUILD_SDIST}" = true; then
echo "--- building sdist ---"
rm -f ../dist/*.tar.gz
# shellcheck disable=SC2086
python -m build \
--sdist \
--outdir ../dist \
Expand All @@ -349,6 +351,7 @@ fi
if test "${BUILD_WHEEL}" = true; then
echo "--- building wheel ---"
rm -f ../dist/*.whl || true
# shellcheck disable=SC2086
python -m build \
--wheel \
--outdir ../dist \
Expand All @@ -360,18 +363,19 @@ if test "${INSTALL}" = true; then
echo "--- installing lightgbm ---"
cd ../dist
if test "${BUILD_WHEEL}" = true; then
PACKAGE_NAME="lightgbm*.whl"
PACKAGE_NAME="$(echo lightgbm*.whl)"
else
PACKAGE_NAME="lightgbm*.tar.gz"
PACKAGE_NAME="$(echo lightgbm*.tar.gz)"
fi
# ref for use of '--find-links': https://stackoverflow.com/a/52481267/3986677
# shellcheck disable=SC2086
pip install \
${PIP_INSTALL_ARGS} \
--force-reinstall \
--no-cache-dir \
--no-deps \
--find-links=. \
${PACKAGE_NAME}
"${PACKAGE_NAME}"
cd ../
fi

Expand Down
Loading