configure script doesn't work on dash-only systems

[bedroge]

Describe the bug
While bumping the Lmod version in Gentoo (gentoo/gentoo#39798), it was reported that the configure script fails for a few people (also see https://bugs.gentoo.org/890939) who use dash for /bin/sh. It seems there are some dash-incompatible checks (e.g. https://github.com/TACC/Lmod/blob/main/configure#L4438 and https://github.com/TACC/Lmod/blob/main/configure#L4462, where the substring expansion is not supported on dash, but there are more).

I tried to reproduce this, and it seems like it only happens on systems with /bin/sh -> dash and which don't have bash at all (otherwise configure is picking up bash anyway?). That made me wonder if dash-only is supposed to be supported by Lmod? If not, maybe it would be nice if it could detect this and fail with a clear error message? If it is supported, some checks need to be changed to something that is supported by dash.

To Reproduce
Steps to reproduce the behavior:

Run ./configure on a system without bash installed, and with /bin/sh -> dash. This is a small reproducer that shows why it breaks in this case:

$ cat substring.sh
#!/bin/sh
PATH_TO_LUA=/some/path
#echo ${PATH_TO_LUA:0:1}
$ ./substring.sh
./substring.sh: 3: Bad substitution

Expected behavior
The ./configure script completes without errors or gives a clear error about dash-only not being supported.

Desktop (please complete the following information):

• OS: Linux
• Linux distribution: Gentoo/Ubuntu
• Lmod Version: 8.7.55

[bedroge]

I tried to change the lines that do not work for dash, and then the configure script does complete. The make install still failed because the proj_mgmt/convert_mode.sh and proj_mgmt/DATE_cmd.sh had a /bin/bash shebang, but changing that to /bin/sh solved it. The installation completed without issues and seems to work fine.

[bedroge]

This is a full diff of my configure (where I chose a printf solution to get the first character, https://stackoverflow.com/questions/27791251/get-first-character-of-a-string-shell has quite a few other ones) and the original one:

# diff configure.orig configure.new
4438c4438
<     if test ${PATH_TO_LUA:0:1} != '/' ; then
---
>     if test $(printf %.1s "$PATH_TO_LUA") != '/' ; then
4462c4462
<     if test ${PATH_TO_LUAC:0:1} != '/' ; then
---
>     if test $(printf %.1s "$PATH_TO_LUAC") != '/' ; then
4738c4738
< if test $MODE == "user_default" ; then
---
> if test $MODE = "user_default" ; then
5405,5406c5405
< 
< if ! command -v $PATH_TO_LUA &>/dev/null ; then
---
> if ! (command -v $PATH_TO_LUA >/dev/null 2>&1) ; then
5413c5412
<    if test "${PATH_TO_LUA:0:1}" = '/' -o "${PATH_TO_LUAC:0:1}" = '/'; then
---
>    if test $(printf %.1s "$PATH_TO_LUA") = '/' -o $(printf %.1s "$PATH_TO_LUA") = '/'; then
5423c5422
<    if test "${PATH_TO_LUA:0:1}" = '/' -a "${PATH_TO_LUAC:0:1}" != '/'; then
---
>    if test $(printf %.1s "$PATH_TO_LUA") = '/' -a $(printf %.1s "$PATH_TO_LUA") != '/'; then
5426c5425
<    if test "${PATH_TO_LUA:0:1}" != '/' -a "${PATH_TO_LUAC:0:1}" = '/'; then
---
>    if test $(printf %.1s "$PATH_TO_LUA") != '/' -a $(printf %.1s "$PATH_TO_LUA") = '/'; then
5436c5435
< if ! command -v $PATH_TO_LUA &>/dev/null ; then
---
> if ! (command -v $PATH_TO_LUA >/dev/null 2>&1) ; then
5442c5441
< if test ${PATH_TO_LUA:0:1} != '/' ; then
---
> if test $(printf %.1s "$PATH_TO_LUA") != '/' ; then
5491c5490
< if ! command -v $PATH_TO_LUAC &>/dev/null ; then
---
> if ! (command -v $PATH_TO_LUAC >/dev/null 2>&1) ; then
5497c5496
< if test "${PATH_TO_LUAC:0:1}" != '/' ; then
---
> if test $(printf %.1s "$PATH_TO_LUA") != '/' ; then
5592c5591
<   if ! $PATH_TO_LUA $ac_confdir/proj_mgmt/luaModuleAvailable $i 2> /dev/null ; then
---
>   if ! ($PATH_TO_LUA $ac_confdir/proj_mgmt/luaModuleAvailable $i >/dev/null) ; then

[rtmclay]

I have taken your diffs and merged them into Lmod. I have tested the changes as best I can by forcing the configure step to use dash. Also, I have created the Lmod branch IS741-dash and pushed it to github. Please test this branch to see if it works for you and let me know the results thru this issue. Thanks very much for creating the issue and the fix you provided!

[bedroge]

Thanks for looking into this! I gave it a try, but it failed with ERROR: Path to lua must be absolute!. It looks like this is because the double quotes on lines https://github.com/TACC/Lmod/blob/IS741-dash/configure#L4438 and https://github.com/TACC/Lmod/blob/IS741-dash/configure#L4462 are placed incorrectly. Changing it to printf %.1s "$PATH_TO_LUA" and printf %.1s "$PATH_TO_LUAC" solved the issue for me.

[rtmclay]

I have updated the IS741-dash branch with your fixes. Please test this updated version in branch IS741-dash so that this can be merged into the main branch. Thanks for reporting this bug!

[bedroge]

Thanks! I've tested it in a Ubuntu container (after running apt remove bash), and now it works like a charm:

# git clone https://github.com/TACC/lmod
Cloning into 'lmod'...
remote: Enumerating objects: 61924, done.
remote: Counting objects: 100% (2954/2954), done.
remote: Compressing objects: 100% (398/398), done.
remote: Total 61924 (delta 2620), reused 2594 (delta 2546), pack-reused 58970 (from 2)
Receiving objects: 100% (61924/61924), 36.10 MiB | 8.26 MiB/s, done.
Resolving deltas: 100% (41443/41443), done.

# cd lmod
# git checkout IS741-dash
Branch 'IS741-dash' set up to track remote branch 'IS741-dash' from 'origin'.
Switched to a new branch 'IS741-dash'

# ./configure --with-lua=/usr/bin/lua5.1 --with-luac=/usr/bin/luac5.1
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether gcc accepts -g... yes
checking for gcc option to enable C11 features... none needed
DUPLICATE_PATHS=no
LMOD_ALLOW_ROOT_USE=yes
SITE_MSG_FILE=<empty>
LMOD_OVERRIDE_LANG=<empty>
CASE_INDEPENDENT_SORTING=no
COLORIZE=yes
DYNAMIC_SPIDER_CACHE=yes
HIDDEN_ITALIC=no
EXACT_MATCH=no
MODE=user_default
LMOD_CONFIG_DIR=/etc/lmod
AUTO_SWAP=yes
EXPORT_MODULE=yes
REDIRECT=no
MODULEPATH_ROOT=NONE/modulefiles
ANCIENT=86400
SHORT_TIME=2
SPIDER_CACHE_DIRS=
SPIDER_CACHE_DESCRIPT_FN=
UPDATE_SYSTEM_FN=
PREPEND_BLOCK=normal
SITE_NAME=<empty>
SYSHOST=<empty>
ALLOW_TCL_MFILES=yes
DISABLE_NAME_AUTOSWAP=no
MPATH_AVAIL=no
SETTARG=no
IGNORE_DIRS=.svn,.git,.hg,.bzr
PIN_VERSIONS=no
LMOD_DOWNSTREAM_CONFLICTS=no
MODULES_AUTO_HANDLING=no
TMOD_PATH_RULE=no
TMOD_FIND_FIRST=no
LUA_INCLUDE=no
CACHED_LOADS=no
TERSE_DECORATIONS=yes
EXTENDED_DEFAULT=yes
SUPPORT_KSH=no
MODULEPATH_INIT=@PKG@/init/.modulespath
SITE_CONTROLLED_PREFIX=no
USE_BUILT_IN_PKGS=no
USE_DOT_CONFIG_DIR_ONLY=no
SILENCE_SHELL_DEBUGGING=yes
FAST_TCL_INTERP=yes
NEARLY_FORBIDDEN_DAYS=14
MODULE_AUTO_HANDLING=no
AVAIL_EXTENSIONS=yes
PATH_TO_LUA=/usr/bin/lua5.1
PATH_TO_LUAC=/usr/bin/luac5.1
LUA_SUFFIX=no
checking for pkg-config... /usr/bin/pkg-config
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for tcl.h... yes
checking for library containing Tcl_CreateInterp... -ltcl
checking for ps... /usr/bin/ps
checking for readlink... /usr/bin/readlink
checking for gexpr... no
checking for expr... /usr/bin/expr
checking for gbasename... no
checking for basename... /usr/bin/basename
checking for bc... /usr/bin/bc
checking for pod2man... /usr/bin/pod2man
checking for git... /usr/bin/git
checking for less... /usr/bin/less
checking for sha1sum... /usr/bin/sha1sum
checking for tclsh... /usr/bin/tclsh
checking for gtr... no
checking for tr... /usr/bin/tr
checking for gls... no
checking for ls... /usr/bin/ls
/usr/bin/lua5.1
/usr/bin/lua5.1
/usr/bin/luac5.1
checking for valid Lua version... 5.1
checking for lua modules: posix... yes
checking for lua modules: lfs... Too old, using providing version
checking for lua modules: term... yes
checking for pkg-config... (cached) /usr/bin/pkg-config
checking for zsh... no
configure: creating ./config.status
config.status: creating makefile

----------------------------------- SUMMARY ----------------------------------

Package version............................................. : Lmod-8.7.55
Package version (git) ...................................... : 8.7.55-6-g3ae4a71c

LUA_INCLUDE................................................. : /usr/include/lua5.1
Lua executable.............................................. : /usr/bin/lua5.1
Luac executable............................................. : /usr/bin/luac5.1
User Controlled Prefix...................................... : no
Prefix...................................................... : /usr/local
Actual Install dir.......................................... : /usr/local/lmod/8.7.55
Lmod config dir............................................. : /etc/lmod

MODULEPATH_ROOT............................................. : NONE/modulefiles
Wait (s) before rebuilding cache............................ : 86400
Allow Duplicate Paths....................................... : no
Do not save Cache if build time < .......................... : 2
SPIDER_CACHE_DIRS........................................... :
Prepending multiple dirs (NORMAL / REVERSED)................ : normal
Colorized output supported.................................. : yes
File that is touched when system is updated................. :
Allow duplicate entry in PATHs.............................. : no
Allow tcl modulefiles....................................... : yes
ZSH Tab Completion Functions Site Directory................. :
Full Settarg support........................................ : no
Have lua-term............................................... : yes
Have luafilesystem.......................................... :
Support Auto Swap when using families....................... : yes
Export the module shell function in Bash.................... : yes
Disable same name autoswapping.............................. : no
Use Spider Cache on Loads................................... : no
Pager used inside Lmod...................................... : /usr/bin/less
System LD_PRELOAD........................................... :
System LD_LIBRARY_PATH...................................... :
Hashsum program used........................................ : /usr/bin/sha1sum
Site Name................................................... : <empty>
SYSHOST..................................................... : <empty>
Site Message file........................................... : <empty>
Override $LANG Language for error etc....................... : <empty>
Which LuaFileSystem is being used........................... : Built-in
Use italic instead of faint for hidden modules.............. : no
If path entry is already there then don't prepend........... : no
Use Tmod Find First rule instead of Find Best for defaults.. : no
MODULEPATH Initial file..................................... : @PKG@/init/.modulespath
Use built-in lua packages instead of system provided pkgs... : no
Silence shell debugging output for bash/zsh................. : yes
Allow root to use Lmod...................................... : yes
Support KSH................................................. : no
MODE........................................................ : user_default
Use the fast TCL interpreter................................ : yes
Modules Auto Handling....................................... : no
LMOD_CONFIG_DIR............................................. : /etc/lmod
Use ~/.config/lmod directory only........................... : no
Display Extensions w/ module avail.......................... : yes
Dynamic Spider Cache support................................ : yes
Module Conflicts remembered................................. : no
Allow terse listing to be decorated......................... : yes
Allow for extended default.(ml intel/17 #-> intel/17.0.4)... : yes

------------------------------------------------------------------------------

******************************************************************************

Lmod overwrites the env var BASH_ENV to make the module command available in
bash scripts. If your site does not set BASH_ENV then you can ignore the
comments below.

If your site already uses BASH_ENV to point to a site specific script, please
consider sourcing Lmod's init/bash from your site's file.

BASH_ENV is defined both in:
   /usr/local/lmod/8.7.55/init/profile
   /usr/local/lmod/8.7.55/init/cshrc

******************************************************************************


******************************************************************************

Lmod is exporting the module command for Bash users. Some sites may have some
problems. First:

   0.  Make sure that all your machines have shellshock bash patch.

If that does not fix things then you have two choices:

   1.  You configure Lmod not to export the module command.
   2.  You can filter out the exported functions in the users environment
       during job submission

If there is a way to do step 2, please try to do so. Otherwise do step 1.
The advantage of exporting the module command is that it is defined in
/bin/sh scripts. Because Lmod defines BASH_ENV to point to init/bash,
it will be defined for /bin/bash scripts.  The trouble is that users
have to remember to put #!/bin/bash at the first line of their shell
script to make that work. We used to see tickets where users would
submit jobs as /bin/sh scripts and wonder why the module command did
not work.

******************************************************************************


Configure complete, Now do:

    $ make install       # -> A complete install
or
    $ make pre-install   # -> Install everything but the symbolic link

# 

# make install

<lots of output>
done
make[2]: Leaving directory '/lmod'
make[1]: Leaving directory '/lmod'
rm -f /usr/local/lmod/lmod
ln -s 8.7.55 /usr/local/lmod/lmod

[rtmclay]

Great! Thanks for working with me on this issue. I'm closing this issue. Feel free to re-open this issue if necessary.
Thanks!