Packaging improvements & upgrade to rebar3.18

[hmmr]

This is a remake of #1097 (which see for PR description and testing routine), now against develop, with some improvements and fixups:

• The version of rebar3_cuttlefish in 1097 was too eagerly upgraded, resulting in extension scripts not being used. This has been fixed.
• For the sake of OSX build, I am including these workarounds. I'll be more than happy to yank them from this PR (ideally with a hint on what exactly is wrong with erlang and/or rebar on OSX).
• Previously attempted patch to warn users about deprecation of hyphenated forms has been scrapped.
• The Alpine Linux build is made from this branch.

[martinsumner]

With make devrel we now get full/absolute file paths in riak.conf to all the platform_*_dir entries.

platform_etc_dir = /Users/username/riak/dev/dev1/riak/etc

Rather than the relative

platform_etc_dir = ./etc

When we then install the dev environment into riak_test, the script copies each dev* directory across into the test area ... but now these references are wrong. So the riak_test tests don't work, when they try and change the config, or reset any changes.

would it be possible to retain the old behaviour of having relative references to all the platform dirs in rel and devrel?

[martinsumner]

If I overcome the issue by sed'ing the riak.conf before copying, I still have a problem. riak_test cannot appear to stop the nodes it starts.

I think it might also be a path related problem. It doesn't look like the sys.config and vm.args are being created in the correct location off of the riak.conf, so the node is started with a name that isn't in the vm.args (which is unchanged from the default), and so although the riak process is up and running, it can't be stopped/pinged from the command line - as nay other command line requests read the vm.args without the updated name.

[martinsumner]

I've checked around, and the folder where riak is built is hard-coded as an absolute path in lots of places. So, as it stands we lose the ability to copy this around, as required by riak_test.

This may well have been done for good reason, perhaps it is necessary to make other things work.

If we can't change this, it may be possible to change the way we set-up riak_test so that we build directly to the test folder rather than first locally and then copying.

[hmmr]

With 39f0ecf, devrels are relocatable again.

[martinsumner]

Going through some riak_test failures. verify_staged_clustering fails. This checks the command line method for staging clustering. there appear to be rogue -n's being added to the lists of parameters:

MartinWork:riak martinsumner$ dev/dev2/riak/bin/riak admin cluster join [email protected]
erl_call: Failed to parse arguments,
see the documentation for allowed term types.
Arguments: -n [[ -n "[email protected]" ]]
MartinWork:riak martinsumner$ dev/dev2/riak/bin/riak admin cluster status
erl_call: Failed to parse arguments,
see the documentation for allowed term types.
Arguments: -n [[ -n "riak-admin cluster status" ]]

Can confirm this on both OTP 22 (where the error is slightly different because of nodetool, but it still fails for the same reason), and OTP 24. If I call riak-admin cluster status rather than riak admin cluster status, everything works as expected.

[hmmr]

There was a bash issue with preparing an erl_call sommand line with multiple parameters, which is fixed in b2a3a5a.

However, it looks like your shell (something other than bash?) has a version of echo that doesn't understand -n. What environment are you running your tests in?

[martinsumner]

I'm running locally on OSX (Mojave): echo $0 returns -bash

[hmmr]

Changes now merged into https://github.com/basho/riak/commits/mas-hmmr-updated-d32, to be refined/supplemented and taken further. Closing.

[hmmr]

Becomes part of #1114.