398 client runner

[gianfra-t]

Addition of an implementation of the runner client taken from interlay clients.

In summary, the runner fetches periodically for the on-chain information about the vault client, both download URI and checksum. It will then proceed to download and execute it, also checking for new releases and handling them.

The only difference on this implementation is:

• The runner client will now verify checksum of the vault client once it is downloaded for the first time, rejecting the download if it does not match.
• The runner will retry the download if it fails (for instance, if the checksum is incorrect)

[ebma]

Looks good overall, I just left some comments about minor issues.

[gianfra-t]

Thanks for finding this @ebma! I have updated all the remarked changes. On that note, let me know if we should open a separate task to build an integration test for the runner.

[ebma]

On that note, let me know if we should open a separate task to build an integration test for the runner.

How would you suggest we approach this kind of test?

[ebma]

The code looks good overall.

How sure are we that this works as expected though @gianfra-t? You said you tested it locally once and it worked with downloading the binary from a target location and replacing the current vault executable, correct?

[gianfra-t]

The code looks good overall.

How sure are we that this works as expected though @gianfra-t? You said you tested it locally once and it worked with downloading the binary from a target location and replacing the current vault executable, correct?

Yes. I tested locally with a release of the client and also tested updating it on a local chain. It detected the changes and downloaded/started the new one.

[ebma]

I also tested it and can confirm that the runner manages to read the URL stored in the clients-info pallet and downloads the release from there. However, the checksum did not match.
@gianfra-t During testing, how did you calculate the checksum for the clientsInfo::setCurrentClientReleases() extrinsic? Using the 'hash a file' option provided by polkadot.js does not seem to work, or at least the result of that hashing process is not the same as the result of the runner hashing the downloaded binary.

To make it work regardless, I just used the hash that the runner calculates and put that on chain. Afterward, downloading etc. worked. But this is not an ideal solution of course.

[gianfra-t]

I actually had the same problem, the polkadot.js hash did not work. I thought it was a problem with it so I used this tool. Do you think it is a different hashing algorithm/implementation?
I it presents a problem I guess I could try a different hashing library on the runner that matches the one created by polkadot.js

[ebma]

Yes, let's try to use the same hashing algorithm for convenience. I tried to find what polkadot.js uses and I think it is blake2. The UI calls this function and according to this test it uses blake2 by default but can be overridden with a custom definition here. I think we don't specify a custom hasher so let's try hashing with blake2 in the vault client and see if the checksum then matches the one generated by polkadot.js upon file upload.

Update:
I just saw that interlay calculate the checksums as part of their release management and put them into an extra file contained in every release, see eg. sha256sums.txt. We could do the same. Do you have any opinions on this @gianfra-t? Would you prefer if we also keep using sha256 or should we switch to blake2?

[gianfra-t]

I would say It really boils down to the deployment workflow we want.
If we plan to have a release of the client that is automated then I would say stick with this hashing algorithm and do something similar to interlay.

[ebma]

Alright. Let's stick to sha256 and not rely on the hash shown in polkadot.js. It's just a UI and we can use other means, like e.g. the tool you linked to.
So in theory this PR is ready for merge now but I'm hesitant to merge it just yet because the CI is failing.

[gianfra-t]

It's the issue related to stellar update, so let's wait for that to be solved and we can merge those changes.

[ebma]

I'll merge this since the ubuntu CI job passed.