Test upgrade from previous version to latest version in CI

[kpollich]

We currently don't have automated test coverage in CI for integration upgrades. It could be helpful to run automated tests against the stack for each package that assert an upgrade from the previous published version to the latest version is successful. This would catch cases where both versions of a package are considered valid, but backwards compatibility is broken between versions.

Relates to #579

[andrewkroh]

I came here to open a new issue, but found this one already opened. Here is what I was going to write.

Test package upgrades

There are bugs that could be detected if we could make elastic-package execute the package upgrade process. Roughly what I would like to see tested is this scenario:

1. Install the latest version from the EPR.
2. Do something to create indices. I think we need indices to catch mapping upgrade problems.
3. Install (upgrade) to package present in the pull request.
4. Run the system test.

I think this would help catch mapping and transform upgrade problems.

Related

• [Okta]: Update 2.11.0 - > 2.12.0 fails integrations#10972
• [Windows Integration]: Error installing windows 1.47.0 integrations#10750
• [TSDB] k8 package upgrade failure from 8.6.2 to 8.8.1 integrations#6554

[jsoriano]

This could be done in the context of the compatibility tests proposed in #579.

[cmacknz]

+1, either as part of #579 or this issue we need this. We have shipped bugs this test would have caught.