[8.10](backport #3477) Refactor and fix all upgrade integration tests

[mergify]

This is an automatic backport of pull request #3477 done by Mergify.
Cherry-pick of 6201e19 has failed:

On branch mergify/bp/8.10/pr-3477
Your branch is up to date with 'origin/8.10'.

You are currently cherry-picking commit 6201e19a5b.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   magefile.go
	modified:   pkg/testing/fetcher_artifact.go
	modified:   pkg/testing/fixture.go
	modified:   pkg/testing/fixture_install.go
	modified:   pkg/testing/tools/agents.go
	new file:   testing/integration/upgrade_broken_package_test.go
	new file:   testing/integration/upgrade_downgrade_test.go
	new file:   testing/integration/upgrade_fleet_test.go
	new file:   testing/integration/upgrade_gpg_test.go
	new file:   testing/integration/upgrade_rollback_test.go
	new file:   testing/integration/upgrade_standalone_inprogress.go
	new file:   testing/integration/upgrade_standalone_retry_test.go
	new file:   testing/integration/upgrade_standalone_test.go
	new file:   testing/integration/upgrade_uninstall_test.go
	new file:   testing/upgradetest/logger.go
	new file:   testing/upgradetest/upgrader.go
	new file:   testing/upgradetest/versions.go
	new file:   testing/upgradetest/watcher.go

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	both modified:   sonar-project.properties
	deleted by them: testing/integration/upgrade_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/github/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

---

Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

• @Mergifyio refresh will re-evaluate the rules
• @Mergifyio rebase will rebase this PR on its base branch
• @Mergifyio update will merge the base branch into this PR
• @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

• look at your merge queues
• generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.com

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-10-05T15:04:05.645+0000

• Duration: 28 min 7 sec

Test stats 🧪

Test	Results
Failed	0
Passed	6277
Skipped	55
Total	6332

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages.

• run integration tests : Run the Elastic Agent Integration tests.

• run end-to-end tests : Generate the packages and run the E2E Tests.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b mergify/bp/8.10/pr-3477 upstream/mergify/bp/8.10/pr-3477
git merge upstream/8.10
git push upstream mergify/bp/8.10/pr-3477

[cmacknz]

@blakerouse we need this to resolve some upgrade test flakiness in 8.10.

Example: https://buildkite.com/elastic/elastic-agent/builds/3723#018afb85-d524-49bc-906d-5668a05c5061

[elasticmachine]

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	98.765% (80/81)	👍
Files	67.596% (194/287)	👎 -0.236
Classes	66.234% (357/539)	👎 -0.247
Methods	53.033% (1119/2110)	👎 -0.078
Lines	38.554% (12725/33006)	👎 -0.01
Conditionals	100.0% (0/0)	💚