ci: add suitespec to GitLab [backport 2.15]

[erikayasuda]

Backport f7c102e from #10877 to 2.17.

We port the suitespec machinery to GitLab. The current implementation uses generated child pipelines, with the tests YAML configuration automatically generated to include the required jobs based on the files listed in the PR. The job information previously contained in the config.templ.yaml file has been migrated to the jobspec.yml configuration. The jobspec can be "modular", in the sense that it can be split into multiple jobspec.yml file within the tests/ subtree (e.g. tests/jobspec.yml, tests/debugging/jobspec.yml, ...)

Checklist

• PR author has checked that all the criteria below are met
• The PR description includes an overview of the change
• The PR description articulates the motivation for the change
• The change includes tests OR the PR description describes a testing strategy
• The PR description notes risks associated with the change, if any
• Newly-added code is easy to change
• The change follows the library release note guidelines
• The change includes or references documentation updates if necessary
• Backport labels are set (if applicable)

Reviewer Checklist

• Reviewer has checked that all the criteria below are met
• Title is accurate
• All changes are related to the pull request's stated goal
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Newly-added code is easy to change
• Release note makes sense to a user of the library
• If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[github-actions]

CODEOWNERS have been resolved as:

.gitlab/testrunner.yml                                                  @DataDog/python-guild @DataDog/apm-core-python
.riot/requirements/196755b.txt                                          @DataDog/apm-python
scripts/gen_gitlab_config.py                                            @DataDog/apm-core-python
tests/README.md                                                         @DataDog/python-guild @DataDog/apm-core-python
tests/appsec/jobspec.yml                                                @DataDog/asm-python
tests/ci_visibility/jobspec.yml                                         @DataDog/ci-app-libraries
tests/contrib/jobspec.yml                                               @DataDog/apm-core-python @DataDog/apm-idm-python
tests/debugging/jobspec.yml                                             @DataDog/debugger-python
tests/jobspec.yml                                                       @DataDog/apm-core-python
tests/llmobs/jobspec.yml                                                @DataDog/ml-observability
.circleci/config.templ.yml                                              @DataDog/python-guild @DataDog/apm-core-python
.github/workflows/requirements-locks.yml                                @DataDog/python-guild @DataDog/apm-core-python
.gitignore                                                              @DataDog/apm-core-python
.gitlab-ci.yml                                                          @DataDog/apm-core-python
.gitlab/package.yml                                                     @DataDog/python-guild @DataDog/apm-core-python
.gitlab/tests.yml                                                       @DataDog/python-guild @DataDog/apm-core-python
docker-compose.yml                                                      @DataDog/apm-core-python
riotfile.py                                                             @DataDog/apm-python
scripts/gen_circleci_config.py                                          @DataDog/apm-core-python
scripts/needs_testrun.py                                                @DataDog/apm-core-python
tests/.suitespec.json                                                   @DataDog/python-guild @DataDog/apm-core-python

[datadog-dd-trace-py-rkomorn]

Datadog Report

Branch report: backport-10877-to-2.15
Commit report: 9588f25
Test service: dd-trace-py

✅ 0 Failed, 592 Passed, 694 Skipped, 19m 26.49s Total duration (15m 36.38s time saved)

[pr-commenter]

Benchmarks

Benchmark execution time: 2024-11-25 17:30:19

Comparing candidate commit 9588f25 in PR branch backport-10877-to-2.15 with baseline commit dad1ed0 in branch 2.15.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 372 metrics, 52 unstable metrics.