-
Notifications
You must be signed in to change notification settings - Fork 668
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'main' into chore/gnss_poser/add_autoware_prefix
- Loading branch information
Showing
501 changed files
with
35,269 additions
and
5,593 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Validating CODEOWNERS rules …
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
109 changes: 109 additions & 0 deletions
109
.github/actions/build-and-test-differential/action.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,109 @@ | ||
| name: build-and-test-differential | ||
| description: "" | ||
|
|
||
| inputs: | ||
| rosdistro: | ||
| description: "" | ||
| required: true | ||
| container: | ||
| description: "" | ||
| required: true | ||
| container-suffix: | ||
| description: "" | ||
| required: true | ||
| runner: | ||
| description: "" | ||
| required: true | ||
| build-depends-repos: | ||
| description: "" | ||
| required: true | ||
| build-pre-command: | ||
| description: "" | ||
| required: true | ||
| codecov-token: | ||
| description: "" | ||
| required: true | ||
|
|
||
| runs: | ||
| using: composite | ||
| steps: | ||
| - name: Show disk space before the tasks | ||
| run: df -h | ||
| shell: bash | ||
|
|
||
| - name: Show machine specs | ||
| run: lscpu && free -h | ||
| shell: bash | ||
|
|
||
| - name: Remove exec_depend | ||
| uses: autowarefoundation/autoware-github-actions/remove-exec-depend@v1 | ||
|
|
||
| - name: Get modified packages | ||
| id: get-modified-packages | ||
| uses: autowarefoundation/autoware-github-actions/get-modified-packages@v1 | ||
|
|
||
| - name: Create ccache directory | ||
| run: | | ||
| mkdir -p ${CCACHE_DIR} | ||
| du -sh ${CCACHE_DIR} && ccache -s | ||
| shell: bash | ||
|
|
||
| - name: Attempt to restore ccache | ||
| uses: actions/cache/restore@v4 | ||
| with: | ||
| path: | | ||
| /root/.ccache | ||
| key: ccache-main-${{ runner.arch }}-${{ inputs.rosdistro }}-${{ github.event.pull_request.base.sha }} | ||
| restore-keys: | | ||
| ccache-main-${{ runner.arch }}-${{ inputs.rosdistro }}- | ||
| - name: Show ccache stats before build | ||
| run: du -sh ${CCACHE_DIR} && ccache -s | ||
| shell: bash | ||
|
|
||
| - name: Export CUDA state as a variable for adding to cache key | ||
| run: | | ||
| build_type_cuda_state=nocuda | ||
| if [[ "${{ inputs.container-suffix }}" == "-cuda" ]]; then | ||
| build_type_cuda_state=cuda | ||
| fi | ||
| echo "BUILD_TYPE_CUDA_STATE=$build_type_cuda_state" >> "${GITHUB_ENV}" | ||
| echo "::notice::BUILD_TYPE_CUDA_STATE=$build_type_cuda_state" | ||
| shell: bash | ||
|
|
||
| - name: Build | ||
| if: ${{ steps.get-modified-packages.outputs.modified-packages != '' }} | ||
| uses: autowarefoundation/autoware-github-actions/colcon-build@v1 | ||
| with: | ||
| rosdistro: ${{ inputs.rosdistro }} | ||
| target-packages: ${{ steps.get-modified-packages.outputs.modified-packages }} | ||
| build-depends-repos: ${{ inputs.build-depends-repos }} | ||
| cache-key-element: ${{ env.BUILD_TYPE_CUDA_STATE }} | ||
| build-pre-command: ${{ inputs.build-pre-command }} | ||
|
|
||
| - name: Show ccache stats after build | ||
| run: du -sh ${CCACHE_DIR} && ccache -s | ||
| shell: bash | ||
|
|
||
| - name: Test | ||
| id: test | ||
| if: ${{ steps.get-modified-packages.outputs.modified-packages != '' }} | ||
| uses: autowarefoundation/autoware-github-actions/colcon-test@v1 | ||
| with: | ||
| rosdistro: ${{ inputs.rosdistro }} | ||
| target-packages: ${{ steps.get-modified-packages.outputs.modified-packages }} | ||
| build-depends-repos: ${{ inputs.build-depends-repos }} | ||
|
|
||
| - name: Upload coverage to CodeCov | ||
| if: ${{ steps.test.outputs.coverage-report-files != '' }} | ||
| uses: codecov/codecov-action@v4 | ||
| with: | ||
| files: ${{ steps.test.outputs.coverage-report-files }} | ||
| fail_ci_if_error: false | ||
| verbose: true | ||
| flags: differential | ||
| token: ${{ inputs.codecov-token }} | ||
|
|
||
| - name: Show disk space after the tasks | ||
| run: df -h | ||
| shell: bash |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| # Overall CI Infrastructure | ||
|
|
||
| ## Machine Types | ||
|
|
||
| ### Standard GitHub-hosted runners | ||
|
|
||
| - [Documentation](https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories) | ||
|
|
||
| These runners are utilized by the majority of the workflows. | ||
| They are free to use for public repositories, with a concurrency limit of 20 jobs per organization. | ||
|
|
||
| **Listed specs:** | ||
|
|
||
| | vCPU | RAM | Storage (SSD) | | ||
| | ---- | ----- | ------------- | | ||
| | 4 | 16 GB | 14 GB | | ||
|
|
||
| **Note:** While the official documentation lists 14 GB of storage, the actual available storage is approximately 73 GB. | ||
|
|
||
| ### AWS CodeBuild runners | ||
|
|
||
| - [AWS CodeBuild Documentation](https://docs.aws.amazon.com/codebuild/latest/userguide/action-runner.html) | ||
|
|
||
| These runners are employed for workflows that require more resources and are funded by the Autoware Foundation budget. | ||
|
|
||
| **Relevant machine types:** | ||
|
|
||
| | Instance Type | Memory | vCPUs | Price per Minute | | ||
| | --------------- | ------ | ----- | ---------------- | | ||
| | arm1.large | 16 GiB | 8 | $0.015 | | ||
| | general1.medium | 7 GB | 4 | $0.01 | | ||
| | general1.large | 15 GB | 8 | $0.02 | | ||
| | general1.xlarge | 72 GiB | 36 | $0.0798 | | ||
| | gpu1.small | 15 GB | 4 | $0.05 | | ||
|
|
||
| **Sources:** | ||
|
|
||
| - [Compute images supported with the CodeBuild-hosted GitHub Actions runner](https://docs.aws.amazon.com/codebuild/latest/userguide/sample-github-action-runners-update-yaml.images.html) | ||
| - [AWS CodeBuild pricing](https://aws.amazon.com/codebuild/pricing/) | ||
|
|
||
| ## Key workflows and their runners | ||
|
|
||
| | Workflow | Trigger | Runner | | ||
| | ---------------------------------- | --------------------- | -------------- | | ||
| | build-and-test (cuda) | merge to main | general1.large | | ||
| | build-and-test-daily | daily on main | github-std | | ||
| | build-and-test-daily-arm64 | daily on main | arm1.large | | ||
| | build-and-test-differential | PR update | github-std | | ||
| | build-and-test-differential (cuda) | PR update | general1.large | | ||
| | build-and-test-differential-arm64 | PR update (arm label) | arm1.large | | ||
|
|
||
| ## Additional notes | ||
|
|
||
| - We use [`taskset`](https://manpages.ubuntu.com/manpages/jammy/man1/taskset.1.html) from GNU Coreutils to limit the number of cores utilized by build processes. This is done to prevent overloading the self-hosted runners. | ||
| - The number of cores is limited to `vCPU count - 1`. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.