feat: upstream to youtalk (#23)

* chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719) * refactor(ci): reuse `load-env.yaml` to load env (autowarefoundation#4734) * reuse load-env.yaml to load env * update base_image Signed-off-by: Yutaka Kondo <[email protected]> * fix base_image Signed-off-by: Yutaka Kondo <[email protected]> --------- Signed-off-by: Yutaka Kondo <[email protected]> * refactor(docker): introduce `src-imported` stage (autowarefoundation#4712) * introduce src-imported stage Signed-off-by: Yutaka Kondo <[email protected]> * empty Signed-off-by: Yutaka Kondo <[email protected]> * generate install package lists * append || true Signed-off-by: Yutaka Kondo <[email protected]> * cat packages Signed-off-by: Yutaka Kondo <[email protected]> * chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719) * wip Signed-off-by: Yutaka Kondo <[email protected]> * wip Signed-off-by: Yutaka Kondo <[email protected]> * Update Dockerfile * Update Dockerfile * install pacmod * rename * Revert "install pacmod" This reverts commit ebff10f. * install pacmod3 * register pamod3 repos * ignore hadolint * revert * run set-dev-env.sh * pip install * copy * rosdep update --------- Signed-off-by: Yutaka Kondo <[email protected]> * feat(docker): install minimum packages in `base` stage (autowarefoundation#4751) * install ssh only and copy only once Signed-off-by: Yutaka Kondo <[email protected]> * install gosu Signed-off-by: Yutaka Kondo <[email protected]> * fix dir Signed-off-by: Yutaka Kondo <[email protected]> * revert ansible Signed-off-by: Yutaka Kondo <[email protected]> --------- Signed-off-by: Yutaka Kondo <[email protected]> * chore: add discussion templates for general category (autowarefoundation#4764) Signed-off-by: Ryohsuke Mitsudome <[email protected]> * feat(ci,docker): change `vcs import` to `COPY src` (autowarefoundation#4738) * introduce src-imported stage Signed-off-by: Yutaka Kondo <[email protected]> * empty Signed-off-by: Yutaka Kondo <[email protected]> * generate install package lists * append || true Signed-off-by: Yutaka Kondo <[email protected]> * cat packages Signed-off-by: Yutaka Kondo <[email protected]> * chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719) * wip Signed-off-by: Yutaka Kondo <[email protected]> * wip Signed-off-by: Yutaka Kondo <[email protected]> * Update Dockerfile * Update Dockerfile * install pacmod * rename * Revert "install pacmod" This reverts commit ebff10f. * install pacmod3 * register pamod3 repos * ignore hadolint * revert * run set-dev-env.sh * pip install * copy * rosdep update * vcs import outside dockerfile Signed-off-by: Yutaka Kondo <[email protected]> * refine order and remove /autoware/src at last * fix hadolint * remove tar files Signed-off-by: Yutaka Kondo <[email protected]> * shell: bash Signed-off-by: Yutaka Kondo <[email protected]> --------- Signed-off-by: Yutaka Kondo <[email protected]> * feat(docker,ansible): download artifacts at runtime (autowarefoundation#4771) * feat(ci): enable Docker build cache to 5 times faster container build (autowarefoundation#4730) * introduce src-imported stage Signed-off-by: Yutaka Kondo <[email protected]> * empty Signed-off-by: Yutaka Kondo <[email protected]> * generate install package lists * append || true Signed-off-by: Yutaka Kondo <[email protected]> * cat packages Signed-off-by: Yutaka Kondo <[email protected]> * chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719) * wip Signed-off-by: Yutaka Kondo <[email protected]> * wip Signed-off-by: Yutaka Kondo <[email protected]> * Update Dockerfile * Update Dockerfile * install pacmod * rename * Revert "install pacmod" This reverts commit ebff10f. * install pacmod3 * register pamod3 repos * ignore hadolint * revert * run set-dev-env.sh * pip install * copy * rosdep update * vcs import outside dockerfile Signed-off-by: Yutaka Kondo <[email protected]> * enable docker build cache Signed-off-by: Yutaka Kondo <[email protected]> * empty Signed-off-by: Yutaka Kondo <[email protected]> * add no-cuda/cuda suffix Signed-off-by: Yutaka Kondo <[email protected]> * remove --download-artifacts Signed-off-by: Yutaka Kondo <[email protected]> * remove --download-artifacts Signed-off-by: Yutaka Kondo <[email protected]> * add --data-path option Signed-off-by: Yutaka Kondo <[email protected]> * wip Signed-off-by: Yutaka Kondo <[email protected]> * run download_artifacts Signed-off-by: Yutaka Kondo <[email protected]> * remove todo Signed-off-by: Yutaka Kondo <[email protected]> * revert run Signed-off-by: Yutaka Kondo <[email protected]> --------- Signed-off-by: Yutaka Kondo <[email protected]> * feat(docker): cache `ccache` to the Docker build cache (autowarefoundation#4772) * fix: add download url of yolox with semantic segmentation header (autowarefoundation#4012) Signed-off-by: badai-nguyen <[email protected]> * chore: add error handling for held packages during rmw installation (autowarefoundation#4781) * chore: add error handling for held packages during apt installation Signed-off-by: Naophis <[email protected]> * feat: Add check for held ROS packages during installation Signed-off-by: Naophis <[email protected]> --------- Signed-off-by: Naophis <[email protected]> * refactor(docker,ci): rename `autoware-openadk` to `autoware` and `adkit` to `openadkit` (autowarefoundation#4785) * rename adkit to openadkit Signed-off-by: Yutaka Kondo <[email protected]> * rename autoware-openadkit to autoware Signed-off-by: Yutaka Kondo <[email protected]> * fix hcl path Signed-off-by: Yutaka Kondo <[email protected]> --------- Signed-off-by: Yutaka Kondo <[email protected]> * feat(ci): disable `Upload Artifacts` on `docker-build-and-push` action (autowarefoundation#4789) * remove upload artifact Signed-off-by: Yutaka Kondo <[email protected]> * remove build and push Signed-off-by: Yutaka Kondo <[email protected]> --------- Signed-off-by: Yutaka Kondo <[email protected]> * feat(docker): cache apt archives to the Docker build cache (autowarefoundation#4778) * mount apt cache Signed-off-by: Yutaka Kondo <[email protected]> * reorder Signed-off-by: Yutaka Kondo <[email protected]> * remove upload artifact Signed-off-by: Yutaka Kondo <[email protected]> * remove build and push Signed-off-by: Yutaka Kondo <[email protected]> --------- Signed-off-by: Yutaka Kondo <[email protected]> * chore: update code owners (autowarefoundation#4797) Signed-off-by: M. Fatih Cırıt <[email protected]> * ci(pre-commit-ansible): autoupdate (autowarefoundation#4676) Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions <[email protected]> * ci: pre-commit autoupdate (autowarefoundation#4670) [pre-commit.ci] pre-commit autoupdate updates: - [github.com/igorshubovych/markdownlint-cli: v0.39.0 → v0.41.0](igorshubovych/markdownlint-cli@v0.39.0...v0.41.0) - [github.com/AleksaC/hadolint-py: v2.12.1-beta → v2.12.1b3](AleksaC/hadolint-py@v2.12.1-beta...v2.12.1b3) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * ci(pre-commit): autoupdate (autowarefoundation#4662) Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions <[email protected]> * ci(pre-commit-optional): autoupdate (autowarefoundation#4766) Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions <[email protected]> * fix(ci): run `vsc import` on `build-main` workflows (autowarefoundation#4799) Signed-off-by: Yutaka Kondo <[email protected]> --------- Signed-off-by: Yutaka Kondo <[email protected]> Signed-off-by: Ryohsuke Mitsudome <[email protected]> Signed-off-by: badai-nguyen <[email protected]> Signed-off-by: Naophis <[email protected]> Signed-off-by: M. Fatih Cırıt <[email protected]> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Ryohsuke Mitsudome <[email protected]> Co-authored-by: badai nguyen <[email protected]> Co-authored-by: Naophis <[email protected]> Co-authored-by: M. Fatih Cırıt <[email protected]> Co-authored-by: awf-autoware-bot[bot] <94889083+awf-autoware-bot[bot]@users.noreply.github.com> Co-authored-by: github-actions <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
youtalk · May 31, 2024 · 7d866a7 · 7d866a7
1 parent 7b30bf4
commit 7d866a7
Show file tree

Hide file tree

Showing 10 changed files with 92 additions and 5 deletions.
diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
@@ -1 +1,6 @@
 * [email protected] [email protected]
+
+.devcontainer/** [email protected] [email protected]
+.github/** [email protected] [email protected]
+ansible/** [email protected] [email protected]
+docker/** [email protected] [email protected]
diff --git a/.github/DISCUSSION_TEMPLATE/general.yaml b/.github/DISCUSSION_TEMPLATE/general.yaml
@@ -0,0 +1,13 @@
+body:
+  - type: checkboxes
+    attributes:
+      label: Code of Conduct
+      options:
+        - label: I have read [CODE OF CONDUCT](https://github.com/autowarefoundation/autoware/blob/main/CODE_OF_CONDUCT.md) and [Support Guidelines](https://autowarefoundation.github.io/autoware-documentation/main/support/support-guidelines/#github-discussions) before creating this Discussion post.
+          required: true
+  - type: textarea
+    attributes:
+      label: Contents
+      description: Write your contents here
+    validations:
+      required: true
diff --git a/.github/workflows/build-main-self-hosted.yaml b/.github/workflows/build-main-self-hosted.yaml
@@ -44,9 +44,25 @@ jobs:
       - name: Check out repository
         uses: actions/checkout@v4
 
+      - name: Install vcstool
+        run: |
+          sudo apt-get -y update
+          sudo apt-get -y install python3-pip
+          pip install --no-cache-dir vcstool
+
+      - name: Set git config
+        uses: autowarefoundation/autoware-github-actions/set-git-config@v1
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+
       - name: Free disk space
         uses: ./.github/actions/free-disk-space
 
+      - name: Run vcs import
+        run: |
+          mkdir src
+          vcs import src < autoware.repos
+
       - name: Build 'autoware-universe'
         uses: ./.github/actions/docker-build-and-push
         with:

diff --git a/.github/workflows/build-main.yaml b/.github/workflows/build-main.yaml
@@ -39,9 +39,25 @@ jobs:
       - name: Check out repository
         uses: actions/checkout@v4
 
+      - name: Install vcstool
+        run: |
+          sudo apt-get -y update
+          sudo apt-get -y install python3-pip
+          pip install --no-cache-dir vcstool
+
+      - name: Set git config
+        uses: autowarefoundation/autoware-github-actions/set-git-config@v1
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+
       - name: Free disk space
         uses: ./.github/actions/free-disk-space
 
+      - name: Run vcs import
+        run: |
+          mkdir src
+          vcs import src < autoware.repos
+
       - name: Build 'autoware-universe'
         uses: ./.github/actions/docker-build-and-push
         with:

diff --git a/.pre-commit-config-ansible.yaml b/.pre-commit-config-ansible.yaml
@@ -1,6 +1,6 @@
 repos:
   - repo: https://github.com/ansible/ansible-lint.git
-    rev: v24.2.2
+    rev: v24.5.0
     hooks:
       - id: ansible-lint
         additional_dependencies:

diff --git a/.pre-commit-config-optional.yaml b/.pre-commit-config-optional.yaml
@@ -1,6 +1,6 @@
 repos:
   - repo: https://github.com/tcort/markdown-link-check
-    rev: v3.12.1
+    rev: v3.12.2
     hooks:
       - id: markdown-link-check
         args: [--quiet, --config=.markdown-link-check.json]
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -17,7 +17,7 @@ repos:
         args: [--markdown-linebreak-ext=md]
 
   - repo: https://github.com/igorshubovych/markdownlint-cli
-    rev: v0.39.0
+    rev: v0.41.0
     hooks:
       - id: markdownlint
         args: [-c, .markdownlint.yaml, --fix]
@@ -44,7 +44,7 @@ repos:
         args: [-w, -s, -i=4]
 
   - repo: https://github.com/AleksaC/hadolint-py
-    rev: v2.12.1-beta
+    rev: v2.12.1b3
     hooks:
       - id: hadolint
 

diff --git a/ansible/roles/artifacts/tasks/main.yaml b/ansible/roles/artifacts/tasks/main.yaml
@@ -213,6 +213,30 @@
     mode: "644"
     checksum: sha256:3540a365bfd6d8afb1b5d8df4ec47f82cb984760d3270c9b41dbbb3422d09a0c
 
+- name: Download tensorrt_yolox/yolox-sPlus-opt-pseudoV2-T4-960x960-T4-seg16cls.onnx
+  become: true
+  ansible.builtin.get_url:
+    url: https://awf.ml.dev.web.auto/perception/models/object_detection_semseg_yolox_s/v1/yolox-sPlus-opt-pseudoV2-T4-960x960-T4-seg16cls.onnx
+    dest: "{{ data_dir }}/tensorrt_yolox/yolox-sPlus-opt-pseudoV2-T4-960x960-T4-seg16cls.onnx"
+    mode: "644"
+    checksum: sha256:73b3812432cedf65cebf02ca4cb630542fc3b1671c4c0fbf7cee50fa38e416a8
+
+- name: Download tensorrt_yolox/yolox-sPlus-opt-pseudoV2-T4-960x960-T4-seg16cls.EntropyV2-calibration.table
+  become: true
+  ansible.builtin.get_url:
+    url: https://awf.ml.dev.web.auto/perception/models/object_detection_semseg_yolox_s/v1/yolox-sPlus-opt-pseudoV2-T4-960x960-T4-seg16cls.EntropyV2-calibration.table
+    dest: "{{ data_dir }}/tensorrt_yolox/yolox-sPlus-opt-pseudoV2-T4-960x960-T4-seg16cls.EntropyV2-calibration.table"
+    mode: "644"
+    checksum: sha256:28cd6524d4bcdb2809592a225d28330433e58dc02c92169ea555b44c1a51a584
+
+- name: Download tensorrt_yolox/semseg_color_map.csv
+  become: true
+  ansible.builtin.get_url:
+    url: https://awf.ml.dev.web.auto/perception/models/object_detection_semseg_yolox_s/v1/semseg_color_map.csv
+    dest: "{{ data_dir }}/tensorrt_yolox/semseg_color_map.csv"
+    mode: "644"
+    checksum: sha256:3d93ca05f31b63424d7d7246a01a2365953705a0ed3323ba5b6fddd744a4bfea
+
 # traffic_light_classifier
 - name: Create traffic_light_classifier directory inside {{ data_dir }}
   ansible.builtin.file:

diff --git a/ansible/roles/rmw_implementation/tasks/main.yaml b/ansible/roles/rmw_implementation/tasks/main.yaml
@@ -3,12 +3,25 @@
   register: rmw_implementation__dash_case_rmw_implementation
   changed_when: false
 
+- name: Hold check of ros-{{ rosdistro + '-' + rmw_implementation__dash_case_rmw_implementation.stdout }}
+  ansible.builtin.command: apt-mark showhold
+  register: held_ros_packages
+  changed_when: false
+
 - name: Install ros-{{ rosdistro + '-' + rmw_implementation__dash_case_rmw_implementation.stdout }}
   become: true
   ansible.builtin.apt:
     name: ros-{{ rosdistro }}-{{ rmw_implementation__dash_case_rmw_implementation.stdout }}
     state: latest
     update_cache: true
+  when: "'ros-' + rosdistro + '-' + rmw_implementation__dash_case_rmw_implementation.stdout not in held_ros_packages.stdout"
+  register: install_result
+  failed_when: false
+
+- name: Display warning if ROS 2 RMW package is held
+  ansible.builtin.debug:
+    msg: ROS 2 RMW package 'ros-{{ rosdistro + '-' + rmw_implementation__dash_case_rmw_implementation.stdout }}' is apt-mark hold. Skipping installation.
+  when: not install_result.changed
 
 - name: Add RMW_IMPLEMENTATION to .bashrc
   ansible.builtin.lineinfile:

diff --git a/docker/run.sh b/docker/run.sh
@@ -101,7 +101,7 @@ set_variables() {
         exit 1
     fi
 
-    # Mount data path if provided
+    # Mount data path
     DATA="-v ${DATA_PATH}:/autoware_data:ro"
 
     # Mount map path if provided