docs: add ansible installation, update manual artifact download proce…

…ss (autowarefoundation#4675)

Signed-off-by: M. Fatih Cırıt <[email protected]>

ansible/README.md

@@ -1 +1,31 @@
 # Ansible Collection - autoware.dev_env
+
+This collection contains the playbooks to set up the development environment for Autoware.
+
+## Set up a development environment
+
+### Ansible installation
+
+```bash
+# Remove apt installed ansible (In Ubuntu 22.04, ansible the version is old)
+sudo apt-get purge ansible
+
+# Install pipx
+sudo apt-get -y update
+sudo apt-get -y install pipx
+
+# Add pipx to the system PATH
+python3 -m pipx ensurepath
+
+# Install ansible
+pipx install --include-deps --force "ansible==6.*"
+```
+
+### Install ansible collections
+
+This step should be repeated when a new playbook is added.
+
+```bash
+cd ~/autoware # The root directory of the cloned repository
+ansible-galaxy collection install -f -r "ansible-galaxy-requirements.yaml"
+```

ansible/playbooks/download_artifacts.yaml

@@ -0,0 +1,4 @@
+- name: Download Autoware artifacts
+  hosts: localhost
+  roles:
+    - autoware.dev_env.artifacts

ansible/roles/artifacts/README.md

@@ -1,144 +1,32 @@
 # Autoware artifacts
 
-The Autoware perception stack uses models for inference. These models are automatically downloaded if using `ansible`, but they can also be downloaded manually.
+The Autoware perception stack uses models for inference. These models are automatically downloaded as part of the `setup-dev-env.sh` script.
 
-## Download instructions
-
-The artifacts files are stored in a common location, hosted by Web.Auto
-
-Any tool that can download files from the web (e.g. `wget` or `curl`) is the only requirement for downloading these files:
-
-```console
-# yabloc_pose_initializer
-
-$ mkdir -p ~/autoware_data/yabloc_pose_initializer/
-$ wget -P ~/autoware_data/yabloc_pose_initializer/ \
-       https://s3.ap-northeast-2.wasabisys.com/pinto-model-zoo/136_road-segmentation-adas-0001/resources.tar.gz
-
-
-# image_projection_based_fusion
-
-$ mkdir -p ~/autoware_data/image_projection_based_fusion/
-$ wget -P ~/autoware_data/image_projection_based_fusion/ \
-       https://awf.ml.dev.web.auto/perception/models/pointpainting/v4/pts_voxel_encoder_pointpainting.onnx \
-       https://awf.ml.dev.web.auto/perception/models/pointpainting/v4/pts_backbone_neck_head_pointpainting.onnx
-
-
-# lidar_apollo_instance_segmentation
-
-$ mkdir -p ~/autoware_data/lidar_apollo_instance_segmentation/
-$ wget -P ~/autoware_data/lidar_apollo_instance_segmentation/ \
-       https://awf.ml.dev.web.auto/perception/models/lidar_apollo_instance_segmentation/vlp-16.onnx \
-       https://awf.ml.dev.web.auto/perception/models/lidar_apollo_instance_segmentation/hdl-64.onnx \
-       https://awf.ml.dev.web.auto/perception/models/lidar_apollo_instance_segmentation/vls-128.onnx
-
-
-# lidar_centerpoint
-
-$ mkdir -p ~/autoware_data/lidar_centerpoint/
-$ wget -P ~/autoware_data/lidar_centerpoint/ \
-       https://awf.ml.dev.web.auto/perception/models/centerpoint/v2/pts_voxel_encoder_centerpoint.onnx \
-       https://awf.ml.dev.web.auto/perception/models/centerpoint/v2/pts_backbone_neck_head_centerpoint.onnx \
-       https://awf.ml.dev.web.auto/perception/models/centerpoint/v2/pts_voxel_encoder_centerpoint_tiny.onnx \
-       https://awf.ml.dev.web.auto/perception/models/centerpoint/v2/pts_backbone_neck_head_centerpoint_tiny.onnx
-
-
-# tensorrt_yolox
-
-$ mkdir -p ~/autoware_data/tensorrt_yolox/
-$ wget -P ~/autoware_data/tensorrt_yolox/ \
-       https://awf.ml.dev.web.auto/perception/models/yolox-tiny.onnx \
-       https://awf.ml.dev.web.auto/perception/models/yolox-sPlus-opt.onnx \
-       https://awf.ml.dev.web.auto/perception/models/yolox-sPlus-opt.EntropyV2-calibration.table \
-       https://awf.ml.dev.web.auto/perception/models/object_detection_yolox_s/v1/yolox-sPlus-T4-960x960-pseudo-finetune.onnx \
-       https://awf.ml.dev.web.auto/perception/models/object_detection_yolox_s/v1/yolox-sPlus-T4-960x960-pseudo-finetune.EntropyV2-calibration.table \
-       https://awf.ml.dev.web.auto/perception/models/label.txt
-
-
-# traffic_light_classifier
-
-$ mkdir -p ~/autoware_data/traffic_light_classifier/
-$ wget -P ~/autoware_data/traffic_light_classifier/ \
-       https://awf.ml.dev.web.auto/perception/models/traffic_light_classifier/v2/traffic_light_classifier_mobilenetv2_batch_1.onnx \
-       https://awf.ml.dev.web.auto/perception/models/traffic_light_classifier/v2/traffic_light_classifier_mobilenetv2_batch_4.onnx \
-       https://awf.ml.dev.web.auto/perception/models/traffic_light_classifier/v2/traffic_light_classifier_mobilenetv2_batch_6.onnx \
-       https://awf.ml.dev.web.auto/perception/models/traffic_light_classifier/v2/traffic_light_classifier_efficientNet_b1_batch_1.onnx \
-       https://awf.ml.dev.web.auto/perception/models/traffic_light_classifier/v2/traffic_light_classifier_efficientNet_b1_batch_4.onnx \
-       https://awf.ml.dev.web.auto/perception/models/traffic_light_classifier/v2/traffic_light_classifier_efficientNet_b1_batch_6.onnx \
-       https://awf.ml.dev.web.auto/perception/models/traffic_light_classifier/v2/lamp_labels.txt \
-       https://awf.ml.dev.web.auto/perception/models/traffic_light_classifier/v3/ped_traffic_light_classifier_mobilenetv2_batch_1.onnx \
-       https://awf.ml.dev.web.auto/perception/models/traffic_light_classifier/v3/ped_traffic_light_classifier_mobilenetv2_batch_4.onnx \
-       https://awf.ml.dev.web.auto/perception/models/traffic_light_classifier/v3/ped_traffic_light_classifier_mobilenetv2_batch_6.onnx \
-       https://awf.ml.dev.web.auto/perception/models/traffic_light_classifier/v3/lamp_labels_ped.txt
+The models are hosted by Web.Auto.
 
+Default `data_dir` location is `~/autoware_data`.
 
+## Download instructions
 
-# traffic_light_fine_detector
-
-$ mkdir -p ~/autoware_data/traffic_light_fine_detector/
-$ wget -P ~/autoware_data/traffic_light_fine_detector/ \
-       https://awf.ml.dev.web.auto/perception/models/tlr_yolox_s/v3/tlr_car_ped_yolox_s_batch_1.onnx \
-       https://awf.ml.dev.web.auto/perception/models/tlr_yolox_s/v3/tlr_car_ped_yolox_s_batch_4.onnx \
-       https://awf.ml.dev.web.auto/perception/models/tlr_yolox_s/v3/tlr_car_ped_yolox_s_batch_6.onnx \
-       https://awf.ml.dev.web.auto/perception/models/tlr_yolox_s/v3/tlr_labels.txt
-
-
-# tvm_utility
-
-$ mkdir -p ~/autoware_data/tvm_utility/models/yolo_v2_tiny
-$ wget -P ~/autoware_data/tvm_utility/ \
-       https://autoware-modelzoo.s3.us-east-2.amazonaws.com/models/3.0.0-20221221/yolo_v2_tiny-x86_64-llvm-3.0.0-20221221.tar.gz
-
-
-# lidar_centerpoint_tvm
-
-$ mkdir -p ~/autoware_data/lidar_centerpoint_tvm/models/centerpoint_encoder
-$ mkdir -p ~/autoware_data/lidar_centerpoint_tvm/models/centerpoint_backbone
-$ wget -P ~/autoware_data/lidar_centerpoint_tvm/ \
-       https://autoware-modelzoo.s3.us-east-2.amazonaws.com/models/3.0.0-20221221/centerpoint_encoder-x86_64-llvm-3.0.0-20221221.tar.gz \
-       https://autoware-modelzoo.s3.us-east-2.amazonaws.com/models/3.0.0-20221221/centerpoint_backbone-x86_64-llvm-3.0.0-20221221.tar.gz
-
+### Requirements
 
-# lidar_apollo_segmentation_tvm
+Install ansible following the instructions in the [ansible installation guide](../../README.md#ansible-installation).
 
-$ mkdir -p ~/autoware_data/lidar_apollo_segmentation_tvm/models/baidu_cnn
-$ wget -P ~/autoware_data/lidar_apollo_segmentation_tvm/ \
-      https://autoware-modelzoo.s3.us-east-2.amazonaws.com/models/3.0.0-20221221/baidu_cnn-x86_64-llvm-3.0.0-20221221.tar.gz
-```
+### Download artifacts
 
-After downloading you can check integrity of the files with `sha256sum`:
+#### Install ansible collections
 
-```console
-#
-$ cd ~/autoware_data/
-$ wget -q -O - https://raw.githubusercontent.com/autowarefoundation/autoware/main/ansible/roles/artifacts/SHA256SUMS | sha256sum -c
+```bash
+cd ~/autoware # The root directory of the cloned repository
+ansible-galaxy collection install -f -r "ansible-galaxy-requirements.yaml"
 ```
 
-Extracting files:
+This step should be repeated when a new playbook is added.
 
-```console
-# yabloc_pose_initializer
+#### Run the playbook
 
-$ tar -xf ~/autoware_data/yabloc_pose_initializer/resources.tar.gz \
-       -C ~/autoware_data/yabloc_pose_initializer/
-
-
-# tvm_utility
-
-$ tar -xf ~/autoware_data/tvm_utility/yolo_v2_tiny-x86_64-llvm-3.0.0-20221221.tar.gz \
-       -C ~/autoware_data/tvm_utility/models/yolo_v2_tiny/
-
-
-# lidar_centerpoint_tvm
-
-$ tar -xf ~/autoware_data/lidar_centerpoint_tvm/centerpoint_encoder-x86_64-llvm-3.0.0-20221221.tar.gz \
-       -C ~/autoware_data/lidar_centerpoint_tvm/models/centerpoint_encoder
-$ tar -xf ~/autoware_data/lidar_centerpoint_tvm/centerpoint_backbone-x86_64-llvm-3.0.0-20221221.tar.gz \
-       -C ~/autoware_data/lidar_centerpoint_tvm/models/centerpoint_backbone
-
-
-# lidar_apollo_segmentation_tvm
-
-$ tar -xf ~/autoware_data/lidar_apollo_segmentation_tvm/baidu_cnn-x86_64-llvm-3.0.0-20221221.tar.gz \
-       -C ~/autoware_data/lidar_apollo_segmentation_tvm/models/baidu_cnn
+```bash
+ansible-playbook autoware.dev_env.download_artifacts -e "data_dir=$HOME/autoware_data" --ask-become-pass
 ```
+
+This will download and extract the artifacts to the specified directory and validate the checksums.