cpp

English | 简体中文

YOLOv5 Quantification Model C++ Deployment Example

This directory provides examples that infer.cc fast finishes the deployment of YOLOv5 on RV1126.

Prepare the deployment

Prepare FastDeploy cross-compilation environment

1. For the environment of software, hardware and cross-compilation, refer to FastDeploy cross-compilation environment preparation

Prepare the quantification model

Users can directly deploy quantized models provided by FastDeploy or prepare quantification models as the following steps:

1. Refer to YOLOv5 to officially convert the ONNX model or use the following command to download it.

wget https://paddle-slim-models.bj.bcebos.com/act/yolov5s.onnx

2. Prepare 300 or so images for quantization.And we can also download the prepared data using the following command:

wget https://bj.bcebos.com/fastdeploy/models/COCO_val_320.tar.gz
tar -xf COCO_val_320.tar.gz

3. Users can use the One-click auto-compression tool provided by FastDeploy to automatically conduct quantification model for deployment.

fastdeploy compress --config_path=./configs/detection/yolov5s_quant.yaml --method='PTQ' --save_dir='./yolov5s_ptq_model_new/'

4. The YOLOv5 model requires heterogeneous computing. Refer to Heterogeneous Computing. Since FastDeploy already provides the YOLOv5 model, we can first test the heterogeneous files to verify whether the accuracy meets the requirements.

# Download the model, unzip it, and copy the subgraph.txt file to your newly quantized model directory
wget https://bj.bcebos.com/fastdeploy/models/yolov5s_ptq_model.tar.gz
tar -xvf yolov5s_ptq_model.tar.gz

Refer to model quantification for more information

Deploy quantized YOLOv5 detection model on RV1126

Refer to the following steps:

1. For cross compiling FastDeploy repo, refer to cross compiling FastDeploy

2. Copy the compiled repo to your current directory through the following command:

cp -r FastDeploy/build/fastdeploy-timvx/ FastDeploy/examples/vision/detection/yolov5/rv1126/cpp

3. Download models and images for deployment in the current location:

cd FastDeploy/examples/vision/detection/yolov5/rv1126/cpp
mkdir models && mkdir images
wget https://bj.bcebos.com/fastdeploy/models/yolov5s_ptq_model.tar.gz
tar -xvf yolov5s_ptq_model.tar.gz
cp -r yolov5s_ptq_model models
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
cp -r 000000014439.jpg images

4. Compile the deployment example through the following command:

cd FastDeploy/examples/vision/detection/yolov5/rv1126/cpp
mkdir build && cd build
cmake -DCMAKE_TOOLCHAIN_FILE=${PWD}/../fastdeploy-timvx/toolchain.cmake -DFASTDEPLOY_INSTALL_DIR=${PWD}/../fastdeploy-timvx -DTARGET_ABI=armhf ..
make -j8
make install
# Successful compilation generates the install folder, containing a running demo and repo required for deployment

5. Deploy YOLOv5 detection model to Rockchip RV1126 based on adb. Refer to the following command:

# Enter the install directory
cd FastDeploy/examples/vision/detection/yolov5/rv1126/cpp/build/install/
# The following commands represent: bash run_with_adb.sh running demo  model path  image path   DEVICE_ID
bash run_with_adb.sh infer_demo yolov5s_ptq_model 000000014439.jpg $DEVICE_ID

vis_result.jpg after successful deployment:

Note that the deployment model on RV1126 is the quantized model. Refer to Model Quantification