Yolov5 + Deep Sort with PyTorch

Introduction

THIS REPO IS FORKED FROM https://github.com/mikel-brostrom/Yolov5_DeepSort_Pytorch AND CHANGED FOR NEEDS!

This repository contains a two-stage-tracker. The detections generated by YOLOv5, a family of object detection architectures and models pretrained on the COCO dataset, are passed to a Deep Sort algorithm which tracks the objects. It can track any object that your Yolov5 model was trained to detect.

Tutorials

• Yolov5 training on Custom Data (link to external repository) 
• Deep Sort deep descriptor training (link to external repository) 
• Yolov5 deep_sort pytorch evaluation 

Jetson Nano Installation

NOTE: TAKES A COUPLE OF HOURS TO COMPLETE!

1. This framework needs clean system. Download Jetson Nano Developer Kit SD Card Image and run first boot. This image includes the latest version of JetPack.
2. After first boot, update OpenCV to the latest version from following link.
   1. https://qengineering.eu/install-opencv-4.5-on-jetson-nano.html
   2. NOTE: In script, comment out following line sudo rm -r /usr/include/opencv4/opencv2, before run.
3. Install following dependencies.
   1. sudo apt-get install -y nano curl
   2. sudo apt-get install -y python3-pip python3-dev python3-setuptools
   3. sudo apt-get install -y libcanberra-gtk0 libcanberra-gtk-module
   4. sudo apt-get install libhdf5-serial-dev hdf5-tools libhdf5-dev zlib1g-dev zip libjpeg8-dev liblapack-dev libblas-dev gfortran
   5. pip3 install -U pip
   6. pip3 install setuptools wheel cython
   7. pip3 install virtualenv virtualenvwrapper
   8. sudo apt install ttf-mscorefonts-installer
   9. sudo fc-cache -f
4. Set CUDA Path.

    $ echo "export PATH=/usr/local/cuda/bin:\${PATH}" >> ${HOME}/.bashrc
    $ echo "export LD_LIBRARY_PATH=/usr/local/cuda/lib64:\${LD_LIBRARY_PATH}" >> ${HOME}/.bashrc
    $ echo "export CPATH=$CPATH:/usr/local/cuda/targets/aarch64-linux/include" >> ${HOME}/.bashrc
    $ echo "export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/cuda/targets/aarch64-linux/lib" >> {HOME}/.bashrc
    $ source ~/.bashrc

5. Build "jetson-inference" framework without PyTorch installation.
   1. https://github.com/dusty-nv/jetson-inference/blob/master/docs/building-repo-2.md
6. Install PyTorch 1.10 and Torchvision 0.11.1 from following link.
   1. https://forums.developer.nvidia.com/t/pytorch-for-jetson-version-1-10-now-available/72048

Now, you can move to requirements installation part.

Before you run the tracker (Install Requirements)

1. Create virtual environment for framework with system packages (https://stackoverflow.com/a/36096474). Therefore, we will use already installed PyTorch, OpenCV, etc.
   1. mkvirtualenv -p $(which python3) --system-site-packages <venv-name>
2. Activate virtual environment.
   1. workon <venv-name>
3. Clone the repository recursively:
   1. git clone --recurse-submodules https://github.com/acikmese/MOT_Yolov5_DeepSort.git
   2. If you already cloned and forgot to use --recurse-submodules you can run git submodule update --init
4. Make sure that you fulfill all the requirements: Python 3.6 or later with all requirements.txt dependencies installed, including torch>=1.7. To install, run:
   1. pip install -Ur requirements.txt
5. Install pycocotools.
   1. pip3 install 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'
   2. pip3 install cython_bbox
6. Install torchreid.
   1. git clone https://github.com/KaiyangZhou/deep-person-reid.git
   2. cd deep-person-reid/
   3. pip install -Ur requirements.txt
   4. python setup.py develop
7. Delete any opencv package in virtualenv, because there will be a conflict with system-site package.
   1. pip uninstall opencv-contrib-python
   2. pip uninstall opencv-python
   3. pip uninstall opencv-python-headless

Tracking sources

Tracking can be run on most video formats

$ python track.py --source 0  # webcam
                           img.jpg  # image
                           vid.mp4  # video
                           path/  # directory
                           path/*.jpg  # glob
                           'https://youtu.be/Zgi9g1ksQHc'  # YouTube
                           'rtsp://example.com/media.mp4'  # RTSP, RTMP, HTTP stream

Select object detection and ReID model

Yolov5

There is a clear trade-off between model inference speed and accuracy. In order to make it possible to fulfill your inference speed/accuracy needs you can select a Yolov5 family model for automatic download

$ python track.py --source 0 --yolo_model yolov5n.pt --img 640
                                            yolov5s.pt
                                            yolov5m.pt
                                            yolov5l.pt 
                                            yolov5x.pt --img 1280
                                            ...

DeepSort

Choose a ReID model based on your needs from this ReID model zoo

$ python track.py --source 0 --deep_sort_model osnet_x1_0
                                            nasnsetmobile
                                            resnext101_32x8d
                                            ...

Filter tracked classes

By default the tracker tracks all MS COCO classes.

If you only want to track persons I recommend you to get these weights for increased performance

python3 track.py --source 0 --yolo_model yolov5/weights/crowdhuman_yolov5m.pt --classes 0  # tracks persons, only

If you want to track a subset of the MS COCO classes, add their corresponding index after the classes flag

python3 track.py --source 0 --yolo_model yolov5s.pt --classes 16 17  # tracks cats and dogs, only

Here is a list of all the possible objects that a Yolov5 model trained on MS COCO can detect. Notice that the indexing for the classes in this repo starts at zero.

MOT compliant results

Can be saved to inference/output by

python3 track.py --source ... --save-txt

Cite

If you find this project useful in your research, please consider cite:

@misc{yolov5deepsort2020,
    title={Real-time multi-object tracker using YOLOv5 and deep sort},
    author={Mikel Broström},
    howpublished = {\url{https://github.com/mikel-brostrom/Yolov5_DeepSort_Pytorch}},
    year={2020}
}