Skip to content

Latest commit

 

History

History
130 lines (113 loc) · 4.11 KB

README.md

File metadata and controls

130 lines (113 loc) · 4.11 KB
Raw

Tracking Anything in High Quality

Technical Report:

Tracking Anything in High Quality (HQTrack) is a framework for high performance video object tracking and segmentation. It mainly consists of a Video Multi-Object Segmenter (VMOS) and a Mask Refiner (MR), can track multiple target objects at the same time and output accurate object masks.

🍺 HQTrack obtains runner-up in the Visual Object Tracking and Segmentaion (VOTS2023) challenge.

📢News

  • [2023/7/30] We provide a demo code that can be run locally.
  • [2023/7/22] We author a technical report for HQTrack.
  • [2023/7/3] HQTrack ranks the 2nd place in the VOTS2023 challenge.

🔥Demo

We also provide a demo script, which supports box and point prompts as inputs. This is a pure python script that allows the user to test arbitrary videos.

🐍Pipeline

image

📑Intallation

  • Install the conda environment
conda create -n hqtrack python=3.8
conda activate hqtrack
  • Install Pytorch
conda install pytorch==1.9 torchvision cudatoolkit=10.2 -c pytorch
  • Install HQ-SAM
cd segment_anything_hq
pip install -e .
pip install opencv-python pycocotools matplotlib onnxruntime onnx
  • Install Pytorch-Correlation-extension package
cd packages/Pytorch-Correlation-extension/
python setup.py install
  • Install ops_dcnv3
cd HQTrack/networks/encoders/ops_dcnv3
./make.sh
  • Install vots toolkit
pip install vot-toolkit
  • Install other packages
pip install easydict
pip install lmdb
pip install einops
pip install jpeg4py
pip install 'protobuf~=3.19.0'
conda install setuptools==58.0.4
pip install timm
pip install tb-nightly
pip install tensorboardx
pip install scikit-image
pip install rsa
pip install six
pip install pillow

🚗Run HQTrack

  • Model Preparation

Download VMOS model from Google Driver or Baidu Driver and put it under

/path/to/HQTrack/result/default_InternT_MSDeAOTL_V2/YTB_DAV_VIP/ckpt/

Download HQ-SAM_h and put it under

/path/to/HQTrack/segment_anything_hq/pretrained_model/
  • Initialize the vots workspace
cd /path/to/VOTS23_workspace
vot initialize tests/multiobject
  • Copy our trackers.ini to your vot workspace
cp /path/to/our/trackers.ini /path/to/VOTS23_workspace/trackers.ini
  • Modify your path in trackers.ini
  • test the tracker and pack the results
bash run.sh

📖 Citation

If you find HQTrack useful for you, please consider citing 📣

@misc{hqtrack,
      title={Tracking Anything in High Quality}, 
      Author = {Jiawen Zhu and Zhenyu Chen and Zeqi Hao and Shijie Chang and Lu Zhang and Dong Wang and Huchuan Lu and Bin Luo and Jun-Yan He and Jin-Peng Lan and Hanyuan Chen and Chenyang Li},
      Title = {Tracking Anything in High Quality},
      Year = {2023},
      Eprint = {arXiv:2307.13974},
      PrimaryClass={cs.CV}
}

♥️ Acknowledgment

This project is based on DeAOT, HQ-SAM, and SAM. Thanks for these excellent works.

📧Contact

If you have any question, feel free to email [email protected]. ^_^