See rknnReadme.md
you need to download rknn-tookit2,and install rknn-tookit2 on your ubuntu(x86-64,amd64)
You need to download model from PySOT Model Zoo.
cd to rootPath(pysot_rknn)
python rknn/siamrpn_alex_dwxcorr.py --config experiments/siamrpn_alex_dwxcorr/config.yaml --snapshot experiments/siamrpn_alex_dwxcorr/model.pth --video demo/bag.avi
The file siamrpn_alex_dwxcorr.py can export rknn model, and test the result on your ubuntu(x86-64,amd64).
Detail about siamrpn_alex_dwxcorr model.
backbone of exemplar.
Target img:
input size: torch.Size([1, 3, 127, 127])
outputsize: torch.Size([1, 256, 6, 6])
backbone of instance.
Original img:
input size: torch.Size([1, 3, 287, 287])
outputsize: torch.Size([1, 256, 26, 26])
rpn Head
rpn head input size: [z_f,x_f]
z_f shape: torch.Size([1, 256, 6, 6])
x_f shape: torch.Size([1, 256, 26, 26])
output size: output[[cls],[loc]]
cls shape: (1, 10, 21, 21)
loc shape: (1, 20, 21, 21)
z_f is the output of backbone of exemplar, x_f is the output of backbone of instance
Install rknn-toolkit-lite2 on your rk device, I run the demo on RK3588.
Copy folders and files on the projict to your rk device, and your need to maintain the structure of the project.
pysot_rknn(root path)
demo/
tools/
rknn/
experiments/
pysot/
python tools/runRKNNLite.py
PySOT is a software system designed by SenseTime Video Intelligence Research team. It implements state-of-the-art single object tracking algorithms, including SiamRPN and SiamMask. It is written in Python and powered by the PyTorch deep learning framework. This project also contains a Python port of toolkit for evaluating trackers.
PySOT has enabled research projects, including: SiamRPN, DaSiamRPN, SiamRPN++, and SiamMask.
Example SiamFC, SiamRPN and SiamMask outputs.
The goal of PySOT is to provide a high-quality, high-performance codebase for visual tracking research. It is designed to be flexible in order to support rapid implementation and evaluation of novel research. PySOT includes implementations of the following visual tracking algorithms:
using the following backbone network architectures:
Additional backbone architectures may be easily implemented. For more details about these models, please see References below.
Evaluation toolkit can support the following datasets:
📎 OTB2015 📎 VOT16/18/19 📎 VOT18-LT 📎 LaSOT 📎 UAV123
We provide a large set of baseline results and trained models available for download in the PySOT Model Zoo.
Please find installation instructions for PyTorch and PySOT in INSTALL.md.
export PYTHONPATH=/path/to/pysot:$PYTHONPATHDownload models in PySOT Model Zoo and put the model.pth in the correct directory in experiments
python tools/demo.py \
--config experiments/siamrpn_r50_l234_dwxcorr/config.yaml \
--snapshot experiments/siamrpn_r50_l234_dwxcorr/model.pth
# --video demo/bag.avi # (in case you don't have webcam)Download datasets and put them into testing_dataset directory. Jsons of commonly used datasets can be downloaded from Google Drive or BaiduYun. If you want to test tracker on new dataset, please refer to pysot-toolkit to setting testing_dataset.
cd experiments/siamrpn_r50_l234_dwxcorr
python -u ../../tools/test.py \
--snapshot model.pth \ # model path
--dataset VOT2018 \ # dataset name
--config config.yaml # config fileThe testing results will in the current directory(results/dataset/model_name/)
assume still in experiments/siamrpn_r50_l234_dwxcorr_8gpu
python ../../tools/eval.py \
--tracker_path ./results \ # result path
--dataset VOT2018 \ # dataset name
--num 1 \ # number thread to eval
--tracker_prefix 'model' # tracker_nameSee TRAIN.md for detailed instruction.
If you meet problem, try searching our GitHub issues first. We intend the issues page to be a forum in which the community collectively troubleshoots problems. But please do not post duplicate issues. If you have similar issue that has been closed, you can reopen it.
ModuleNotFoundError: No module named 'pysot'
🎯Solution: Run export PYTHONPATH=path/to/pysot first before you run the code.
🎯Solution: if you just want run the demo on WINDOWS, Run SET PYTHONPATH=path\\pyProject\\pysot
🎯Solution: If you use vscode you can see the .vscode/settings.json
ImportError: cannot import name region
🎯Solution: Build region by python setup.py build_ext —-inplace as decribled in INSTALL.md.
Some problem about cython
🎯Solution: pip install Cython==0.29.21.
-
Fast Online Object Tracking and Segmentation: A Unifying Approach. Qiang Wang, Li Zhang, Luca Bertinetto, Weiming Hu, Philip H.S. Torr. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
-
SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks. Bo Li, Wei Wu, Qiang Wang, Fangyi Zhang, Junliang Xing, Junjie Yan. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
-
Distractor-aware Siamese Networks for Visual Object Tracking. Zheng Zhu, Qiang Wang, Bo Li, Wu Wei, Junjie Yan, Weiming Hu. The European Conference on Computer Vision (ECCV), 2018.
-
High Performance Visual Tracking with Siamese Region Proposal Network. Bo Li, Wei Wu, Zheng Zhu, Junjie Yan, Xiaolin Hu. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
-
Fully-Convolutional Siamese Networks for Object Tracking. Luca Bertinetto, Jack Valmadre, João F. Henriques, Andrea Vedaldi, Philip H. S. Torr. The European Conference on Computer Vision (ECCV) Workshops, 2016.
PySOT is released under the Apache 2.0 license.