Global Tracking Transformers

Global Tracking Transformers,
Xingyi Zhou, Tianwei Yin, Vladlen Koltun, Philipp Krähenbühl,
CVPR 2022 (arXiv 2203.13250)

Features

• Object association within a long temporal window (32 frames).

• Classification after tracking for long-tail recognition.

• "Detector" of global trajectories.

Installation

See installation instructions.

Demo

Run our demo using Colab (no GPU needed):

Try Replicate web demo here

We use the default detectron2 demo interface. For example, to run TAO model on an example video (video source: TAO/YFCC100M dataset), download the model and run

python demo.py --config-file configs/GTR_TAO_DR2101.yaml --video-input docs/yfcc_v_acef1cb6d38c2beab6e69e266e234f.mp4 --output output/demo_yfcc.mp4 --opts MODEL.WEIGHTS models/GTR_TAO_DR2101.pth

If setup correctly, the output on output/demo_yfcc.mp4 should look like:

Benchmark evaluation and training

Please first prepare datasets, then check our MODEL ZOO to reproduce results in our paper. We highlight key results below:

• MOT17 test set

MOTA	IDF1	HOTA	DetA	AssA	FPS
75.3	71.5	59.1	61.6	57.0	19.6

• TAO test set

Track mAP	FPS
20.1	11.2

License

The majority of GTR is licensed under the Apache 2.0 license, however portions of the project are available under separate license terms: trackeval in gtr/tracking/trackeval/, is licensed under the MIT license. FairMOT in gtr/tracking/local_tracker is under MIT license. Please see NOTICE for license details. The demo video is from TAO dataset, which is originally from YFCC100M dataset. Please be aware of the original dataset license.

Citation

If you find this project useful for your research, please use the following BibTeX entry.

@inproceedings{zhou2022global,
  title={Global Tracking Transformers},
  author={Zhou, Xingyi and Yin, Tianwei and Koltun, Vladlen and Kr{\"a}henb{\"u}hl, Philipp},
  booktitle={CVPR},
  year={2022}
}