Maintained by Yong-Lu Li, Xiaoqian Wu, Xinpeng Liu, Yiming Dou, and Cewu Lu.
- Pangea (CVPR 2024 Highlight): Unified action semantic space (label system) tp aggregate all action datasets into ONE. Pangea
Sandwich is a project for general human activity understanding. It consists of:
- A novel structured semantic space to re-explain all the action classes from existing action datasets of multiple modalities.
- A large-scale multi-modal action database, Pangea, aggregating a plenty of existing action datasets.
- A bidirectional mapping system to learn the bi-mapping between human action physical-semantic space and facilitate the action recognition and generation.
We are gathering more and more human action datasets and linking their label systems to Pangea (CVPR 2024), for more details please refer to this page.
- Given our new hierarchical semantic system, you can train big models on all action datasets under ONE label system with principled knowledge!
For python environment, see requirements.txt.
The data is given here. Download the folder, rename it as "Data", and put it in the root dir.
See p2s_mapping. Run eval_node.sh to infer verb node labels.
