Skip to content

Commit

Permalink
update dataset descriptions; release metadata
Browse files Browse the repository at this point in the history
  • Loading branch information
Ir1d authored May 27, 2024
1 parent 61b2019 commit 2759f93
Showing 1 changed file with 11 additions and 7 deletions.
18 changes: 11 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,26 +2,30 @@

The official implementation of paper "Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Models".

[[Project Page]](https://vita-group.github.io/Diffusion4D/) | [[Video (narrated)]](https://www.youtube.com/watch?v=9q8SV1Xf_Xw) | [[Video (results)]](https://www.youtube.com/watch?v=gXVoPTGb734) | [[Arxiv]](https://arxiv.org/abs/2403.16993)
[[Project Page]](https://vita-group.github.io/Diffusion4D/) | [[Video (narrated)]](https://www.youtube.com/watch?v=9q8SV1Xf_Xw) | [[Video (results)]](https://www.youtube.com/watch?v=gXVoPTGb734) | [[Arxiv]](https://arxiv.org/abs/2403.16993) | [[Dataset]](https://huggingface.co/datasets/hw-liang/Diffusion4D)

## News

- 2024.5.28: Released data preparation code!
- 2024.5.27: Released metadata for objects!
- 2024.5.26: Released on arxiv!



# Preparation of 4D data
We curate a large-scale, high-quality dynamic 3D(4D) dataset sourced from the
vast 3D data corpus of [Objaverse-1.0](https://objaverse.allenai.org/objaverse-1.0/) and [Objaverse-XL](https://github.com/allenai/objaverse-xl). We apply a series of empirical rules to filter the dataset. You can find more details in our paper. In this part, we release the selected 4D assets including:
We collect a large-scale, high-quality dynamic 3D(4D) dataset sourced from the
vast 3D data corpus of [Objaverse-1.0](https://objaverse.allenai.org/objaverse-1.0/) and [Objaverse-XL](https://github.com/allenai/objaverse-xl). We apply a series of empirical rules to filter the dataset. You can find more details in our paper. In this part, we release the selected 4D assets, including:
1. Selected high-quality 4D object ID.
2. A render script using Blender, providing optional settings to render your personalized data.
3. (To be uploaded) Rendered 4D images by our team to save you GPU time.

## 4D Dataset ID
We curated a total of 54K dynamic 3D assets from Objaverse-1.0 and Objaverse-xl. With objaverse-1.0, we provide the selected 11K ids in `rendering/src/ObjV1_curated.txt`. Uncurated 42K ids of all the animated cases in objaverse-1.0 is in `rendering/src/ObjV1_all_animated.txt`
## 4D Dataset ID/Metadata
We collect a total of 54K dynamic 3D assets from Objaverse-1.0 and Objaverse-xl. With objaverse-1.0, we provide the selected 11K ids in `rendering/src/ObjV1_curated.txt`. Uncurated 42K ids of all the animated objects from objaverse-1.0 are in `rendering/src/ObjV1_all_animated.txt`.

Metadata of animated objects from objaverse-xl can be found in [huggingface](https://huggingface.co/datasets/hw-liang/Diffusion4D). We also release the metadata of all successfully rendered objects from objaverse-xl's Github subset.

For text-to-4D generation, the captions are obtained from the work [Cap3D](https://huggingface.co/datasets/tiange/Cap3D).
More about the dataset and curation scripts coming soon!
More about the dataset and curation scripts are coming soon!

## 4D Dataset Rendering Script
1. Clone the repository and enter the rendering directory:
Expand Down Expand Up @@ -120,4 +124,4 @@ This project is based on numerous outstanding research efforts and open-source c
```
@article{
}
```
```

0 comments on commit 2759f93

Please sign in to comment.