Fantasia3D on Google Colab

[blueangel1313]

I hope this message finds you well. I have been trying to create a Google Colab notebook for Fantasia3D, but I have encountered some difficulties while installing the environment. I wanted to reach out and share my experience, in case it is helpful for you in creating your own notebook.

During the installation process, I faced some issues that prevented me from successfully setting up the Fantasia3D environment.

Thank you for your attention and support. I am looking forward to your guidance on resolving the installation issues for Fantasia3D on Google Colab.

https://colab.research.google.com/drive/1bqQAr211he89yJxDKRciuXsfqrKpkeHz?usp=sharing&authuser=1#scrollTo=qQ-WGNGUF8SQ

[ARuiChen]

Hi, @blueangel1313 ! I couldn't open up the URL. Could you check it again?

[blueangel1313]

https://colab.research.google.com/drive/1bqQAr211he89yJxDKRciuXsfqrKpkeHz?usp=sharing

Hi, how about now?

[zxhxixi]

https://colab.research.google.com/drive/1bqQAr211he89yJxDKRciuXsfqrKpkeHz?usp=sharing

Hi, how about now?

same issue, have you fixed it now?

[git2andi]

Hey, so I have tried doing the same. It seems to work as intended so far, however there is an issue with Cuda out of memmory at the geometry stage. The GPU usage shown while computing in Colab only is about 4-6GB, but at iteration ~1500, the error occurs. Is there a way to reduce the memory consumption? I have tried reducing the batch size to 4, but the error remains.
Also, I am not sure if all requirements implemented are required as I took most of them from threestudio.

https://colab.research.google.com/drive/159utSA6lON6jez5sigf1BuECLAeIkanK?usp=sharing

And one last thing, when initializing a sphere system.geometry.shape_init=ellipsoid system.geometry.shape_init_params="[0.3,0.3,0.7]", is the first value here (0.3) the width, the second (0.3) the depth and the third (0.7) the height of this sphere? or does the second parameter correspond to the height of the sphere?

[ARuiChen]

@git2andi For geometry modeling, the VAE encoder is not used during the coarse stage, so memory usage is relatively low. After 1500 iterations, it will enter the finer stage. Due to the improvement in resolution, the memory usage will increase, which will require approximately 12GB or more.
Here are some suggestions:
1、For geometry modeling, you can modify some parameters as follows:
"sdf_init_shape": "custom_mesh",
"base_mesh": "data/head/head.obj",
"sdf_init_shape_scale": [1.2, 1.2, 1.2],
"iter": 2000,
"coarse_iter": 2000,
"negative_text": "low quality",
"early_time_step_range": [0.2, 0.6],
"late_time_step_range": [0.05, 0.5]

negative prompt can improve the quality.

2、For appearance modeling, you can add the negative prompt as follows:
"negative_text": "shadow, oversaturated, low quality, unrealistic",
"sds_weight_strategy": 2,
"early_time_step_range": [0.05, 0.98],
"late_time_step_range": [0.05, 0.7],

Please refer to the following diagram for the coordinate system