Mochi docs #9934
Mochi docs #9934
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| prompt = "Close-up of a chameleon's eye, with its scaly skin changing color. Ultra high resolution 4k." | ||
|
|
||
| with torch.autocast("cuda", torch.bfloat16, cache_enabled=False): | ||
| frames = pipe(prompt, num_frames=84).frames[0] |
There was a problem hiding this comment.
num_frames=84 should be num_frames=85, (14 * 6 + 1) like mentioned here
|
|
||
| ## Using a lower precision variant to save memory | ||
|
|
||
| The following example will use the `bfloat16` variant of the model and requires 22GB VRAM to run. There is a slight drop in the quality of the generated video as a result. |
There was a problem hiding this comment.
should bf16 be used? fp16 seems to have better precision, unless the autocasted weights compute in fp32 and somehow provide better precision?
There was a problem hiding this comment.
We followed the official repo, which cast to bf16.
https://github.com/genmoai/models/blob/1a96b5df0c83018fcbe7aebbbb1b34781e179921/src/genmo/mochi_preview/pipelines.py#L344
I haven't fully compared bf16 vs fp16 for full downcasting though. Are you seeing much better results with fp16?
There was a problem hiding this comment.
I am indeed, though I'm also doing mixed weight precision between float8 and f16. It could be beneficial to test further.
There was a problem hiding this comment.
We can keep it to bfloat16 for now (as done in this PR) following the official recommendations. Then @Ednaordinary you could maybe help us opening a PR to the docs including your findings? WDYT?
| @@ -25,6 +25,50 @@ Make sure to check out the Schedulers [guide](../../using-diffusers/schedulers.m | |||
|
|
|||
There was a problem hiding this comment.
(One line above this) Only FlowMatchEulerDiscreteScheduler has invert_sigmas, so anything else wouldn't work as of now as I understand it
| pipe.enable_vae_tiling() | ||
|
|
||
| prompt = "Close-up of a chameleon's eye, with its scaly skin changing color. Ultra high resolution 4k." | ||
| frames = pipe(prompt, num_frames=84).frames[0] |
There was a problem hiding this comment.
Comments from @Ednaordinary are already great, so let's resolve them. Maybe we could add a section on how to reproduce some of their videos generated with the original inference code and params? I think most people would be interested in that.
Additionally, it seems like we should suggest using a maximum sequence length of 256?
#9769 (comment)
Already the case:
What does this PR do?
Update Mochi docs
Fixes # (issue)
Before submitting
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.