Voice Control ideas (brainstorming)

[MithrilMan]

Since this is a diffusion model, I'm wondering if would be possible to implement something like ControlNet did for StableDiffusion/Flux

Basically is it possible to build models that drive the audio generation by, for example, extracting the emotion from a voice reference (different from the reference used to "inpaint"), or the roughness, etc...

I'm not asking now to implement this but I just wanted to understand the possibilities given by a diffusion model for voices

[SWivid]

yes, possible. the reason we are open-sourcing is exactly to make it easier for all people to take a try for any idea