Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[design] Speech to text and text to speech nodes in pipelines #1136

Open
snopoke opened this issue Feb 4, 2025 · 0 comments
Open

[design] Speech to text and text to speech nodes in pipelines #1136

snopoke opened this issue Feb 4, 2025 · 0 comments
Labels
pipelines Issue is related to pipelines

Comments

@snopoke
Copy link
Collaborator

snopoke commented Feb 4, 2025

Related to #1137

Write up a proposal for how to handle TTS and STT in pipelines.

In legacy bots it is handled in the Channels (https://github.ctom/dimagi/open-chat-studio/blob/08b37e392b229a4d6ff432347da403679881def0/apps/chat/channels.py#L430 &

def _reply_voice_message(self, text: str):
).

This approach still works with pipelines but the pipeline should be responsible for handling this to make it more flexible - maybe you want to use Whisper or maybe you are using a multi-modal modal that will accept the attachment directly

@snopoke snopoke added the pipelines Issue is related to pipelines label Feb 4, 2025
@snopoke snopoke moved this to Prioritized in OpenChatStudio Feb 4, 2025
@snopoke snopoke changed the title Speech to text and text to speech nodes in pipelines [design] Speech to text and text to speech nodes in pipelines Feb 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pipelines Issue is related to pipelines
Projects
Status: Prioritized
Development

No branches or pull requests

1 participant