[Bug]: Execution Blocked in terminate() Function Due to Infinite Loop in DeepgramTranscriber

[cyrilS-dev]

Brief Description

In the terminate() function within vocode/streaming/streaming_conversation.py, after the log entry Terminating events Task, the rest of the function does not execute. The logs indicate that the loop in DeepgramTranscriber runs indefinitely, preventing further execution.

Actual Behavior

The terminate function stalls after attempting to terminate the events task. The Deepgram connection enters an infinite loop of restarts, as shown in the logs.

Relevant Logs:

2024-09-03 19:19:05.023 | DEBUG    | vocode.streaming.streaming_conversation:process:577 - Message sent: Au revoir !
2024-09-03 19:19:05.076 | DEBUG    | vocode.streaming.streaming_conversation:terminate:1007 - Terminating check_for_idle Task
2024-09-03 19:19:05.076 | DEBUG    | vocode.streaming.streaming_conversation:terminate:1010 - Terminating events Task
2024-09-03 19:19:10.019 | DEBUG    | vocode.streaming.transcriber.deepgram_transcriber:sender:419 - Terminating Deepgram transcriber sender
2024-09-03 19:19:17.081 | INFO     | vocode.streaming.transcriber.deepgram_transcriber:receiver:458 - Ignoring deepgram response type: Metadata
2024-09-03 19:19:17.085 | DEBUG    | vocode.streaming.transcriber.deepgram_transcriber:receiver:435 - Got error received 1011 (internal error) Deepgram did not receive audio data or a text message within the timeout window. See https://dpgr.am/net0001; then sent 1011 (internal error) Deepgram did not receive audio data or a text message within the timeout window. See https://dpgr.am/net0001 in Deepgram receiver
2024-09-03 19:19:17.085 | DEBUG    | vocode.streaming.transcriber.deepgram_transcriber:receiver:528 - Terminating Deepgram transcriber receiver
2024-09-03 19:19:17.085 | DEBUG    | vocode.streaming.transcriber.deepgram_transcriber:_run_loop:176 - Deepgram connection died, restarting, num_restarts: 1
2024-09-03 19:19:17.085 | INFO     | vocode.streaming.transcriber.deepgram_transcriber:process:394 - Connecting to Deepgram at wss://api.deepgram.com/v1/listen?encoding=linear16&sample_rate=48000&channels=1&interim_results=true&language=fr&model=nova-2&punctuate=true
2024-09-03 19:19:22.386 | DEBUG    | vocode.streaming.transcriber.deepgram_transcriber:sender:419 - Terminating Deepgram transcriber sender
2024-09-03 19:19:29.386 | DEBUG    | vocode.streaming.transcriber.deepgram_transcriber:receiver:435 - Got error received 1011 (internal error) Deepgram did not receive audio data or a text message within the timeout window. See https://dpgr.am/net0001; then sent 1011 (internal error) Deepgram did not receive audio data or a text message within the timeout window. See https://dpgr.am/net0001 in Deepgram receiver
2024-09-03 19:19:29.386 | DEBUG    | vocode.streaming.transcriber.deepgram_transcriber:receiver:528 - Terminating Deepgram transcriber receiver
2024-09-03 19:19:29.386 | DEBUG    | vocode.streaming.transcriber.deepgram_transcriber:_run_loop:176 - Deepgram connection died, restarting, num_restarts: 2
2024-09-03 19:19:29.386 | INFO     | vocode.streaming.transcriber.deepgram_transcriber:process:394 - Connecting to Deepgram at wss://api.deepgram.com/v1/listen?encoding=linear16&sample_rate=48000&channels=1&interim_results=true&language=fr&model=nova-2&punctuate=true
2024-09-03 19:19:34.629 | DEBUG    | vocode.streaming.transcriber.deepgram_transcriber:sender:419 - Terminating Deepgram transcriber sender
2024-09-03 19:19:41.630 | DEBUG    | vocode.streaming.transcriber.deepgram_transcriber:receiver:435 - Got error received 1011 (internal error) Deepgram did not receive audio data or a text message within the timeout window. See https://dpgr.am/net0001; then sent 1011 (internal error) Deepgram did not receive audio data or a text message within the timeout window. See https://dpgr.am/net0001 in Deepgram receiver
2024-09-03 19:19:41.630 | DEBUG    | vocode.streaming.transcriber.deepgram_transcriber:receiver:528 - Terminating Deepgram transcriber receiver
2024-09-03 19:19:41.630 | DEBUG    | vocode.streaming.transcriber.deepgram_transcriber:_run_loop:176 - Deepgram connection died, restarting, num_restarts: 3
2024-09-03 19:19:41.630 | INFO     | vocode.streaming.transcriber.deepgram_transcriber:process:394 - Connecting to Deepgram at wss://api.deepgram.com/v1/listen?encoding=linear16&sample_rate=48000&channels=1&interim_results=true&language=fr&model=nova-2&punctuate=true
2024-09-03 19:19:46.861 | DEBUG    | vocode.streaming.transcriber.deepgram_transcriber:sender:419 - Terminating Deepgram transcriber sender
2024-09-03 19:19:53.861 | DEBUG    | vocode.streaming.transcriber.deepgram_transcriber:receiver:435 - Got error received 1011 (internal error) Deepgram did not receive audio data or a text message within the timeout window. See https://dpgr.am/net0001; then sent 1011 (internal error) Deepgram did not receive audio data or a text message within the timeout window. See https://dpgr.am/net0001 in Deepgram receiver
2024-09-03 19:19:53.862 | DEBUG    | vocode.streaming.transcriber.deepgram_transcriber:receiver:528 - Terminating Deepgram transcriber receiver
2024-09-03 19:19:53.862 | DEBUG    | vocode.streaming.transcriber.deepgram_transcriber:_run_loop:176 - Deepgram connection died, restarting, num_restarts: 4
2024-09-03 19:19:53.862 | INFO     | vocode.streaming.transcriber.deepgram_transcriber:process:394 - Connecting to Deepgram at wss://api.deepgram.com/v1/listen?encoding=linear16&sample_rate=48000&channels=1&interim_results=true&language=fr&model=nova-2&punctuate=true
2024-09-03 19:19:58.980 | DEBUG    | vocode.streaming.transcriber.deepgram_transcriber:sender:419 - Terminating Deepgram transcriber sender

LLM

None

Transcription Services

None

Synthesis Services

None

Telephony Services

None

Conversation Type and Platform

No response

Steps to Reproduce

Trigger a conversation termination that calls the terminate function.
Observe the logs and note that after Terminating events Task, no further log entries are produced from terminate.
The logs from DeepgramTranscriber indicate repeated attempts to restart the connection without success.

Expected Behavior

The terminate function should complete its execution, including all tasks such as tearing down the synthesizer, terminating the agent, output device, and workers.

Screenshots

No response

[cyrilS-dev]

@arpagon @ajar98

[akshaysinghas]

+1 faced the same issue

[cyrilS-dev]

@akshaysinghas I'm not sure if this project is still maintained

[akshaysinghas]

@cyrilS-dev looks like its not.

Any other alternative to this that you have explored? I found this one quite decent in terms of functionality but maintenance is must. Hope community support it or we are working on a wrong project.

[cyrilS-dev]

@akshaysinghas I couldn’t agree more that we might be working on the wrong project. I’ve started exploring Pipecat AI, it’s a solid and serious project. Its features are on par with, if not better than Vocode.

[akshaysinghas]

Oh. Cool. Thanks for introducing that to me. I will also explore.
Thanks @cyrilS-dev

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[brown532]

Hi @cyrilS-dev, I've also been working on Vocode and ran into the same issue - a blocking terminate function for Deepgram (particularly, the receiver function doesn't end).
Has anyone been able to find a solution for this?