PlayHT gRPC broken due to invalid language codes

[james-cohen]

Description

Environment

• pipecat-ai version: 0.0.52
• python version: 3.10
• OS: MacOS Sequoia 15.2

Issue description

We would like to keep using PlayHT's gRPC connection, so we are using the PlayHTHttpTTSService as instructed. However, even with default values and no modification, there is an error due to invalid language codes being passed. Even when trying to pass language codes or patching the underlying code, the same error persists. This occurs when using PlayHT 2.0 turbo which does not have language support and should be ignored, although the same issue occurs when passing Play3.0-mini-grpc as a model as well.

The issue seems to occur at this point in the pyht client code. Rather unhelpfully, hardcoding a valid value here e.g. Language.EN still causes an error, albeit a different one. This could therefore be an issue with pyht rather than pipecat.

Repro steps

Use the PlayHTHttpTTSService with a grpc model and default values:

        tts = PlayHTHttpTTSService(
            user_id=os.getenv("PLAYHT_USER_ID") or "",
            api_key=os.getenv("PLAYHT_API_KEY") or "",
            voice_url=tts_config["manifest_uri"],
            voice_engine="PlayHT2.0-turbo",
        )

Expected behavior

TTS works and produces streamed audio

Actual behavior

Error thrown due to invalid language.

Logs

2025-01-03 18:46:39.142 | DEBUG | pipecat.services.playht:run_tts:408 - Generating TTS: [Hi.]
2025-01-03 18:46:39.143 | ERROR | pipecat.services.playht:run_tts:444 - PlayHTHttpTTSService#0 error generating TTS: 'english'
2025-01-03 18:46:39.303 | DEBUG | pipecat.services.playht:run_tts:408 - Generating TTS: [ How are you today?]
2025-01-03 18:46:39.303 | ERROR | pipecat.services.playht:run_tts:444 - PlayHTHttpTTSService#0 error generating TTS: 'english'

[james-cohen]

I identified the cause of the subsequent issue and have opened a PR for a fix

[markbackman]

Thanks for flagging this. PR to fix this: #921.

I reached out to the PlayHT team about language missing from the PlayHT python client, as their documentation states that it's supported. For now, I'm removing language from the PlayHTHttpTTSService. If language support is required, the websocket service is available (and is generally recommended for lower latency performance!).

[james-cohen]

the websocket service is available (and is generally recommended for lower latency performance!)

Interesting, I was under the impression that gRPC was more performant? From my own experience of trying out both it seemed to be the case anyway. Is there any benchmarking available from PlayHT (or elsewhere) to demonstrate the differences?

[markbackman]

the websocket service is available (and is generally recommended for lower latency performance!)

Interesting, I was under the impression that gRPC was more performant? From my own experience of trying out both it seemed to be the case anyway. Is there any benchmarking available from PlayHT (or elsewhere) to demonstrate the differences?

We haven't done any benchmarking, but the websocket service should be more performant because the websocket service makes a permanent connection, which eliminates the overhead of establishing a new connection per request. I don't know anything about PlayHT's grpc connection, so that might be a question for the PlayHT team on which one they recommend for the lowest latency.

[james-cohen]

I will ask them, thank you.

Related, just trying out websockets now and it seems that interruptions are not handled consistently - sometimes the speech will stop and but more often it will continue until the generated speech has finished streaming, before the response to the interruption then continues immediately after.

Is this a known bug or something that requires specific handling? (I can open a separate issue if required)

EDIT: It seems that interruptions are handled the first time but then not after that, as if the original StartInterruptionFrame has not been flushed

[markbackman]

I will ask them, thank you.

Related, just trying out websockets now and it seems that interruptions are not handled consistently - sometimes the speech will stop and but more often it will continue until the generated speech has finished streaming, before the response to the interruption then continues immediately after.

Is this a known bug or something that requires specific handling? (I can open a separate issue if required)

EDIT: It seems that interruptions are handled the first time but then not after that, as if the original StartInterruptionFrame has not been flushed

That's a bug in the PlayHTTTSService implementation. We recently upgraded to the PlayHT v4 websocket implementation and it seems like there were changes to how the service operates. There's no documented way to stop the response. We could reconnect the websocket, but I'd prefer for a cancel message that can be pushed on the websocket. Let's see what the PlayHT says. We'll make an update once we hear back. Until then, I think the Http service via grpc sounds like a good option!

[markbackman]

I've fixed the language options issue in: #921. I'm closing this issue. I'll open another issue regarding the interruptions issue with PlayHTTTSService.

@james-cohen In talking with the PlayHT team, they pointed out that the grpc option for the PlayHTHttpsTTSService is only applicable if you're running on-prem. The available options are Play3.0-mini-http or Play3.0-mini-ws.