Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Langfuse integration: completionStartedAt is not logged, required for streaming metrics like TTFT and TPM/TPS #1261

Open
marcklingen opened this issue Dec 23, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@marcklingen
Copy link
Contributor

Describe the bug

When using the Langfuse integration, time to first token metrics are not available (the report I received mentioned Azure OpenAI but this might apply to all LLMs).

Langfuse supports time to first token via the “completionStartedAt” attribute on generations.

To Reproduce

Use Langfuse integration with AzureOpenAI in Haystack.

Describe your environment (please complete the following information):

Reported by user. Can try to get details on this if helpful.

@marcklingen marcklingen added the bug Something isn't working label Dec 23, 2024
@marcklingen
Copy link
Contributor Author

This is a bit vague as it was reported by a user but I assume the attribute is not set at all and thus it could be sufficiently detailed.

I think this would be a great addition to the integration.

@marcklingen marcklingen changed the title Langfuse integration: Missing time to first token for AzureOpenAI Langfuse integration: completionStartedAt is not logged, required for streaming metrics like TTFT and TPM/TPS Dec 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant