Langfuse integration

[masci]

Summary and motivation

Langfuse would be extremely helpful in monitoring and debugging Haystack pipelines, by visualising inputs and outputs for each component execution during a pipeline.run().

Detailed design

• The integration will use Haystack's tracing API
• A working MVP detailing the integration design is available here https://github.com/deepset-ai/haystack-core-integrations/tree/staging/integrations/langfuse

Checklist

If the request is accepted, ensure the following checklist is complete before closing this issue.

Tasks

Preview Give feedback

1. The code is documented with docstrings and was merged in the main branch
2. Docs are published at https://docs.haystack.deepset.ai/
3. There is a Github workflow running the tests for the integration nightly and at every PR
4. A label named like integration:<your integration name> has been added to this repo
5. The labeler.yml file has been updated
6. The package has been released on PyPI
7. An integration tile has been added to https://github.com/deepset-ai/haystack-integrations #709
   integration:langfuse +1
   
8. The integration has been listed in the Inventory section of this repo README
9. There is an example available to demonstrate the feature
10. The feature was announced through social media #710
    +0
    

[masci]

@deepset-ai/devrel FYI this will be scheduled for the upcoming sprint

[masci]

@vblagoje reopening as the task is not done, please make sure the tasklist is up to date!

[vblagoje]

Ok will do @masci - it got closed because of the fixes link on PR. All subtasks are finished except docs publishing (although they are completed) and social media announcements

[vblagoje]

@dfokina @annthurium When the respective sub tasks are completed would you please mark them on this issue

[dfokina]

The docs are live: https://docs.haystack.deepset.ai/docs/langfuseconnector

[caubechankiu]

The docs are live: https://docs.haystack.deepset.ai/docs/langfuseconnector

I followed the documentation but the result is

Here is my code

from haystack.dataclasses import ChatMessage
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.builders import DynamicChatPromptBuilder
from haystack import Pipeline
from haystack_integrations.components.connectors.langfuse import LangfuseConnector
from haystack import Pipeline
import os

os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-...."
os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-...."
os.environ["OPENAI_API_KEY"] = "sk-...."
os.environ["LANGFUSE_HOST"] = "https://cloud.langfuse.com"
os.environ["TOKENIZERS_PARALLELISM"] = "false"
os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"


pipe = Pipeline()
pipe.add_component("tracer", LangfuseConnector("Chat example"))
pipe.add_component("prompt_builder", DynamicChatPromptBuilder())
pipe.add_component("llm", OpenAIChatGenerator(model="gpt-3.5-turbo"))

pipe.connect("prompt_builder.prompt", "llm.messages")
messages = [
    ChatMessage.from_system("Always respond in German even if some input data is in other languages."),
    ChatMessage.from_user("Tell me about {{location}}"),
]

response = pipe.run(
    data={"prompt_builder": {"template_variables": {"location": "Berlin"}, "prompt_source": messages}}
)
print(response["llm"]["replies"][0])
print(response["tracer"]["trace_url"])
# ChatMessage(content='Berlin ist die Hauptstadt von Deutschland und zugleich eines der bekanntesten kulturellen Zentren Europas. Die Stadt hat eine faszinierende Geschichte, die bis in die Zeiten des Zweiten Weltkriegs und des Kalten Krieges zurückreicht. Heute ist Berlin für seine vielfältige Kunst- und Musikszene, seine historischen Stätten wie das Brandenburger Tor und die Berliner Mauer sowie seine lebendige Street-Food-Kultur bekannt. Berlin ist auch für seine grünen Parks und Seen beliebt, die den Bewohnern und Besuchern Raum für Erholung bieten.', role=<ChatRole.ASSISTANT: 'assistant'>, name=None, meta={'model': 'gpt-3.5-turbo-0125', 'index': 0, 'finish_reason': 'stop', 'usage': {'completion_tokens': 137, 'prompt_tokens': 29, 'total_tokens': 166}})
# https://cloud.langfuse.com/trace/YOUR_UNIQUE_IDENTIFYING_STRING

[vblagoje]

Hey @caubechankiu due to python class loading mechanism you need to import os first, set env values and then import everything else. Then it'll work. Also select the nodes below chat example (right panel) as the root doesn't have input only haystack.pipeline.run node and below. See below:

And let us know if adjusting these worked. Here is my pip install on fresh virtual env:

pip install git+https://github.com/deepset-ai/haystack.git langfuse-haystack

[caubechankiu]

Hey @caubechankiu due to python class loading mechanism you need to import os first, set env values and then import everything else. Then it'll work. Also select the nodes below chat example (right panel) as the root doesn't have input only haystack.pipeline.run node and below. See below:

And let us know if adjusting these worked. Here is my pip install on fresh virtual env:

pip install git+https://github.com/deepset-ai/haystack.git langfuse-haystack

Thank you! It worked.

[vblagoje]

@annthurium would you please close this issue after social media announcement

[annthurium]

yep!

[vblagoje]

Ok, I think we can finally close this one @annthurium @masci