Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: warn if LangfuseTracer initialized without tracing enabled #1231

Merged
merged 2 commits into from
Dec 10, 2024

Conversation

isfuku
Copy link
Contributor

@isfuku isfuku commented Dec 8, 2024

Related Issues

Proposed Changes:

Add a warning during LangfuseTracer initialization in case tracing is disabled.

How did you test it?

Tested locally by running the tracer with LangufuseConnector in the following cases:

  • importing haystack.Pipeline before setting HAYSTACK_CONTENT_TRACING_ENABLED (warning raised)
  • importing haystack.Pipeline after setting HAYSTACK_CONTENT_TRACING_ENABLED (no warning)

Notes for the reviewer

I have not added unit tests because it requires to clear haystack modules and re-import it while having the appropriate HAYSTACK_CONTENT_TRACING_ENABLED env variable set. I though about adding it by removing haystack imports via sys.modules, and re-importing having HAYSTACK_CONTENT_TRACING_ENABLED=false for raising the warning, but apparently by doing so other tests fails, I think I would have to re-import the module while having HAYSTACK_CONTENT_TRACING_ENABLED=true. Let me know if adding a test for it is necessary.

Checklist

@isfuku isfuku requested a review from a team as a code owner December 8, 2024 16:27
@isfuku isfuku requested review from julian-risch and removed request for a team December 8, 2024 16:27
@CLAassistant
Copy link

CLAassistant commented Dec 8, 2024

CLA assistant check
All committers have signed the CLA.

Copy link
Member

@julian-risch julian-risch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@isfuku Thank you for contributing to Haystack! The changes look very good to me. Could you please just add a test before we merge? That test should check if the warning is triggered of we initialize the tracer while the env variable is not set to true. Should be something similar to

import logging

import pytest

...

def test_init_with_tracing_disabled(self, monkeypatch, caplog):
    monkeypatch.setenv("HAYSTACK_CONTENT_TRACING_ENABLED", “false”)
    with caplog.at_level(logging.WARNING):
        assert "tracing is disabled" in caplog.text

in integrations/langfuse/tests/test_tracer.py

@isfuku isfuku requested a review from julian-risch December 9, 2024 12:37
Copy link
Member

@julian-risch julian-risch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 👍 We could further extend the test with @pytest.mark.parametrize and also check that no warning is emitted if tracing is enabled. And we could create a helper function with the code that clears imported haystack modules so that it can be reused. Still, the code in its current shape is ready to be merged in my opinion.
Thank you and congratulations on your first contribution to Haystack @isfuku !

@julian-risch julian-risch merged commit d3677be into deepset-ai:main Dec 10, 2024
10 checks passed
@julian-risch
Copy link
Member

I'll release a new version of the integration tomorrow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Warn if LangfuseTracer initialized without HAYSTACK_CONTENT_TRACING_ENABLED
3 participants