Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
chore(langchain): disable flaky tests
There appear to be stability issues with using snapshots and/or LangChain in general. There are failures in the mocked tests that look like: ``` builtins.AssertionError: assert 0 == 1 + where 0 = <MagicMock name='LLMObsSpanWriter().enqueue' id='127482146130048'>.call_count + where <MagicMock name='LLMObsSpanWriter().enqueue' id='127482146130048'> = <MagicMock name='LLMObsSpanWriter()' id='127482147073440'>.enqueue ``` as well as failures with snapshot based tests: ``` builtins.Failed: At request <Request GET /test/session/snapshot >: At snapshot (token='tests.contrib.langchain.test_langchain_community.test_lcel_chain_simple'): - Directory: /go/src/github.com/DataDog/apm-reliability/dd-trace-py/tests/snapshots - CI mode: 1 - Trace File: /go/src/github.com/DataDog/apm-reliability/dd-trace-py/tests/snapshots/tests.contrib.langchain.test_langchain_community.test_lcel_chain_simple.json - Stats File: /go/src/github.com/DataDog/apm-reliability/dd-trace-py/tests/snapshots/tests.contrib.langchain.test_langchain_community.test_lcel_chain_simple_tracestats.json At compare of 1 expected trace(s) to 0 received trace(s): Did not receive expected traces: 'langchain.request' ``` While we investigate a more stable method of testing it makes sense to disable the tests to avoid noise to our neighbours in the library :). DOWN WITH FLAKY TESTS
- Loading branch information