Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python[minor]: pytest integration #1362

Merged
merged 65 commits into from
Jan 21, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
ed71173
rfc: manually set test case inputs/outputs
baskaryan Dec 30, 2024
80cbf66
fmt
baskaryan Dec 31, 2024
b33f75a
fmt
baskaryan Jan 2, 2025
37f43cb
fmt
baskaryan Jan 2, 2025
cf82fd6
Merge branch 'main' into bagatur/rfc_set_test_vals
baskaryan Jan 2, 2025
80d4205
Merge branch 'main' into bagatur/rfc_set_test_vals
baskaryan Jan 3, 2025
fa8882f
Merge branch 'main' into bagatur/rfc_set_test_vals
baskaryan Jan 3, 2025
6b999b4
fmt
baskaryan Jan 3, 2025
cbbf3a3
fmt
baskaryan Jan 3, 2025
a61d7d0
fmt
baskaryan Jan 3, 2025
cf37a91
fmt
baskaryan Jan 3, 2025
c9addf0
fmt
baskaryan Jan 3, 2025
d814ec5
Merge branch 'main' into bagatur/rfc_set_test_vals
baskaryan Jan 6, 2025
8700d4b
fmt
baskaryan Jan 6, 2025
376a645
fmt
baskaryan Jan 6, 2025
81e41f4
Merge branch 'main' into bagatur/rfc_set_test_vals
baskaryan Jan 8, 2025
d5d4ebb
rc release
baskaryan Jan 8, 2025
6940ace
Merge branch 'main' into bagatur/rfc_set_test_vals
baskaryan Jan 8, 2025
d3ed9c6
Merge branch 'main' into bagatur/rfc_set_test_vals
baskaryan Jan 8, 2025
bd72391
add better error messaging
isahers1 Jan 8, 2025
542d76e
Merge branch 'main' into bagatur/rfc_set_test_vals
baskaryan Jan 8, 2025
aaf41c9
fmt
baskaryan Jan 8, 2025
d7beea8
wait example updates
baskaryan Jan 8, 2025
c4de666
Merge branch 'bagatur/rfc_set_test_vals' of github.com:langchain-ai/l…
baskaryan Jan 8, 2025
44897c7
fmt
baskaryan Jan 9, 2025
2192c09
fmt
baskaryan Jan 9, 2025
7b433a2
fmt
baskaryan Jan 9, 2025
48670e3
rc2
baskaryan Jan 9, 2025
78eeabf
fmt
baskaryan Jan 9, 2025
e1c9d7c
fmt
baskaryan Jan 9, 2025
400e38b
ptyest plugin
baskaryan Jan 10, 2025
9250bbf
Merge branch 'main' into bagatur/rfc_set_test_vals
baskaryan Jan 10, 2025
414ef69
Merge branch 'main' into bagatur/rfc_set_test_vals
baskaryan Jan 10, 2025
1446452
fmt
baskaryan Jan 11, 2025
790617d
Merge branch 'main' into bagatur/rfc_set_test_vals
baskaryan Jan 11, 2025
37ceca0
fmt
baskaryan Jan 13, 2025
1fca587
rc6
baskaryan Jan 13, 2025
88e6f63
fmt
baskaryan Jan 13, 2025
43509c2
Merge branch 'main' into bagatur/rfc_set_test_vals
baskaryan Jan 14, 2025
233ae72
update table
baskaryan Jan 14, 2025
996e21f
fmt
baskaryan Jan 15, 2025
d79bec9
Merge branch 'main' into bagatur/rfc_set_test_vals
baskaryan Jan 15, 2025
99069fb
group by test suite
baskaryan Jan 15, 2025
6076c29
fmt
baskaryan Jan 15, 2025
d4d1695
fmt
baskaryan Jan 15, 2025
441a8b1
fmt
baskaryan Jan 15, 2025
05dbf16
fmt
baskaryan Jan 15, 2025
82cd582
rc7
baskaryan Jan 15, 2025
e56a68f
fmt
baskaryan Jan 15, 2025
d049218
update api ref
baskaryan Jan 16, 2025
e400c93
fix LANGSMITH_TEST_TRACKING=false
baskaryan Jan 16, 2025
5752078
fmt
baskaryan Jan 16, 2025
8f336f4
fix entrypoint
baskaryan Jan 16, 2025
a5ad0ef
rm script
baskaryan Jan 17, 2025
6aa0062
rename plugin
baskaryan Jan 17, 2025
65dbe26
rename output plugin
baskaryan Jan 17, 2025
3773cb8
rc12
baskaryan Jan 17, 2025
48ff3c9
fix val logging
baskaryan Jan 18, 2025
e7675c1
rc13
baskaryan Jan 19, 2025
2bc8ad5
rc14
baskaryan Jan 20, 2025
66c7799
rc15
baskaryan Jan 20, 2025
ce08bfa
cr
baskaryan Jan 20, 2025
b2f6f87
merge
baskaryan Jan 20, 2025
4084685
nit
baskaryan Jan 20, 2025
341798c
merge
baskaryan Jan 21, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
ptyest plugin
  • Loading branch information
baskaryan committed Jan 10, 2025
commit 400e38b1742f3b7430193755e7aad9ef54a2492a
27 changes: 27 additions & 0 deletions python/langsmith/pytest_plugin.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
"""LangSmith Pytest hooks."""

import pytest

from langsmith.testing._internal import test as ls_test


def pytest_configure(config):
"""Register the 'langsmith' marker."""
config.addinivalue_line(
"markers", "langsmith: mark test to be tracked in LangSmith"
)


@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_call(item):
"""Apply LangSmith tracking to tests marked with @pytest.mark.langsmith."""
marker = item.get_closest_marker("langsmith")
if marker:
# Get marker kwargs if any (e.g.,
# @pytest.mark.langsmith(output_keys=["expected"]))
kwargs = marker.kwargs if marker else {}
# Wrap the test function with our test decorator
original_func = item.obj
item.obj = ls_test(**kwargs)(original_func)

yield
2 changes: 0 additions & 2 deletions python/langsmith/testing/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,10 @@
log_inputs,
log_outputs,
log_reference_outputs,
test,
trace_feedback,
)

__all__ = [
"test",
"log_inputs",
"log_outputs",
"log_reference_outputs",
Expand Down
Loading
Loading