Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Create new multipart encoder for each retry attempt #1114

Merged
merged 3 commits into from
Oct 23, 2024
Merged

fix: Create new multipart encoder for each retry attempt #1114

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c0d22f5
fix: Create new multipart encoder for each retry attempt
nfcampos Oct 23, 2024
6d74823
Fix double reqs
nfcampos Oct 23, 2024
4d3ed2f
0.1.137
nfcampos Oct 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 38 additions & 24 deletions python/langsmith/client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Client for interacting with the LangSmith API.

Check notice on line 1 in python/langsmith/client.py

View workflow job for this annotation

GitHub Actions / benchmark

Benchmark results 

.........................................
create_5_000_run_trees: Mean +- std dev: 564 ms +- 39 ms
.........................................
create_10_000_run_trees: Mean +- std dev: 1.12 sec +- 0.05 sec
.........................................
create_20_000_run_trees: Mean +- std dev: 1.11 sec +- 0.06 sec
.........................................
dumps_class_nested_py_branch_and_leaf_200x400: Mean +- std dev: 766 us +- 12 us
.........................................
dumps_class_nested_py_leaf_50x100: Mean +- std dev: 26.8 ms +- 0.5 ms
.........................................
dumps_class_nested_py_leaf_100x200: Mean +- std dev: 111 ms +- 2 ms
.........................................
dumps_dataclass_nested_50x100: Mean +- std dev: 27.1 ms +- 0.2 ms
.........................................
WARNING: the benchmark result may be unstable
* the standard deviation (6.33 ms) is 11% of the mean (57.3 ms)

Try to rerun the benchmark with more runs, values and/or loops.
Run 'python -m pyperf system tune' command to reduce the system jitter.
Use pyperf stats, pyperf dump and pyperf hist to analyze results.
Use --quiet option to hide these warnings.

dumps_pydantic_nested_50x100: Mean +- std dev: 57.3 ms +- 6.3 ms
.........................................
WARNING: the benchmark result may be unstable
* the standard deviation (28.0 ms) is 13% of the mean (212 ms)

Try to rerun the benchmark with more runs, values and/or loops.
Run 'python -m pyperf system tune' command to reduce the system jitter.
Use pyperf stats, pyperf dump and pyperf hist to analyze results.
Use --quiet option to hide these warnings.

dumps_pydanticv1_nested_50x100: Mean +- std dev: 212 ms +- 28 ms

Check notice on line 1 in python/langsmith/client.py

View workflow job for this annotation

GitHub Actions / benchmark

Comparison against main 

+------------------------------------+----------+------------------------+
| Benchmark                          | main     | changes                |
+====================================+==========+========================+
| dumps_pydantic_nested_50x100       | 60.1 ms  | 57.3 ms: 1.05x faster  |
+------------------------------------+----------+------------------------+
| dumps_pydanticv1_nested_50x100     | 221 ms   | 212 ms: 1.04x faster   |
+------------------------------------+----------+------------------------+
| dumps_class_nested_py_leaf_50x100  | 27.6 ms  | 26.8 ms: 1.03x faster  |
+------------------------------------+----------+------------------------+
| create_20_000_run_trees            | 1.13 sec | 1.11 sec: 1.02x faster |
+------------------------------------+----------+------------------------+
| dumps_dataclass_nested_50x100      | 27.5 ms  | 27.1 ms: 1.02x faster  |
+------------------------------------+----------+------------------------+
| dumps_class_nested_py_leaf_100x200 | 112 ms   | 111 ms: 1.01x faster   |
+------------------------------------+----------+------------------------+
| Geometric mean                     | (ref)    | 1.02x faster           |
+------------------------------------+----------+------------------------+

Benchmark hidden because not significant (3): create_5_000_run_trees, create_10_000_run_trees, dumps_class_nested_py_branch_and_leaf_200x400

Use the client to customize API keys / workspace ocnnections, SSl certs,
etc. for tracing.
Expand Down Expand Up @@ -1700,32 +1700,46 @@
# send the request
self._send_multipart_req(acc_parts, _context="; ".join(acc_context))

def _send_multipart_req(self, parts: MultipartParts, *, _context: str):
def _send_multipart_req(
self, parts: MultipartParts, *, _context: str, attempts: int = 3
):
for api_url, api_key in self._write_api_urls.items():
try:
encoder = MultipartEncoder(parts, boundary=BOUNDARY)
self.request_with_retries(
"POST",
f"{api_url}/runs/multipart",
request_kwargs={
"data": encoder,
"headers": {
**self._headers,
X_API_KEY: api_key,
"Content-Type": encoder.content_type,
},
},
to_ignore=(ls_utils.LangSmithConflictError,),
stop_after_attempt=3,
_context=_context,
)
except Exception as e:
for idx in range(1, attempts + 1):
try:
exc_desc_lines = traceback.format_exception_only(type(e), e)
exc_desc = "".join(exc_desc_lines).rstrip()
logger.warning(f"Failed to multipart ingest runs: {exc_desc}")
except Exception:
logger.warning(f"Failed to multipart ingest runs: {repr(e)}")
encoder = MultipartEncoder(parts, boundary=BOUNDARY)
self.request_with_retries(
"POST",
f"{api_url}/runs/multipart",
request_kwargs={
"data": encoder,
"headers": {
**self._headers,
X_API_KEY: api_key,
"Content-Type": encoder.content_type,
},
},
stop_after_attempt=1,
_context=_context,
)
break
except ls_utils.LangSmithConflictError:
break
except (
ls_utils.LangSmithConnectionError,
ls_utils.LangSmithRequestTimeout,
ls_utils.LangSmithAPIError,
) as exc:
if idx == attempts:
logger.warning(f"Failed to multipart ingest runs: {exc}")
else:
continue
except Exception as e:
try:
exc_desc_lines = traceback.format_exception_only(type(e), e)
exc_desc = "".join(exc_desc_lines).rstrip()
logger.warning(f"Failed to multipart ingest runs: {exc_desc}")
except Exception:
logger.warning(f"Failed to multipart ingest runs: {repr(e)}")

def update_run(
self,
Expand Down
2 changes: 1 addition & 1 deletion python/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "langsmith"
version = "0.1.136"
version = "0.1.137"
description = "Client library to connect to the LangSmith LLM Tracing and Evaluation Platform."
authors = ["LangChain <[email protected]>"]
license = "MIT"
Expand Down
Loading