DOC: <Issue related to /tutorials/Developers/backtesting>

[labdmitriy]

Hi,

I tried to reproduce the example from the documentation for backtesting and also Backtesting | LangSmith Evaluations - Part 19, but each time I have the following error during convert_runs_to_test invocation:

AttributeError                            Traceback (most recent call last)
Cell In[18], line 27
     24 dataset_name = f'{project_name}-backtesting {start_time.strftime("%Y-%m-%d")}-{end_time.strftime("%Y-%m-%d")}'
     25 # This converts the runs to a dataset + experiment
     26 # It does not actually invoke your model
---> 27 convert_runs_to_test(
     28     prod_runs,
     29     # Name of the resulting dataset
     30     dataset_name=dataset_name,
     31     # Whether to include the run outputs as reference[/ground](http://localhost:8888/ground) truth
     32     include_outputs=False,
     33     # Whether to include the full traces in the resulting experiment
     34     # (default is to just include the root run)
     35 
     36     # AttributeError: 'NoneType' object has no attribute 'items'
     37     load_child_runs=True,
     38 )

File [.../lib/python3.11/site-packages/langsmith/_internal/_beta_decorator.py:19](.../lib/python3.11/site-packages/langsmith/_internal/_beta_decorator.py#line=18), in warn_beta.<locals>.wrapper(*args, **kwargs)
     16 @functools.wraps(func)
     17 def wrapper(*args, **kwargs):
     18     _warn_once(f"Function {func.__name__} is in beta.")
---> 19     return func(*args, **kwargs)

File [.../lib/python3.11/site-packages/langsmith/beta/_evals.py:165](.../lib/python3.11/site-packages/langsmith/beta/_evals.py#line=164), in convert_runs_to_test(runs, dataset_name, test_project_name, client, load_child_runs, include_outputs)
    155 project = client.create_project(
    156     project_name=test_project_name,
    157     reference_dataset_id=ds.id,
   (...)
    161     },
    162 )
    164 for new_run in to_create:
--> 165     client.create_run(**new_run, project_name=test_project_name)
    167 _ = client.update_project(
    168     project.id, end_time=datetime.datetime.now(tz=datetime.timezone.utc)
    169 )
    170 return project

File [.../lib/python3.11/site-packages/langsmith/client.py:1277](.../lib/python3.11/site-packages/langsmith/client.py#line=1276), in Client.create_run(self, name, inputs, run_type, project_name, revision_id, **kwargs)
   1275 if not self._filter_for_sampling([run_create]):
   1276     return
-> 1277 run_create = self._run_transform(run_create, copy=True)
   1278 if revision_id is not None:
   1279     run_create["extra"]["metadata"]["revision_id"] = revision_id

File [.../lib/python3.11/site-packages/langsmith/client.py:1181](.../lib/python3.11/site-packages/langsmith/client.py#line=1180), in Client._run_transform(self, run, update, copy)
   1177         run_create = {k: v for k, v in run_create.items() if k != "serialized"}
   1178     else:
   1179         # Drop graph
   1180         serialized = {
-> 1181             k: v for k, v in run_create["serialized"].items() if k != "graph"
   1182         }
   1183         run_create = {**run_create, "serialized": serialized}
   1185 return run_create

AttributeError: 'NoneType' object has no attribute 'items'

When I set load_child_runs=False then there are no errors, but the dataset will be created empty without any runs.

Thank you.