Skip to content

Commit

Permalink
docs: add scores and sessions to tracing data model
Browse files Browse the repository at this point in the history
  • Loading branch information
marcklingen committed Dec 11, 2024
1 parent 52a167d commit b578461
Show file tree
Hide file tree
Showing 3 changed files with 134 additions and 48 deletions.
1 change: 1 addition & 0 deletions pages/docs/_meta.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ export default {
title: "Tracing",
},
tracing: "Introduction",
"tracing-data-model": "Data Model",
"get-started": "Quickstart",
"tracing-features": "Tracing Features",
sdk: "SDKs",
Expand Down
126 changes: 126 additions & 0 deletions pages/docs/tracing-data-model.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
---
title: Tracing Data Model in Langfuse
description: Langfuse (open source) helps you trace and analyze LLM applications. Learn how traces and observations are structured in Langfuse.
---

# Tracing Data Model

[Tracing](/docs/tracing) in Langfuse is a way to log and analyze the execution of your LLM applications. The following reference provides a detailed overview of the data model used. It is inspired by OpenTelemetry.

## Traces and Observations

A `trace` typically represents a single request or operation.
It contains the overall input and output of the function, as well as metadata about the request, such as the user, the session, and tags.
Usually, _a trace corresponds to a single api call of an application_.

Each trace can contain multiple `observations` to log the individual steps of the execution.

- Observations are of different types:
- `Events` are the basic building blocks. They are used to track discrete events in a trace.
- `Spans` represent durations of units of work in a trace.
- `Generations` are spans used to log generations of AI models. They contain additional attributes about the model, the prompt, and the completion. For generations, [token usage and costs](/docs/model-usage-and-cost) are automatically calculated.
- Observations can be nested.

<div className="grid grid-cols-2 mt-6 gap-2 md:gap-4">

<div className="border rounded py-4 text-center bg-card">

**Hierarchical structure of traces in Langfuse**

```mermaid
classDiagram
Trace "1" *-- "n" Observation
Observation <|-- Event
Observation <|-- Span
Observation <|-- Generation
Observation o-- Observation: Nesting
```

</div>

<div className="border rounded py-4 text-center block dark:hidden bg-card">

**Example trace in Langfuse UI**

![Trace in Langfuse UI](/images/docs/tracing-observation-tree-light.png)

</div>

<div className="border rounded py-4 text-center bg-background hidden dark:block">

**Example trace in Langfuse UI**

![Trace in Langfuse UI](/images/docs/tracing-observation-tree-dark.png)

</div>

</div>

## Sessions

Optionally, traces can be grouped into [sessions](/docs/tracing-features/sessions).
Sessions are used to group traces that are part of the same user interaction.
A common example is a thread in a chat interface.

Please refer to the [Sessions](/docs/tracing-features/sessions) documentation to add sessions to your traces.

<div className="grid grid-cols-2 mt-6 gap-2 md:gap-4">

<div className="border rounded py-4 text-center bg-card">

**Optionally, sessions aggregate traces**

```mermaid
classDiagram
Session "1" o-- "n" Trace
```

</div>

<div className="border rounded pt-4 px-2 pb-2 text-center block bg-card">

**Example session in Langfuse UI**

<Frame border fullWidth>
![Session view](/images/docs/session.png)
</Frame>

</div>

</div>

## Scores

Traces and observations can be evaluated using [scores](/docs/scores). Scores are flexible objects that store evaluation metrics and can be:

- Numeric, categorical, or boolean values
- Associated with a trace (required)
- Linked to a specific observation (optional)
- Annotated with comments for additional context
- Validated against a score configuration schema (optional)

Please refer to the [scores documentation](/docs/scores) to get started. For more details on score types and attributes, refer to the [score data model](/docs/scores/data-model) documentation.

<div className="border rounded py-4 mt-6 text-center bg-card">

**Optionally, scores can add evaluation metrics to traces**

```mermaid
classDiagram
namespace Evaluation {
class Score
class ScoreConfig
}
namespace Tracing {
class Trace
class Observation
}
Trace *-- Observation
Score --> Trace
Score --> Observation : optional, reference observation within a trace
Score --> ScoreConfig : optional, ensure schema
```

</div>
55 changes: 7 additions & 48 deletions pages/docs/tracing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -34,58 +34,11 @@ import TracingOverview from "@/components-mdx/tracing-overview-gifs.mdx";

<TracingOverview />

## Introduction to Observability & Traces in Langfuse

A trace in Langfuse consists of the following objects:

- A `trace` typically represents a single request or operation. It contains the overall input and output of the function, as well as metadata about the request, such as the user, the session, and tags.
- Each trace can contain multiple `observations` to log the individual steps of the execution.
- Observations are of different types:
- `Events` are the basic building blocks. They are used to track discrete events in a trace.
- `Spans` represent durations of units of work in a trace.
- `Generations` are spans used to log generations of AI models. They contain additional attributes about the model, the prompt, and the completion. For generations, [token usage and costs](/docs/model-usage-and-cost) are automatically calculated.
- Observations can be nested.

<div className="grid grid-cols-2 mt-4 gap-2 md:gap-4">

<div className="border rounded py-4 text-center bg-card">

**Hierarchical structure of traces in Langfuse**

```mermaid
classDiagram
Trace "1" *-- "n" Observation
Observation <|-- Event
Observation <|-- Span
Observation <|-- Generation
Observation o-- Observation: Nesting
```

</div>

<div className="border rounded py-4 text-center block dark:hidden bg-card">

**Example trace in Langfuse UI**

![Trace in Langfuse UI](/images/docs/tracing-observation-tree-light.png)

</div>

<div className="border rounded py-4 text-center bg-background hidden dark:block">

**Example trace in Langfuse UI**

![Trace in Langfuse UI](/images/docs/tracing-observation-tree-dark.png)

</div>

</div>

## Get Started

Follow the quickstart to add Langfuse tracing to your LLM app.

import { Rocket, Joystick } from "lucide-react";
import { Rocket, BookOpen, Joystick } from "lucide-react";

<Cards num={3}>
<Card
Expand All @@ -94,6 +47,12 @@ import { Rocket, Joystick } from "lucide-react";
href="/docs/get-started"
arrow
/>
<Card
icon={<BookOpen size="24" />}
title="Data Model"
href="/docs/tracing-data-model"
arrow
/>
<Card
icon={<Joystick size="24" />}
title="Interactive demo"
Expand Down

0 comments on commit b578461

Please sign in to comment.