-
Notifications
You must be signed in to change notification settings - Fork 67
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'main' into dependabot/npm_and_yarn/react-syntax-highlig…
…hter-15.6.1
- Loading branch information
Showing
204 changed files
with
18,040 additions
and
8,378 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,15 @@ | ||
| # http://editorconfig.org | ||
|
|
||
| root = true | ||
|
|
||
| [*] | ||
| charset = utf-8 | ||
| indent_style = space | ||
| indent_size = 2 | ||
| end_of_line = lf | ||
| insert_final_newline = true | ||
| trim_trailing_whitespace = true | ||
|
|
||
| # Specific settings for the .mdx file you want to format as JavaScript | ||
| [pages/_app.mdx] | ||
| language = javascript |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| name: "CI" | ||
|
|
||
| on: | ||
| pull_request: | ||
| merge_group: | ||
|
|
||
| jobs: | ||
| build: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v3 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - uses: pnpm/action-setup@v4 | ||
| id: pnpm-install | ||
| with: | ||
| version: 9.5.0 | ||
| run_install: false | ||
|
|
||
| - uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: 20 | ||
|
|
||
| - run: pnpm install | ||
|
|
||
| - name: Build next.js app | ||
| # change this if your site requires a custom build command | ||
| run: pnpm build |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| <Tabs items={["Python", "JS/TS", "UI"]}> | ||
| <Tab> | ||
|
|
||
| ```python | ||
| langfuse.create_dataset_item( | ||
| dataset_name="<dataset_name>", | ||
| # any python object or value, optional | ||
| input={ | ||
| "text": "hello world" | ||
| }, | ||
| # any python object or value, optional | ||
| expected_output={ | ||
| "text": "hello world" | ||
| }, | ||
| # metadata, optional | ||
| metadata={ | ||
| "model": "llama3", | ||
| } | ||
| ) | ||
| ``` | ||
|
|
||
| _See [low-level SDK](/docs/sdk/python/low-level-sdk) docs for details on how to initialize the Python client._ | ||
|
|
||
| </Tab> | ||
| <Tab> | ||
|
|
||
| ```ts | ||
| langfuse.createDatasetItem({ | ||
| datasetName: "<dataset_name>", | ||
| // any JS object or value | ||
| input: { | ||
| text: "hello world", | ||
| }, | ||
| // any JS object or value, optional | ||
| expectedOutput: { | ||
| text: "hello world", | ||
| }, | ||
| // metadata, optional | ||
| metadata: { | ||
| model: "llama3", | ||
| }, | ||
| }); | ||
| ``` | ||
|
|
||
| </Tab> | ||
| <Tab> | ||
|
|
||
| Datasets > Items: `+ New item` | ||
|
|
||
| </Tab> | ||
| </Tabs> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| <Tabs items={["Python", "JS/TS", "UI"]}> | ||
| <Tab> | ||
|
|
||
| ```python | ||
| langfuse.create_dataset( | ||
| name="<dataset_name>", | ||
| # optional description | ||
| description="My first dataset", | ||
| # optional metadata | ||
| metadata={ | ||
| "author": "Alice", | ||
| "date": "2022-01-01", | ||
| "type": "benchmark" | ||
| } | ||
| ) | ||
| ``` | ||
|
|
||
| _See [low-level SDK](/docs/sdk/python/low-level-sdk) docs for details on how to initialize the Python client._ | ||
|
|
||
| </Tab> | ||
| <Tab> | ||
|
|
||
| ```ts | ||
| langfuse.createDataset({ | ||
| name: "<dataset_name>", | ||
| // optional description | ||
| description: "My first dataset", | ||
| // optional metadata | ||
| metadata: { | ||
| author: "Alice", | ||
| date: "2022-01-01", | ||
| type: "benchmark", | ||
| }, | ||
| }); | ||
| ``` | ||
|
|
||
| </Tab> | ||
| <Tab> | ||
|
|
||
| Datasets: `+ New dataset` | ||
|
|
||
| </Tab> | ||
| </Tabs> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| import { CloudflareVideo } from "@/components/Video"; | ||
|
|
||
| <Tabs items={["Manage Datasets", "Compare experiments", "Add new items from Langfuse Traces"]}> | ||
| <Tab> | ||
|
|
||
| Collaboratively manage datasets via UI, API, or SDKs. | ||
|
|
||
| <CloudflareVideo | ||
| videoId="68748382151bd71f08dcface0649fbc2" | ||
| aspectRatio={16 / 9} | ||
| gifStyle | ||
| /> | ||
|
|
||
| </Tab> | ||
| <Tab> | ||
|
|
||
| Run experiments on datasets. Compare metrics across experiments, and see results side-by-side. | ||
|
|
||
| <CloudflareVideo | ||
| videoId="f8f2cf7ff86f2b54d1b90c0921d2c7e9" | ||
| aspectRatio={16 / 9} | ||
| gifStyle | ||
| /> | ||
|
|
||
| </Tab> | ||
| <Tab> | ||
|
|
||
| Directly add new items from production traces to datasets to improve your datasets over time. | ||
|
|
||
| <CloudflareVideo | ||
| videoId="0d71bdd2fc094c5ac4ddebaf44757493" | ||
| aspectRatio={16 / 9} | ||
| gifStyle | ||
| /> | ||
|
|
||
| </Tab> | ||
| </Tabs> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| import { CloudflareVideo } from "@/components/Video"; | ||
|
|
||
| <Tabs items={["Analytics", "User Feedback", "LLM-as-a-Judge", "Annotation Queue", "Custom"]}> | ||
| <Tab> | ||
|
|
||
| Plot evaluation results in the Langfuse Dashboard. | ||
|
|
||
| <CloudflareVideo | ||
| videoId="3c331720f6c3a90c83cbaaa5d81b29af" | ||
| aspectRatio={16 / 9} | ||
| gifStyle | ||
| /> | ||
|
|
||
| </Tab> | ||
| <Tab> | ||
|
|
||
| Collect feedback from your users. Can be captured in the frontend via our Browser SDK, server-side via the SDKs or API. Video includes example application. | ||
|
|
||
| <CloudflareVideo | ||
| videoId="e36f0a43d27bf733122d639128677f82" | ||
| aspectRatio={16 / 9} | ||
| gifStyle | ||
| /> | ||
|
|
||
| </Tab> | ||
| <Tab> | ||
|
|
||
| Run fully managed LLM-as-a-judge evaluations on production or development traces. Can be applied to any step within your application for step-wise evaluations. | ||
|
|
||
| <CloudflareVideo | ||
| videoId="0c3797a6042c29dbcd01e6abbcd3bbe6" | ||
| aspectRatio={16 / 9} | ||
| gifStyle | ||
| /> | ||
|
|
||
| </Tab> | ||
| <Tab> | ||
|
|
||
| Baseline your evaluation workflow with human annotations via Annotation Queues. | ||
|
|
||
| <CloudflareVideo | ||
| videoId="0154084df3c8ae0de290c60720dd1fb2" | ||
| aspectRatio={16 / 9} | ||
| gifStyle | ||
| /> | ||
|
|
||
| </Tab> | ||
| <Tab> | ||
|
|
||
| Add custom evaluation results, supports numeric, boolean and categorical values. | ||
|
|
||
| ```bash | ||
| POST /api/public/scores | ||
| ``` | ||
|
|
||
| Add scores via Python or JS SDK. | ||
|
|
||
| ```python filename="Example (Python)" | ||
| langfuse.score( | ||
| trace_id="123", | ||
| name="my_custom_evaluator", | ||
| value=0.5, | ||
| ) | ||
| ``` | ||
|
|
||
| </Tab> | ||
| </Tabs> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| <Tabs items={["Images", "Audio", "Attachments"]}> | ||
| <Tab> | ||
| <Frame border className="mt-0"> | ||
|  | ||
| </Frame> | ||
| </Tab> | ||
| <Tab> | ||
| <Frame border className="mt-0"> | ||
|  | ||
| </Frame> | ||
| </Tab> | ||
| <Tab> | ||
| <Frame border className="mt-0"> | ||
|  | ||
| </Frame> | ||
| </Tab> | ||
| </Tabs> |
Oops, something went wrong.