Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding-csv-to-attachements #604

Merged
merged 6 commits into from
Dec 23, 2024
Merged

adding-csv-to-attachements #604

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
6dd2197
adding-csv-to-attachements
Dec 20, 2024
c4f59c6
Update docs/observability/how_to_guides/tracing/upload_files_with_tra…
tanushree-sharma Dec 20, 2024
f4707a6
linting
Dec 20, 2024
a029716
Update docs/observability/how_to_guides/tracing/upload_files_with_tra…
tanushree-sharma Dec 23, 2024
4bb04d8
linting
Dec 23, 2024
13e6c8d
fixing linting again
Dec 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
141 changes: 76 additions & 65 deletions docs/observability/how_to_guides/tracing/upload_files_with_traces.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ The following features are available in the following SDK versions:
- JS/TS SDK: >=0.2.5
:::

LangSmith supports uploading binary files (such as images, audio, videos, and PDFs) with your traces. This is particularly useful when working with LLM pipelines using multimodal inputs or outputs.
LangSmith supports uploading binary files (such as images, audio, videos, PDFs, and CSVs) with your traces. This is particularly useful when working with LLM pipelines using multimodal inputs or outputs.

In both the Python and TypeScript SDKs, attachments can be added to your traces by specifying the MIME type and binary content of each file.
This guide explains how to define and trace attachments using the `Attachment` type in Python and `Uint8Array` / `ArrayBuffer` in TypeScript.
Expand All @@ -41,8 +41,9 @@ def trace_with_attachments(
audio: Attachment,
video: Attachment,
pdf: Attachment,
csv: Attachment,
):
return f"Processed: {val}, {text}, {len(image.data)}, {len(audio.data)}, {len(video.data)}, {len(pdf.data)}"\n\n
return f"Processed: {val}, {text}, {len(image.data)}, {len(audio.data)}, {len(video.data)}, {len(pdf.data), {len(csv.data)}}"\n\n
# Helper function to load files as bytes
def load_file(file_path: str) -> bytes:
with open(file_path, "rb") as f:
Expand All @@ -52,89 +53,99 @@ image_data = load_file("my_image.png")
audio_data = load_file("my_mp3.mp3")
video_data = load_file("my_video.mp4")
pdf_data = load_file("my_document.pdf")
csv_data = load_file("my_csv.csv")
image_attachment = Attachment(mime_type="image/png", data=image_data)
audio_attachment = Attachment(mime_type="audio/mpeg", data=audio_data)
video_attachment = Attachment(mime_type="video/mp4", data=video_data)
pdf_attachment = Attachment(mime_type="application/pdf", data=pdf_data)
csv_attachment = Attachment(mime_type="text/csv", data=csv_data)

# Define other parameters

val = 42
text = "Hello, world!"

# Call the function with traced attachments

result = trace_with_attachments(
val=val,
text=text,
image=image_attachment,
audio=audio_attachment,
video=video_attachment,
pdf=pdf_attachment,
val=val,
text=text,
image=image_attachment,
audio=audio_attachment,
video=video_attachment,
pdf=pdf_attachment,
csv=csv_attachment,
)`,
`In the Python SDK, you can use the \`Attachment\` type to add files to your traces.
Each \`Attachment\` requires:\n

- \`mime_type\` (str): The MIME type of the file (e.g., \`"image/png"\`).
- \`data\` (bytes): The binary content of the file.\n
Simply decorate a function with \`@traceable\` and include your \`Attachment\` instances as arguments.`
),
TypeScriptBlock(
`import { traceable } from "langsmith/traceable";\n
const traceableWithAttachments = traceable(
(
val: number,
text: string,
attachment: Uint8Array,
attachment2: ArrayBuffer,
attachment3: Uint8Array,
attachment4: ArrayBuffer
) =>
\`Processed: \${val}, \${text}, \${attachment.length}, \${attachment2.byteLength}, \${attachment3.length}, \${attachment4.byteLength}\`,
{
name: "traceWithAttachments",
extractAttachments: (
val: number,
text: string,
attachment: Uint8Array,
attachment2: ArrayBuffer,
attachment3: Uint8Array,
attachment4: ArrayBuffer
) => [
{
"image inputs": ["image/png", attachment],
"mp3 inputs": ["audio/mpeg", new Uint8Array(attachment2)],
"video inputs": ["video/mp4", attachment3],
"pdf inputs": ["application/pdf", new Uint8Array(attachment4)],
},
{ val, text },
],
}
);\n
const fs = Deno // or Node.js fs module\n
const image = await fs.readFile("my_image.png"); // Uint8Array
const mp3Buffer = await fs.readFile("my_mp3.mp3");
const mp3ArrayBuffer = mp3Buffer.buffer; // Convert to ArrayBuffer\n
const video = await fs.readFile("my_video.mp4"); // Uint8Array
const pdfBuffer = await fs.readFile("my_document.pdf");
const pdfArrayBuffer = pdfBuffer.buffer; // Convert to ArrayBuffer\n
// Define example parameters
const val = 42;
const text = "Hello, world!";\n
// Call traceableWithAttachments with the files
const result = await traceableWithAttachments(val, text, image, mp3ArrayBuffer, video, pdfArrayBuffer);`,
`In the TypeScript SDK, you can add attachments to traces by using \`Uint8Array\` or \`ArrayBuffer\` as data types.
Simply decorate a function with \`@traceable\` and include your \`Attachment\` instances as arguments.`),
TypeScriptBlock(`import { traceable } from "langsmith/traceable";\n
const traceableWithAttachments = traceable(
(
val: number,
text: string,
attachment: Uint8Array,
attachment2: ArrayBuffer,
attachment3: Uint8Array,
attachment4: ArrayBuffer,
attachment5: Uint8Array,
) =>
\`Processed: \${val}, \${text}, \${attachment.length}, \${attachment2.byteLength}, \${attachment3.length}, \${attachment4.byteLength}\, \${attachment5.byteLength}\`,
{
name: "traceWithAttachments",
extractAttachments: (
val: number,
text: string,
attachment: Uint8Array,
attachment2: ArrayBuffer,
attachment3: Uint8Array,
attachment4: ArrayBuffer,
attachment5: Uint8Array,
) => [
{
"image inputs": ["image/png", attachment],
"mp3 inputs": ["audio/mpeg", new Uint8Array(attachment2)],
"video inputs": ["video/mp4", attachment3],
"pdf inputs": ["application/pdf", new Uint8Array(attachment4)],
"csv inputs": ["text/csv", new Uint8Array(attachment5)]
},
{ val, text },
],
}
);\n
const fs = Deno // or Node.js fs module\n
const image = await fs.readFile("my_image.png"); // Uint8Array
const mp3Buffer = await fs.readFile("my_mp3.mp3");
const mp3ArrayBuffer = mp3Buffer.buffer; // Convert to ArrayBuffer\n
const video = await fs.readFile("my_video.mp4"); // Uint8Array
const pdfBuffer = await fs.readFile("my_document.pdf");
const pdfArrayBuffer = pdfBuffer.buffer; // Convert to ArrayBuffer
const csv = await fs.readFile("test-vals.csv"); // Uint8Array\n
// Define example parameters
const val = 42;
const text = "Hello, world!";\n
// Call traceableWithAttachments with the files
const result = await traceableWithAttachments(val, text, image, mp3ArrayBuffer, video, pdfArrayBuffer, csv);`,
`In the TypeScript SDK, you can add attachments to traces by using \`Uint8Array\` or \`ArrayBuffer\` as data types.
Each attachment's MIME type is specified within \`extractAttachments\`:\n\n
- \`Uint8Array\`: Useful for handling binary data directly.
- \`ArrayBuffer\`: Represents fixed-length binary data, which can be converted to \`Uint8Array\` as needed.\n
Wrap your function with \`traceable\` and include your attachments within the \`extractAttachments\` option.\n
In the TypeScript SDK, the \`extractAttachments\` function is an optional parameter in the \`traceable\` configuration. When the traceable-wrapped function is invoked, it extracts binary data (e.g., images, audio files) from your inputs and logs them alongside other trace data, specifying their MIME types.\n
\`\`\`
type AttachmentData = Uint8Array | ArrayBuffer;
type Attachments = Record<string, [string, AttachmentData]>;\n
extractAttachments?: (
...args: Parameters<Func>
) => [Attachments | undefined, KVMap];
\`\`\``
),
In the TypeScript SDK, the \`extractAttachments\` function is an optional parameter in the \`traceable\` configuration. When the traceable-wrapped function is invoked, it extracts binary data (e.g., images, audio files) from your inputs and logs them alongside other trace data, specifying their MIME types.\n
\`\`\`
type AttachmentData = Uint8Array | ArrayBuffer;
type Attachments = Record<string, [string, AttachmentData]>;\n
extractAttachments?: (
...args: Parameters<Func>
) => [Attachments | undefined, KVMap];
\`\`\``
),
]}
groupId="client-language"
/>
/>

Here is how the above would look in the LangSmith UI. You can expand each attachment to view its contents.
![](./static/trace_with_attachments.png)
Loading