Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

instrumentation know how and span json - user guide update #14

Merged
merged 2 commits into from
Jul 18, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 71 additions & 1 deletion Monocle_User_Guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,4 +73,74 @@ setup_monocle_telemetry(
wrapper=atask_wrapper)
])

```
```

### Default configuration of instrumented methods in Monocle

The following files comprise of default configuration of instrumented methods and span names corresponding to them, for each framework respectively.

[src/monocle_apptrace/langchain/__init__.py](src/monocle_apptrace/langchain/__init__.py),
[src/monocle_apptrace/llamaindex/__init__.py](src/monocle_apptrace/llamaindex/__init__.py),
[src/monocle_apptrace/haystack/__init__.py](src/monocle_apptrace/haystack/__init__.py)

Following configuration instruments ```invoke(..)``` of ```RunnableSequence```, aka chain or worflow in Langchain parlance, to emit the span.

```
{
"package": "langchain.schema.runnable",
"object": "RunnableSequence",
"method": "invoke",
"span_name": "langchain.workflow",
"wrapper": task_wrapper
}
```

#### span json

Monocle generates spans which adhere to [Tracing API | OpenTelemetry](https://opentelemetry.io/docs/specs/otel/trace/api/#span) format. Please note that ```trace_id``` groups related spans and is auto generated with-in Monocle.

| Span JSON | Description |
| ------------- | ------------- |
| {||
| "```name```": "langchain.workflow",|span name and is configurable in [__init.py__](src/monocle_apptrace/langchain/__init__.py) or in ```setup_okahu_telemetry(...)```|
| "```context```": {|this gets autogenerated|
|   "```trace_id```": "0xe5269f0e534efa098b240f974220d6b7",||
|   "```span_id```": "0x30b13075eca52f44",||
|   "```trace_state```": "[]"||
|   },||
|"```kind```": "SpanKind.INTERNAL",| An enum that describes what this span is about. Default value is SpanKind.INTERNAL, as current enums do not cover ML apps |
|"```parent_id```": null,|If null, this is root span|
|"```start_time```": "2024-07-16T17:05:15.544861Z",||
|"```end_time```": "2024-07-16T17:05:43.502007Z",||
|"```status```": {||
|  "```status_code```": "UNSET"| This to be updated to set to OK or ERROR. Default is UNSET|
|  },||
|"```attributes```": {||
|  "workflow_name": "ml_rag_app",|defines the name of the service being set in ```setup_okahu_telemetry(...)``` during initialization of instrumentation|
|  "workflow_type": "workflow.langchain"|type of framework that generated this span|
|  },||
|"```events```": [|Captures the log records|
|  {||
|   "```name```": "input",|name of the event. If the span is about LLM, then this will be 'input'. For vector store retrieval, this would be 'context_input'|
|   "```timestamp```": "2024-07-16T17:05:15.544874Z",||
|   "```attributes```": {|captures the 'input' attributes. Based on the workflow of the ML framework being used, the attributes change|
|    "question": "What is Task Decomposition?",|represents LLM query|
|    "q_a_pairs": "..." |represents questions and answers for a few shot LLM prompting |
|   }||
|  },||
|  {||
|   "```name```": "output",|represents 'ouput' event of LLM|
|   "```timestamp```": "2024-07-16T17:05:43.501996Z",||
|  "```attributes```": {||
|    "response": "Task Decomposition is ..."|response to LLM query. |
|   }||
|  }||
|  ],||
|  "```links```": [],|unpopulated and unused|
|  "```resource```": {|represents the service name or server or machine or container which generated the span|
|    "```attributes```": {||
|     "service.name": "ml_rag_app"|only service.name is being populated and defaults to the value of 'workflow_name' |
|    },||
|  "```schema_url```": ""|unpopulated and unused|
|   }||
|} | |
Loading