Skip to content

Commit

Permalink
instrumentation know how and span json - user guide update
Browse files Browse the repository at this point in the history
  • Loading branch information
Ravi Anne committed Jul 17, 2024
1 parent a12d8cd commit 4d0fb8d
Showing 1 changed file with 70 additions and 1 deletion.
71 changes: 70 additions & 1 deletion Monocle_User_Guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,4 +73,73 @@ setup_monocle_telemetry(
wrapper=atask_wrapper)
])

```
```

### Default configuration of instrumented methods in Monocle

The following files comprise of default configuration of instrumented methods and span names corresponding to them, for each framework respectively.
[src/monocle_apptrace/langchain/__init__.py](src/monocle_apptrace/langchain/__init__.py)
[src/monocle_apptrace/llamaindex/__init__.py](src/monocle_apptrace/llamaindex/__init__.py)
[src/monocle_apptrace/haystack/__init__.py](src/monocle_apptrace/haystack/__init__.py)

Following configuration instruments ```invoke(..)``` of ```RunnableSequence```, aka chain or worflow in Langchain parlance, to emit the span.

```
{
"package": "langchain.schema.runnable",
"object": "RunnableSequence",
"method": "invoke",
"span_name": "langchain.workflow",
"wrapper": task_wrapper
}
```

#### span json

Monocle generates spans which adhere to [Tracing API | OpenTelemetry](https://opentelemetry.io/docs/specs/otel/trace/api/#span) format. Please note that ```trace_id``` groups related spans and is auto generated with-in Monocle.

| Span JSON | Description |
| ------------- | ------------- |
| {||
| "```name```": "langchain.workflow",|span name and is configurable in [__init.py__](src/monocle_apptrace/langchain/__init__.py) or in ```setup_okahu_telemetry(...)```|
| "```context```": {|this gets autogenerated|
|   "```trace_id```": "0xe5269f0e534efa098b240f974220d6b7",||
|   "```span_id```": "0x30b13075eca52f44",||
|   "```trace_state```": "[]"||
|   },||
|"```kind```": "SpanKind.INTERNAL",| An enum that describes what this span is about. Default value is SpanKind.INTERNAL, as current enums do not cover ML apps |
|"```parent_id```": null,|If null, this is root span|
|"```start_time```": "2024-07-16T17:05:15.544861Z",||
|"```end_time```": "2024-07-16T17:05:43.502007Z",||
|"```status```": {||
|  "```status_code```": "UNSET"| This to be updated to set to OK or ERROR. Default is UNSET|
|  },||
|"```attributes```": {||
|  "workflow_name": "ml_rag_app",|defines the name of the service being set in ```setup_okahu_telemetry(...)``` during initialization of instrumentation|
|  "workflow_type": "workflow.langchain"|type of framework that generated this span|
|  },||
|"```events```": [|Captures the log records|
|  {||
|   "```name```": "input",|name of the event. If the span is about LLM, then this will be 'input'. For vector store retrieval, this would be 'context_input'|
|   "```timestamp```": "2024-07-16T17:05:15.544874Z",||
|   "```attributes```": {|captures the 'input' attributes. Based on the workflow of the ML framework being used, the attributes change|
|    "question": "What is Task Decomposition?",|represents LLM query|
|    "q_a_pairs": "..." |represents questions and answers for a few shot LLM prompting |
|   }||
|  },||
|  {||
|   "```name```": "output",|represents 'ouput' event of LLM|
|   "```timestamp```": "2024-07-16T17:05:43.501996Z",||
|  "```attributes```": {||
|    "response": "Task Decomposition is ..."|response to LLM query. |
|   }||
|  }||
|  ],||
|  "```links```": [],|unpopulated and unused|
|  "```resource```": {|represents the service name or server or machine or container which generated the span|
|    "```attributes```": {||
|     "service.name": "ml_rag_app"|only service.name is being populated and defaults to the value of 'workflow_name' |
|    },||
|  "```schema_url```": ""|unpopulated and unused|
|   }||
|} | |

0 comments on commit 4d0fb8d

Please sign in to comment.