Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

instrumentation know how and span json - user guide update #14

Merged
merged 2 commits into from
Jul 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 71 additions & 1 deletion Monocle_User_Guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,4 +73,74 @@ setup_monocle_telemetry(
wrapper=atask_wrapper)
])

```
```

### Default configuration of instrumented methods in Monocle

The following files comprise of default configuration of instrumented methods and span names corresponding to them, for each framework respectively.

[src/monocle_apptrace/langchain/__init__.py](src/monocle_apptrace/langchain/__init__.py),
[src/monocle_apptrace/llamaindex/__init__.py](src/monocle_apptrace/llamaindex/__init__.py),
[src/monocle_apptrace/haystack/__init__.py](src/monocle_apptrace/haystack/__init__.py)

Following configuration instruments ```invoke(..)``` of ```RunnableSequence```, aka chain or worflow in Langchain parlance, to emit the span.

```
{
"package": "langchain.schema.runnable",
"object": "RunnableSequence",
"method": "invoke",
"span_name": "langchain.workflow",
"wrapper": task_wrapper
}
```

#### span json

Monocle generates spans which adhere to [Tracing API | OpenTelemetry](https://opentelemetry.io/docs/specs/otel/trace/api/#span) format. Please note that ```trace_id``` groups related spans and is auto generated with-in Monocle.

| Span JSON | Description |
| ------------- | ------------- |
| {||
| "```name```": "langchain.workflow",|span name and is configurable in [__init.py__](src/monocle_apptrace/langchain/__init__.py) or in ```setup_okahu_telemetry(...)```|
| "```context```": {|this gets autogenerated|
|   "```trace_id```": "0xe5269f0e534efa098b240f974220d6b7",||
|   "```span_id```": "0x30b13075eca52f44",||
|   "```trace_state```": "[]"||
|   },||
|"```kind```": "SpanKind.INTERNAL",| an enum that describes what this span is about. Default value is SpanKind.INTERNAL, as current enums do not cover ML apps |
|"```parent_id```": null,|if null, this is root span|
|"```start_time```": "2024-07-16T17:05:15.544861Z",||
|"```end_time```": "2024-07-16T17:05:43.502007Z",||
|"```status```": {||
|  "```status_code```": "UNSET"| status of span to OK or ERROR. Default is UNSET|
|  },||
|"```attributes```": {||
|  "workflow_name": "ml_rag_app",|defines the name of the service being set in ```setup_okahu_telemetry(...)``` during initialization of instrumentation|
|  "workflow_type": "workflow.langchain"|type of framework that generated this span|
|  },||
|"```events```": [|captures the log records|
|  {||
|   "```name```": "input",|name of the event. If the span is about LLM, then this will be 'input'. For vector store retrieval, this would be 'context_input'|
|   "```timestamp```": "2024-07-16T17:05:15.544874Z",||
|   "```attributes```": {|captures the 'input' attributes. Based on the workflow of the ML framework being used, the attributes change|
|    "question": "What is Task Decomposition?",|represents LLM query|
|    "q_a_pairs": "..." |represents questions and answers for a few shot LLM prompting |
|   }||
|  },||
|  {||
|   "```name```": "output",|represents 'ouput' event of LLM|
|   "```timestamp```": "2024-07-16T17:05:43.501996Z",||
|  "```attributes```": {||
|    "response": "Task Decomposition is ..."|response to LLM query. |
|   }||
|  }||
|  ],||
|&emsp; "```links```": [],|unused. Ideally this links other causally-related spans,<br/> but as spans are grouped by ```trace_id```, and ```parent_id``` links to parent span, this is unused|
|&emsp; "```resource```": {|represents the service name or server or machine or container which generated the span|
|&emsp;&emsp;&emsp; "```attributes```": {||
|&emsp;&emsp;&emsp;&emsp; "service.name": "ml_rag_app"|only service.name is being populated and defaults to the value of 'workflow_name' |
|&emsp;&emsp;&emsp; },||
|&emsp;&emsp;"```schema_url```": ""|unused|
|&emsp;&emsp; }||
|} | |
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,12 @@ The traces are compatible with OpenTelemetry format. They are further enriched t
```
setup_okahu_telemetry(workflow_name="your-app-name")
```
Please refer to Monocle user guide for more details
Please refer to [Monocle user guide](Monocle_User_Guide.md) for more details

## Get involved
### Provide feedback
- Submit issues and enhancements requests via Github issues

### Contribute
- Monocle is community based open source project. We welcome your contributions. Please refer to the CONTRIBUTING and CODE_OF_CONDUCT for guidelines. The contributor's guide provides technical details of the project.
- Monocle is community based open source project. We welcome your contributions. Please refer to the CONTRIBUTING and CODE_OF_CONDUCT for guidelines. The [contributor's guide](CONTRIBUTING.md) provides technical details of the project.

Loading