[Feature Request]: Add observability to model input and output for app testing / evaluation

[dooriya]

Scenario

It is important to add observability to an AI bot built with teams-ai SDK since the AI may have non-deterministic behaviors. Currently it is quite hard to evaluate against an AI bot since developers cannot get the input and output from the AI model. Ideally the AI components in this library should be able to emit such information as structure log and traces (following Open Telemetry specification).

For example, we have a very simple bot app that responds to user questions like an AI assistant:
https://github.com/OfficeDev/teams-toolkit/blob/dev/templates/python/custom-copilot-basic/src/bot.py.tpl#L48-L55

As a developer I want to capture the model's input and output pair (with other metadata, such as prompt, token count, latency, etc) in a structure way. This data can then be used for evaluation to understand how well the app performs.

Solution

As a developer I want to capture the model's input and output pair (with other metadata, such as prompt, token count, latency, etc) in a structure way. This data can then be used for evaluation to understand how well the app performs.

Additional Context

No response

[singhk97]

Thanks for the feature request, we'll add this to our backlog.