Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include LLM analytics in the evaluation mode experiments results #322

Open
FelixTJDietrich opened this issue Jul 26, 2024 · 0 comments
Open
Labels
athena package playground Pull requests that update the playground

Comments

@FelixTJDietrich
Copy link
Collaborator

Is your feature request related to a problem?

Currently the LLM analytics data is missing when exporting the experiment results after conducting an evaluation using the playground.

Describe the solution you'd like

The Athena Playground should be able to request the analytics data from Athena during export. This can be a global endpoint that uses the already included header information to request the right traces from LangSmith or LangFuse in the future.

Describe the alternatives you've considered

Exporting it manually using a separate script that is not integrated. Not good if an instructor wants to conduct an experiment.

Additional context

No response

@FelixTJDietrich FelixTJDietrich added playground Pull requests that update the playground athena package labels Jul 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
athena package playground Pull requests that update the playground
Projects
None yet
Development

No branches or pull requests

1 participant