Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Decouple Benchmark and MultiEnvironment #16

Open
1 task done
dandansamax opened this issue Aug 25, 2024 · 0 comments
Open
1 task done

[Feature Request] Decouple Benchmark and MultiEnvironment #16

dandansamax opened this issue Aug 25, 2024 · 0 comments
Assignees
Labels
core Models, pipeline changing to CRAB core enhancement New feature or request

Comments

@dandansamax
Copy link
Collaborator

Required prerequisites

  • I have searched the Issue Tracker that this hasn't already been reported. (+1 or comment there if it has.)

Motivation

Current Benchmark class in coupled with GraphEvaluator and Multiple Environment features. While uses may want to use the environments only without evaluation.

Solution

The Benchmark should be divided into two classs:

  • Benchmark class is reponsible for organizing Tasks and Evaluators
  • MultiEnvironment class is responsible for manage agent intreaction with multiple environments

Users can use MultiEnvironment class with agents to operate environments without evaluators.

Additional context

No response

@dandansamax dandansamax added the enhancement New feature or request label Aug 25, 2024
@dandansamax dandansamax self-assigned this Aug 25, 2024
@dandansamax dandansamax added the core Models, pipeline changing to CRAB core label Aug 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Models, pipeline changing to CRAB core enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant