Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Extend the evalatuion.csv with the current maximum values, and keep them updated in the repositories #404

Open
1 of 10 tasks
zimmski opened this issue Jan 30, 2025 · 0 comments
Open
1 of 10 tasks

Extend the evalatuion.csv with the current maximum values, and keep them updated in the repositories #404

zimmski opened this issue Jan 30, 2025 · 0 comments
Assignees
@bauersimon
Labels
enhancement New feature or request
Milestone
v1.1.0

Comments

@zimmski
Copy link
Member

zimmski commented Jan 30, 2025 •
edited
Loading

Please help us keep the number of duplicated issues small. We kindly ask you to add your input to the appropriate issue or PR in case your feature idea is already being tracked.

Problem

Maximum metrics can change over time for a language, repository, task, case, ... to freeze the maximums, we want to save them in the evaluation.csv

Solution (optional)

Tasks:

  • Read maximums from the repository files
  • Insert
    • add files-executed-maximum
    • add coverage-maximum
    • add tests-passing-maximum
    • add response-no-error-maximum
    • add response-with-code-maximum
    • add response-no-excess-maximum
  • Update maximums if the value gets bigger in the repository configuration
@zimmski zimmski added the enhancement New feature or request label Jan 30, 2025
@zimmski zimmski added this to the v1.1.0 milestone Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bauersimon bauersimon

Labels
enhancement New feature or request
Projects
None yet
Milestone
v1.1.0
Development

No branches or pull requests
2 participants
@zimmski @bauersimon