Code

Metrics for evaluation and coverage#8

Code

Merged

zimmski merged 6 commits intomainsymflower/eval-dev-quality:mainfrom metricssymflower/eval-dev-quality:metricsCopy head branch name to clipboardApr 3, 2024

+343-58

Commits

Commits on Apr 3, 2024

• Ability to select repositories in CLI so one can only run certain test candidates

  bauersimon

  committed
  b90ddba

  Copy full SHA for b90ddba

• Collect problems of models with test data and report number of encountered problems

  bauersimon

  committed
  8008ade

  Copy full SHA for 8008ade

• Extendable metrics collection

  

  bauersimon

  authored and

  zimmski

  committed
  05bc5c5

  Copy full SHA for 05bc5c5

• Extract statement coverage when executing repositories

  

  bauersimon

  authored and

  zimmski

  committed
  97bd61d

  Copy full SHA for 97bd61d

• Log LLM model response

  

  bauersimon

  authored and

  zimmski

  committed
  c4267d2

  Copy full SHA for c4267d2

• Include file name and import path for LLM prompt for better results of LLMs

  zimmski

  committed
  5b5a6d1

  Copy full SHA for 5b5a6d1