Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add swiss legal evals as new community tasks #389

Open
wants to merge 25 commits into
base: main
Choose a base branch
from
Open

Add swiss legal evals as new community tasks #389

wants to merge 25 commits into from
+627 −0

Commits on Nov 11, 2024

  1. Add swiss legal evals as new community tasks

    @JoelNiklaus
    JoelNiklaus committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    e2a27a7 View commit details
    Browse the repository at this point in the history

Commits on Nov 12, 2024

  1. Removed nltk and numpy dependencies.

    @JoelNiklaus
    JoelNiklaus committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    aa409c8 View commit details
    Browse the repository at this point in the history

  2. Added short dataset descriptions.

    @JoelNiklaus
    JoelNiklaus committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    a8ee2a5 View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2024

  1. Merge branch 'main' into add_swiss_legal_evals

    @clefourrier
    clefourrier authored Nov 13, 2024
    Configuration menu
    Copy the full SHA
    8f68844 View commit details
    Browse the repository at this point in the history

  2. Removed open judge models and added COMET and METEOR.

    @JoelNiklaus
    JoelNiklaus committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    c7f7038 View commit details
    Browse the repository at this point in the history

Commits on Nov 19, 2024

  1. Merge branch 'main' into add_swiss_legal_evals

    @clefourrier
    clefourrier authored Nov 19, 2024
    Configuration menu
    Copy the full SHA
    0ca5af6 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into add_swiss_legal_evals

    @NathanHB
    NathanHB authored Nov 19, 2024
    Configuration menu
    Copy the full SHA
    1d51a01 View commit details
    Browse the repository at this point in the history

Commits on Nov 20, 2024

  1. Ran pre-commit hooks.

    @JoelNiklaus
    JoelNiklaus committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    5d41ce0 View commit details
    Browse the repository at this point in the history

  2. Changed prompt template.

    @JoelNiklaus
    JoelNiklaus committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    8194125 View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2024

  1. Added legal translation specific judge prompt.

    @JoelNiklaus
    JoelNiklaus committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    c58ae44 View commit details
    Browse the repository at this point in the history

  2. Improved judge prompt.

    @JoelNiklaus
    JoelNiklaus committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    ff3705f View commit details
    Browse the repository at this point in the history

  3. Changed metric selection.

    @JoelNiklaus
    JoelNiklaus committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    091ec11 View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2024

  1. Made generation_size dependent on the config.

    @JoelNiklaus
    JoelNiklaus committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    5a47956 View commit details
    Browse the repository at this point in the history

  2. Fixed error in config.

    @JoelNiklaus
    JoelNiklaus committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    6bf7fa2 View commit details
    Browse the repository at this point in the history

  3. Fixed error in config.

    @JoelNiklaus
    JoelNiklaus committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    6cf1c2a View commit details
    Browse the repository at this point in the history

  4. Added support for multiple devices.

    @JoelNiklaus
    JoelNiklaus committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    b548801 View commit details
    Browse the repository at this point in the history

Commits on Nov 25, 2024

  1. Fixed some bugs for evaluation on GPUs.

    @JoelNiklaus
    JoelNiklaus committed Nov 25, 2024
    Configuration menu
    Copy the full SHA
    ee2a83c View commit details
    Browse the repository at this point in the history

Commits on Nov 26, 2024

  1. Added batch inference for heavy metrics and multiplied each score by … 

    …100.
    @JoelNiklaus
    JoelNiklaus committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    36b7e94 View commit details
    Browse the repository at this point in the history

  2. Added few shot examples and did some refactoring.

    @JoelNiklaus
    JoelNiklaus committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    5ba218f View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into add_swiss_legal_evals

    @JoelNiklaus
    JoelNiklaus authored Nov 26, 2024
    Configuration menu
    Copy the full SHA
    8490841 View commit details
    Browse the repository at this point in the history

  4. Switched to an own judge class.

    @JoelNiklaus
    JoelNiklaus committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    576b847 View commit details
    Browse the repository at this point in the history

  5. Fixed issue with judge metric not showing up in results.

    @JoelNiklaus
    JoelNiklaus committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    41bb59a View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2024

  1. Fixed issue with evaluation on GPUs.

    @JoelNiklaus
    JoelNiklaus committed Nov 27, 2024
    Configuration menu
    Copy the full SHA
    d82cd91 View commit details
    Browse the repository at this point in the history

  2. Speed up metric computation on GPUs.

    @JoelNiklaus
    JoelNiklaus committed Nov 27, 2024
    Configuration menu
    Copy the full SHA
    1b13d9f View commit details
    Browse the repository at this point in the history

  3. Added more logging.

    @JoelNiklaus
    JoelNiklaus committed Nov 27, 2024
    Configuration menu
    Copy the full SHA
    df0f3f0 View commit details
    Browse the repository at this point in the history