Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix bug, solutions not found #334

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
+3 −3
Open

fix bug, solutions not found #334

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions src/open_r1/rewards.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,14 +75,14 @@ def reasoning_steps_reward(completions, **kwargs):
return [min(1.0, count / 3) for count in matches]


def len_reward(completions: list[Dict[str, str]], solutions: list[str], **kwargs) -> float:
def len_reward(completions: list[Dict[str, str]], solution: list[str], **kwargs) -> float:
"""Compute length-based rewards to discourage overthinking and promote token efficiency.

Taken from from the Kimi 1.5 tech report: https://arxiv.org/abs/2501.12599

Args:
completions: List of model completions
solutions: List of ground truth solutions
solution: List of ground truth solution

Returns:
List of rewards where:
Expand All @@ -93,7 +93,7 @@ def len_reward(completions: list[Dict[str, str]], solutions: list[str], **kwargs

# First check correctness of answers
correctness = []
for content, sol in zip(contents, solutions):
for content, sol in zip(contents, solution):
gold_parsed = parse(
sol,
extraction_mode="first_match",
Expand Down