Skip to content

Latest commit

 

History

History
205 lines (203 loc) · 4.7 KB

File metadata and controls

205 lines (203 loc) · 4.7 KB

Results of Code LLama 34B on HumanEval/13

Date: 01 September, 2023
Model: codellama-34b-instruct
Temperature: 0.2
Passing runs: 10
Total runs: 10
Success rate: 100%

Prompt

def greatest_common_divisor(a: int, b: int) -> int:
    """ Return a greatest common divisor of two integers a and b
    >>> greatest_common_divisor(3, 5)
    1
    >>> greatest_common_divisor(25, 15)
    5
    """

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

def greatest_common_divisor(a: int, b: int) -> int:
    """ Return a greatest common divisor of two integers a and b
    >>> greatest_common_divisor(3, 5)
    1
    >>> greatest_common_divisor(25, 15)
    5
    """while b:
        a, b = b, a % b
    return a

Unit tests

METADATA = {
    'author': 'jt',
    'dataset': 'test'
}


def check(candidate):
    assert candidate(3, 7) == 1
    assert candidate(10, 15) == 5
    assert candidate(49, 14) == 7
    assert candidate(144, 60) == 12

Runs

✅ Run 1

Start time: 2023-09-01 02:48:28
Duration: 6.06s
Result: Pass

def greatest_common_divisor(a: int, b: int) -> int:
    """ Return a greatest common divisor of two integers a and b
    >>> greatest_common_divisor(3, 5)
    1
    >>> greatest_common_divisor(25, 15)
    5
    """
    while b:
        a, b = b, a % b
    return a

✅ Run 2

Start time: 2023-09-01 02:51:26
Duration: 5.16s
Result: Pass

def greatest_common_divisor(a: int, b: int) -> int:
    """ Return a greatest common divisor of two integers a and b
    >>> greatest_common_divisor(3, 5)
    1
    >>> greatest_common_divisor(25, 15)
    5
    """
    while b != 0:
        a, b = b, a % b
    return a

✅ Run 3

Start time: 2023-09-01 02:54:22
Duration: 5.25s
Result: Pass

def greatest_common_divisor(a: int, b: int) -> int:
    """ Return the greatest common divisor of two integers a and b
    >>> greatest_common_divisor(3, 5)
    1
    >>> greatest_common_divisor(25, 15)
    5
    """
    while b != 0:
        a, b = b, a % b
    return a

✅ Run 4

Start time: 2023-09-01 02:57:20
Duration: 5.49s
Result: Pass

def greatest_common_divisor(a: int, b: int) -> int:
    """ Return a greatest common divisor of two integers a and b
    >>> greatest_common_divisor(3, 5)
    1
    >>> greatest_common_divisor(25, 15)
    5
    """
    while b != 0:
        a, b = b, a % b
    return a

✅ Run 5

Start time: 2023-09-01 03:00:20
Duration: 6.25s
Result: Pass

def greatest_common_divisor(a: int, b: int) -> int:
    """ Return a greatest common divisor of two integers a and b
    >>> greatest_common_divisor(3, 5)
    1
    >>> greatest_common_divisor(25, 15)
    5
    """
    while b != 0:
        a, b = b, a % b
    return a

✅ Run 6

Start time: 2023-09-01 03:03:21
Duration: 5.38s
Result: Pass

def greatest_common_divisor(a: int, b: int) -> int:
    """ Return the greatest common divisor of two integers a and b
    >>> greatest_common_divisor(3, 5)
    1
    >>> greatest_common_divisor(25, 15)
    5
    """
    while b != 0:
        a, b = b, a % b
    return a

✅ Run 7

Start time: 2023-09-01 03:06:23
Duration: 5.43s
Result: Pass

def greatest_common_divisor(a: int, b: int) -> int:
    """ Return a greatest common divisor of two integers a and b
    >>> greatest_common_divisor(3, 5)
    1
    >>> greatest_common_divisor(25, 15)
    5
    """
    while b != 0:
        a, b = b, a % b
    return a

✅ Run 8

Start time: 2023-09-01 03:09:29
Duration: 5.21s
Result: Pass

def greatest_common_divisor(a: int, b: int) -> int:
    """ Return the greatest common divisor of two integers a and b
    >>> greatest_common_divisor(3, 5)
    1
    >>> greatest_common_divisor(25, 15)
    5
    """
    while b != 0:
        a, b = b, a % b
    return a

✅ Run 9

Start time: 2023-09-01 03:12:21
Duration: 5.54s
Result: Pass

def greatest_common_divisor(a: int, b: int) -> int:
    """ Return a greatest common divisor of two integers a and b
    >>> greatest_common_divisor(3, 5)
    1
    >>> greatest_common_divisor(25, 15)
    5
    """
    while b != 0:
        a, b = b, a % b
    return a

✅ Run 10

Start time: 2023-09-01 03:15:28
Duration: 6.79s
Result: Pass

def greatest_common_divisor(a: int, b: int) -> int:
    """ Return the greatest common divisor of two integers a and b
    >>> greatest_common_divisor(3, 5)
    1
    >>> greatest_common_divisor(25, 15)
    5
    """
    while b != 0:
        a, b = b, a % b
    return a