passtest: AI-Powered Test Driven Development

AI-Powered Test Driven Development (AI-TDD) improves the test-driven development process by using GPT to generate Python function implementations that pass specified unit tests. The tool uses GPT-4 to generate code, and the Python standard library unittest to test the AI-generated implementations.

Features

• Takes a Python script containing unittest test cases as input
• Accepts a type-annotated function signature
• Uses GPT-4 to generate a function implementation that passes the provided tests
• Continuously iterates and improves the function implementation based on test results, until all tests pass or a specified maximum number of attempts is reached

Installation

This project uses Poetry as its package manager. To install and run the project, follow these steps:

1. Clone the repository:

git clone [email protected]:jordan-dimov/passtest.git
cd passtest

2. Install dependencies using Poetry:

poetry install

3. Set the Python path correctly (this allows importing modules from src/):

export PYTHONPATH=.

Usage

Basic Usage

To run the AI-TDD tool, execute the following command:

poetry run python src/passtest.py implement FILE_PATH

Replace FILE_PATH with the path to your Python script containing type-annotated function signature and unittest test cases.

Limit number of attempts

By default, the tool attempts up to 3 iterations. You can override this default with the --max-iterations INTEGER option:

poetry run python src/passtest.py implement FILE_PATH --max-iterations 5

Example

Suppose you have the unit tests in a file called examples/test_oddeven.py. The first line of the file should contain a type-annotated function signature, and the rest of the file should contain the test cases:

# def is_even(n: int) -> bool:
import unittest
from implementation import is_even

class TestOddEven(unittest.TestCase):
    def test_even_numbers(self):
        self.assertTrue(is_even(2))
        self.assertTrue(is_even(4))
        self.assertTrue(is_even(0))

    def test_odd_numbers(self):
        self.assertFalse(is_even(1))
        self.assertFalse(is_even(3))
        self.assertFalse(is_even(5))

if __name__ == "__main__":
    unittest.main()

Place the test file in your project directory, and then run the AI-TDD tool with the following command:

poetry run python src/passtest.py implement examples/test_oddeven.py

The tool will generate an implementation for the is_even() function that passes all provided tests and save the implementation to a file implementation.py.

AI-assisted linting

The AI-TDD tool can also be used to assist with linting. To do so, run the following command:

poetry run python src/passtest.py lint FILE_PATH --max-iterations 3

The tool will attempt to fix all linting errors (as reported by ruff) in the implementation for the tests in FILE_PATH. By default, it looks at implementation.py. If any changes were made, the linted file will be saved to linted_implementation.py

The lint command will check whether the linted implementation still passes the tests in FILE_PATH. If it does, the linted implementation will be saved to implementation.py