Skip to content

Commit

Permalink
Merge pull request #17 from crytic/fuzz-utils
Browse files Browse the repository at this point in the history
Rename project fuzz-utils
  • Loading branch information
montyly authored Feb 16, 2024
2 parents d0cfc46 + e5e38f2 commit 9dcb85a
Show file tree
Hide file tree
Showing 14 changed files with 44 additions and 44 deletions.
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,34 +2,34 @@

# Automated tool for generating Foundry unit tests from smart contract fuzzer failed properties

`test-generator` is a Python tool that generates unit tests from [Echidna](https://github.com/crytic/echidna) and [Medusa](https://github.com/crytic/medusa/tree/master) failed properties, using the generated reproducer files. It uses [Slither](https://github.com/crytic/slither) for determining types and jinja2 for generating the test files using string templates.
`fuzz-utils` is a Python tool that generates unit tests from [Echidna](https://github.com/crytic/echidna) and [Medusa](https://github.com/crytic/medusa/tree/master) failed properties, using the generated reproducer files. It uses [Slither](https://github.com/crytic/slither) for determining types and jinja2 for generating the test files using string templates.

**Disclaimer**: Please note that `test-generator` is **under development**. Currently, not all Solidity types are supported and some types (like `bytes*`, and `string`) might be improperly decoded from the corpora call sequences. We are investigating a better corpus format that will ease the creation of unit tests.
**Disclaimer**: Please note that `fuzz-utils` is **under development**. Currently, not all Solidity types are supported and some types (like `bytes*`, and `string`) might be improperly decoded from the corpora call sequences. We are investigating a better corpus format that will ease the creation of unit tests.

## Features
`test-generator` provides support for:
`fuzz-utils` provides support for:
- ✔️ Generating Foundry unit tests from the fuzzer corpus of single entry point fuzzing harnesses
- ✔️ Medusa and Echidna corpora
- ✔️ Solidity types: `bool`,`uint*`, `int*`, `address`, `struct`, `enum`, single-dimensional fixed-size arrays and dynamic arrays, multi-dimensional fixed-size arrays.

Multi-dimensional dynamic arrays, function pointers, and other more complex types are in the works, but are currently not supported.
## Installation and pre-requisites

In order to be able to use `test-generator`, you will need to install it first:
In order to be able to use `fuzz-utils`, you will need to install it first:

```bash
git clone [email protected]:crytic/test-generator.git
cd test-generator
git clone [email protected]:crytic/fuzz-utils.git
cd fuzz-utils
pip3 install -e .
```

These commands will install all the Python libraries and tools required to run `test-generator`. However, it won't install Echidna or Medusa, so you will need to download and install the latest version yourself from its official releases ([Echidna](https://github.com/crytic/echidna/releases), [Medusa](https://github.com/crytic/medusa/releases)).
These commands will install all the Python libraries and tools required to run `fuzz-utils`. However, it won't install Echidna or Medusa, so you will need to download and install the latest version yourself from its official releases ([Echidna](https://github.com/crytic/echidna/releases), [Medusa](https://github.com/crytic/medusa/releases)).

## Example

In order to generate a test file for the [BasicTypes.sol](test/src/BasicTypes.sol) contract, based on the Echidna corpus reproducers for this contract ([corpus-basic](tests/test_data/echidna-corpora/corpus-basic/)), we need to `cd` into the `tests/test_data` directory which contains the Foundry project and run the command:
```bash
test-generator ./src/BasicTypes.sol --corpus-dir echidna-corpora/corpus-basic --contract "BasicTypes" --test-directory "./test/" --inheritance-path "../src/" --fuzzer echidna
fuzz-utils ./src/BasicTypes.sol --corpus-dir echidna-corpora/corpus-basic --contract "BasicTypes" --test-directory "./test/" --inheritance-path "../src/" --fuzzer echidna
```

Running this command should generate a `BasicTypes_Echidna_Test.sol` file in the [tests](/tests/test_data/test/) directory of the Foundry project.
Expand All @@ -48,4 +48,4 @@ Additional options are available for the script:
For information about how to contribute to this project, check out the [CONTRIBUTING](CONTRIBUTING.md) guidelines.

## License
`test-generator` is licensed and distributed under the [AGPLv3](LICENSE).
`fuzz-utils` is licensed and distributed under the [AGPLv3](LICENSE).
File renamed without changes.
20 changes: 10 additions & 10 deletions test_generator/fuzzers/Echidna.py → fuzz_utils/fuzzers/Echidna.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@
from slither.core.declarations.structure import Structure
from slither.core.declarations.structure_contract import StructureContract
from slither.core.declarations.enum import Enum
from test_generator.utils.crytic_print import CryticPrint
from test_generator.templates.foundry_templates import templates
from test_generator.utils.encoding import parse_echidna_byte_string
from test_generator.utils.error_handler import handle_exit
from fuzz_utils.utils.crytic_print import CryticPrint
from fuzz_utils.templates.foundry_templates import templates
from fuzz_utils.utils.encoding import parse_echidna_byte_string
from fuzz_utils.utils.error_handler import handle_exit


class Echidna:
Expand Down Expand Up @@ -174,7 +174,7 @@ def _match_elementary_types(self, param: dict, recursive: bool) -> str | NoRetur
return interpreted_string
case _:
handle_exit(
f"\n* The parameter tag `{param['tag']}` could not be found in the call object. This could indicate an issue in decoding the call sequence, or a missing feature. Please open an issue at https://github.com/crytic/test-generator/issues"
f"\n* The parameter tag `{param['tag']}` could not be found in the call object. This could indicate an issue in decoding the call sequence, or a missing feature. Please open an issue at https://github.com/crytic/fuzz-utils/issues"
)

def _match_array_type(
Expand Down Expand Up @@ -202,7 +202,7 @@ def _match_array_type(
return name, definitions, index
case _:
handle_exit(
f"\n* The parameter tag `{param['tag']}` could not be found in the call object. This could indicate an issue in decoding the call sequence, or a missing feature. Please open an issue at https://github.com/crytic/test-generator/issues"
f"\n* The parameter tag `{param['tag']}` could not be found in the call object. This could indicate an issue in decoding the call sequence, or a missing feature. Please open an issue at https://github.com/crytic/fuzz-utils/issues"
)

def _match_user_defined_type(
Expand All @@ -218,7 +218,7 @@ def _match_user_defined_type(
return definitions, f"{input_parameter}({','.join(func_params)})"
case _:
handle_exit(
f"\n* The parameter type `{input_parameter.type}` could not be found. This could indicate an issue in decoding the call sequence, or a missing feature. Please open an issue at https://github.com/crytic/test-generator/issues"
f"\n* The parameter type `{input_parameter.type}` could not be found. This could indicate an issue in decoding the call sequence, or a missing feature. Please open an issue at https://github.com/crytic/fuzz-utils/issues"
)
case "AbiUInt":
if isinstance(input_parameter.type, Enum):
Expand All @@ -227,11 +227,11 @@ def _match_user_defined_type(

# TODO is this even reachable?
handle_exit(
f"\n* The parameter type `{input_parameter.type}` does not match the intended type `Enum`. This could indicate an issue in decoding the call sequence, or a missing feature. Please open an issue at https://github.com/crytic/test-generator/issues"
f"\n* The parameter type `{input_parameter.type}` does not match the intended type `Enum`. This could indicate an issue in decoding the call sequence, or a missing feature. Please open an issue at https://github.com/crytic/fuzz-utils/issues"
)
case _:
handle_exit(
f"\n* The parameter tag `{param['tag']}` could not be found in the call object. This could indicate an issue in decoding the call sequence, or a missing feature. Please open an issue at https://github.com/crytic/test-generator/issues"
f"\n* The parameter tag `{param['tag']}` could not be found in the call object. This could indicate an issue in decoding the call sequence, or a missing feature. Please open an issue at https://github.com/crytic/fuzz-utils/issues"
)

def _decode_function_params(
Expand Down Expand Up @@ -270,7 +270,7 @@ def _decode_function_params(
case _:
# TODO should handle all cases, but keeping this just in case
CryticPrint().print_information(
f"\n* Attempted to decode an unidentified type {input_parameter}, this call will be skipped. Please open an issue at https://github.com/crytic/test-generator/issues"
f"\n* Attempted to decode an unidentified type {input_parameter}, this call will be skipped. Please open an issue at https://github.com/crytic/fuzz-utils/issues"
)
continue

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@
from slither.core.declarations.structure_contract import StructureContract
from slither.core.declarations.enum import Enum
from slither.core.declarations.enum_contract import EnumContract
from test_generator.utils.crytic_print import CryticPrint
from test_generator.templates.foundry_templates import templates
from test_generator.utils.encoding import parse_medusa_byte_string
from test_generator.utils.error_handler import handle_exit
from fuzz_utils.utils.crytic_print import CryticPrint
from fuzz_utils.templates.foundry_templates import templates
from fuzz_utils.utils.encoding import parse_medusa_byte_string
from fuzz_utils.utils.error_handler import handle_exit


class Medusa:
Expand Down Expand Up @@ -147,7 +147,7 @@ def _match_elementary_types(self, param: str, recursive: bool, input_parameter:
return param

handle_exit(
f"\n* The parameter type `{input_type}` could not be found in the call object. This could indicate an issue in decoding the call sequence, or a missing feature. Please open an issue at https://github.com/crytic/test-generator/issues"
f"\n* The parameter type `{input_type}` could not be found in the call object. This could indicate an issue in decoding the call sequence, or a missing feature. Please open an issue at https://github.com/crytic/fuzz-utils/issues"
)

def _match_array_type(
Expand Down Expand Up @@ -185,7 +185,7 @@ def _match_user_defined_type(
return "", f"{input_parameter}({param})" # type: ignore[unreachable]
case _:
handle_exit(
f"\n* The parameter type `{input_parameter.type}` could not be found in the call object. This could indicate an issue in decoding the call sequence, or a missing feature. Please open an issue at https://github.com/crytic/test-generator/issues"
f"\n* The parameter type `{input_parameter.type}` could not be found in the call object. This could indicate an issue in decoding the call sequence, or a missing feature. Please open an issue at https://github.com/crytic/fuzz-utils/issues"
)

def _decode_function_params(
Expand Down Expand Up @@ -224,7 +224,7 @@ def _decode_function_params(
case _:
# TODO should handle all cases, but keeping this just in case
CryticPrint().print_information(
f"\n* Attempted to decode an unidentified type {input_parameter}, this call will be skipped. Please open an issue at https://github.com/crytic/test-generator/issues"
f"\n* Attempted to decode an unidentified type {input_parameter}, this call will be skipped. Please open an issue at https://github.com/crytic/fuzz-utils/issues"
)
continue
else:
Expand Down Expand Up @@ -259,7 +259,7 @@ def _decode_function_params(
case _:
# TODO should handle all cases, but keeping this just in case
CryticPrint().print_information(
f"\n* Attempted to decode an unidentified type {input_parameter}, this call will be skipped. Please open an issue at https://github.com/crytic/test-generator/issues"
f"\n* Attempted to decode an unidentified type {input_parameter}, this call will be skipped. Please open an issue at https://github.com/crytic/fuzz-utils/issues"
)
continue

Expand Down
File renamed without changes.
18 changes: 9 additions & 9 deletions test_generator/main.py → fuzz_utils/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@

from slither import Slither
from slither.core.declarations.contract import Contract
from test_generator.utils.crytic_print import CryticPrint
from test_generator.templates.foundry_templates import templates
from test_generator.fuzzers.Medusa import Medusa
from test_generator.fuzzers.Echidna import Echidna
from test_generator.utils.error_handler import handle_exit
from fuzz_utils.utils.crytic_print import CryticPrint
from fuzz_utils.templates.foundry_templates import templates
from fuzz_utils.fuzzers.Medusa import Medusa
from fuzz_utils.fuzzers.Echidna import Echidna
from fuzz_utils.utils.error_handler import handle_exit


class FoundryTest:
Expand Down Expand Up @@ -97,7 +97,7 @@ def create_poc(self) -> str:
def main() -> None: # type: ignore[func-returns-value]
"""The main entry point"""
parser = argparse.ArgumentParser(
prog="test-generator", description="Generate test harnesses for Echidna failed properties."
prog="fuzz-utils", description="Generate test harnesses for Echidna failed properties."
)
parser.add_argument("file_path", help="Path to the Echidna test harness.")
parser.add_argument(
Expand Down Expand Up @@ -125,7 +125,7 @@ def main() -> None: # type: ignore[func-returns-value]
parser.add_argument(
"--version",
help="displays the current version",
version=require("test-generator")[0].version,
version=require("fuzz-utils")[0].version,
action="version",
)

Expand Down Expand Up @@ -157,10 +157,10 @@ def main() -> None: # type: ignore[func-returns-value]
CryticPrint().print_information(
f"Generating Foundry unit tests based on the {fuzzer.name} reproducers..."
)
test_generator = FoundryTest(
foundry_test = FoundryTest(
inheritance_path, target_contract, corpus_dir, test_directory, slither, fuzzer
)
test_generator.create_poc()
foundry_test.create_poc()
CryticPrint().print_success("Done!")


Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
""" Utility function for error handling"""
import sys
from typing import NoReturn
from test_generator.utils.crytic_print import CryticPrint
from fuzz_utils.utils.crytic_print import CryticPrint


def handle_exit(reason: str) -> NoReturn:
Expand Down
8 changes: 4 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@ requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "test_generator"
name = "fuzz-utils"
version = "0.0.1"
authors = [{ name = "Trail of Bits" }]
description = "A tool for automatically generating unit tests from Echidna and Medusa reproducers."
readme = "README.md"
license = { "text" = "AGPL-3.0" }
urls = { "Homepage" = "https://github.com/crytic/test-generator" }
urls = { "Homepage" = "https://github.com/crytic/fuzz-utils" }
requires-python = ">=3.10"
dependencies = [
"colorama>=0.4.0",
Expand All @@ -27,11 +27,11 @@ test = [
"solc-select>=0.1.4"
]
dev = [
"test_generator[lint,test]"
"fuzz_utils[lint,test]"
]

[project.scripts]
test-generator = "test_generator.main:main"
fuzz-utils = "fuzz_utils.main:main"

[tool.setuptools.packages.find]
where = ["."]
Expand Down
6 changes: 3 additions & 3 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
import pytest

from slither import Slither
from test_generator.main import FoundryTest
from test_generator.fuzzers.Echidna import Echidna
from test_generator.fuzzers.Medusa import Medusa
from fuzz_utils.main import FoundryTest
from fuzz_utils.fuzzers.Echidna import Echidna
from fuzz_utils.fuzzers.Medusa import Medusa


class TestGenerator:
Expand Down

0 comments on commit 9dcb85a

Please sign in to comment.