diff --git a/pkgs/tooling/monorepo_manager/README.md b/pkgs/tooling/monorepo_manager/README.md index dbffce72..28d632a1 100644 --- a/pkgs/tooling/monorepo_manager/README.md +++ b/pkgs/tooling/monorepo_manager/README.md @@ -1,13 +1,13 @@ # Monorepo Manager -**Monorepo Manager** is a unified command-line tool for managing a Python monorepo that contains multiple standalone packages, each with its own `pyproject.toml`. It consolidates several common tasks—including dependency management, version bumping, remote dependency resolution, test analysis, and project configuration updates—into one robust CLI. +**Monorepo Manager** is a unified command-line tool for managing a Python monorepo that contains multiple standalone packages—each with its own `pyproject.toml`. It consolidates common tasks such as dependency management, version bumping, remote dependency resolution, test execution and analysis, and project configuration updates into one robust CLI. ## Features - **Dependency Management** - **Lock:** Generate a `poetry.lock` file. - **Install:** Install dependencies with options for extras and development dependencies. - - **Show Freeze:** Display installed packages using `pip freeze`. + - **Show Freeze:** (Included via internal commands) Display installed packages using `pip freeze`. - **Version Management** - **Version:** Bump (major, minor, patch, finalize) or explicitly set package versions in `pyproject.toml`. @@ -16,38 +16,39 @@ - **Remote Fetch:** Fetch the version from a remote GitHub repository’s `pyproject.toml`. - **Remote Update:** Update a local `pyproject.toml` file with version information from remote Git dependencies. -- **Test Analysis** - - **Test:** Parse a JSON file with test results, display summary statistics, and evaluate threshold conditions. +- **Testing and Analysis** + - **Test:** Run your tests using pytest. Optionally, run tests in parallel (supports [pytest‑xdist](https://pypi.org/project/pytest-xdist/)). + - **Analyze:** Analyze test results provided in a JSON file, display summary statistics, and evaluate threshold conditions for passed/skipped tests. - **Pyproject Operations** - **Pyproject:** Extract local (path) and Git-based dependencies from a `pyproject.toml` file and optionally update dependency versions. ## Installation - ```bash - pip install monorepo-manager - ``` +Install using pip: -_This installs the `monorepo-manager` CLI (provided via the entry point `monorepo-manager`) into your system PATH._ +```bash +pip install monorepo-manager +``` + +_This command installs the `monorepo-manager` CLI, which is provided via the entry point `monorepo-manager`, into your system PATH._ ## Usage -Once installed, you can invoke the CLI using: +After installation, run the following to see a list of available commands: ```bash monorepo-manager --help ``` -This displays the list of available commands. - ### Command Examples #### 1. Lock Dependencies -Generate a `poetry.lock` file for a package by specifying the directory or file path: +Generate a `poetry.lock` file by specifying either a directory or a file path containing a `pyproject.toml`: ```bash -# Lock using a directory containing pyproject.toml: +# Lock using a directory: monorepo-manager lock --directory ./packages/package1 # Lock using an explicit pyproject.toml file: @@ -56,13 +57,13 @@ monorepo-manager lock --file ./packages/package1/pyproject.toml #### 2. Install Dependencies -Install dependencies with various options: +Install dependencies with options for extras and including development dependencies: ```bash # Basic installation: monorepo-manager install --directory ./packages/package1 -# Install using an explicit pyproject.toml file: +# Using an explicit pyproject.toml file: monorepo-manager install --file ./packages/package1/pyproject.toml # Install including development dependencies: @@ -77,13 +78,13 @@ monorepo-manager install --directory ./packages/package2 --all-extras #### 3. Version Management -Bump the version or set it explicitly for a given package: +Bump or explicitly set the version in a package's `pyproject.toml`: ```bash -# Bump the patch version (e.g. from 1.2.3.dev1 to 1.2.3.dev2): +# Bump the patch version (e.g., from 1.2.3.dev1 to 1.2.3.dev2): monorepo-manager version ./packages/package1/pyproject.toml --bump patch -# Finalize a development version (remove the ".dev" suffix): +# Finalize a dev version (remove the .dev suffix): monorepo-manager version ./packages/package1/pyproject.toml --bump finalize # Set an explicit version: @@ -92,38 +93,52 @@ monorepo-manager version ./packages/package1/pyproject.toml --set 2.0.0.dev1 #### 4. Remote Operations -Fetch version information from a remote GitHub repository’s `pyproject.toml` and update your local configuration accordingly. +Fetch remote version information and update your local dependency configuration: ```bash -# Fetch the remote version: +# Fetch the version from a remote GitHub repository's pyproject.toml: monorepo-manager remote fetch --git-url https://github.com/YourOrg/YourRepo.git --branch main --subdir "src/" -# Update a local pyproject.toml with versions resolved from remote dependencies. +# Update a local pyproject.toml with remote-resolved versions: # (If --output is omitted, the input file is overwritten.) monorepo-manager remote update --input ./packages/package1/pyproject.toml --output ./packages/package1/pyproject.updated.toml ``` -#### 5. Test Analysis +#### 5. Testing and Analysis -Analyze test results provided in a JSON file, and enforce percentage thresholds for passed and skipped tests: +Run your tests using pytest and analyze test results from a JSON report: -```bash -# Analyze test results without thresholds: -monorepo-manager test test-results.json +- **Run Tests:** + Execute tests in a specified directory. Use the `--num-workers` flag to run tests in parallel (requires pytest‑xdist). -# Analyze test results with thresholds: require that passed tests are greater than 75% and skipped tests are less than 20%: -monorepo-manager test test-results.json --required-passed gt:75 --required-skipped lt:20 -``` + ```bash + # Run tests sequentially: + monorepo-manager test --directory ./tests + + # Run tests in parallel using 4 workers: + monorepo-manager test --directory ./tests --num-workers 4 + ``` + +- **Analyze Test Results:** + Analyze a JSON test report and enforce thresholds for passed and skipped tests. + + ```bash + # Analyze test results without thresholds: + monorepo-manager analyze test-results.json + + # Analyze test results with thresholds (e.g., passed tests > 75% and skipped tests < 20%): + monorepo-manager analyze test-results.json --required-passed gt:75 --required-skipped lt:20 + ``` #### 6. Pyproject Operations -Operate on the `pyproject.toml` file to extract dependency information and optionally update dependency versions: +Extract and update dependency information from a `pyproject.toml` file: ```bash # Extract local (path) and Git-based dependencies: monorepo-manager pyproject --pyproject ./packages/package1/pyproject.toml -# Update local dependency versions to 2.0.0 (updates parent file and, if possible, each dependency's own pyproject.toml): +# Update local dependency versions to 2.0.0 (updates the parent file and, if possible, each dependency's own pyproject.toml): monorepo-manager pyproject --pyproject ./packages/package1/pyproject.toml --update-version 2.0.0 ``` @@ -135,7 +150,7 @@ monorepo-manager pyproject --pyproject ./packages/package1/pyproject.toml --upda monorepo_manager/ ├── __init__.py ├── cli.py # Main CLI entry point -├── poetry_ops.py # Poetry operations (lock, install, build, publish, etc.) +├── poetry_ops.py # Poetry operations (lock, install, build, publish, run tests, etc.) ├── version_ops.py # Version bumping and setting operations ├── remote_ops.py # Remote Git dependency version fetching/updating ├── test_ops.py # Test result analysis operations @@ -144,14 +159,6 @@ pyproject.toml # Package configuration file containing metadata README.md # This file ``` -### Running Tests - -For development purposes, you can use your favorite test runner (such as `pytest`) to run tests for the CLI and modules. - -```bash -pytest -``` - ## Contributing Contributions are welcome! Feel free to open issues or submit pull requests for improvements or bug fixes. @@ -161,6 +168,3 @@ Contributions are welcome! Feel free to open issues or submit pull requests for This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details. ``` ---- - -This `README.md` provides a detailed overview, installation instructions, and comprehensive usage examples for each command offered by your CLI tool. Feel free to adjust sections or add any additional details specific to your project needs. \ No newline at end of file diff --git a/pkgs/tooling/monorepo_manager/monorepo_manager/cli.py b/pkgs/tooling/monorepo_manager/monorepo_manager/cli.py index c3c4d319..b185ce8b 100644 --- a/pkgs/tooling/monorepo_manager/monorepo_manager/cli.py +++ b/pkgs/tooling/monorepo_manager/monorepo_manager/cli.py @@ -7,11 +7,12 @@ - Manage Poetry-based operations (lock, install, show pip-freeze, recursive build, publish) - Manage version operations (bump or set versions in pyproject.toml) - Manage remote operations (fetch/update Git dependency versions) + - Run tests using pytest (with optional parallelism) - Analyze test results from a JSON file - Operate on pyproject.toml files (extract and update dependency versions) The commands are intentionally named with simple terms (e.g. "lock" instead of "poetry lock", -"install" instead of "poetry install", and "test" instead of "test-analyze"). +"install" instead of "poetry install"). """ import argparse @@ -76,12 +77,19 @@ def main(): update_parser.add_argument("--output", help="Optional output file path (defaults to overwriting the input)") # ------------------------------------------------ - # Command: test + # Command: test (run pytest) # ------------------------------------------------ - test_parser = subparsers.add_parser("test", help="Analyze test results from a JSON file") - test_parser.add_argument("file", help="Path to the JSON file with test results") - test_parser.add_argument("--required-passed", type=str, help="Threshold for passed tests (e.g. 'gt:75')") - test_parser.add_argument("--required-skipped", type=str, help="Threshold for skipped tests (e.g. 'lt:20')") + test_parser = subparsers.add_parser("test", help="Run tests using pytest") + test_parser.add_argument("--directory", type=str, default=".", help="Directory to run tests in (default: current directory)") + test_parser.add_argument("--num-workers", type=int, default=1, help="Number of workers to use for parallel testing (requires pytest-xdist)") + + # ------------------------------------------------ + # Command: analyze (analyze test results from JSON) + # ------------------------------------------------ + analyze_parser = subparsers.add_parser("analyze", help="Analyze test results from a JSON file") + analyze_parser.add_argument("file", help="Path to the JSON file with test results") + analyze_parser.add_argument("--required-passed", type=str, help="Threshold for passed tests (e.g. 'gt:75')") + analyze_parser.add_argument("--required-skipped", type=str, help="Threshold for skipped tests (e.g. 'lt:20')") # ------------------------------------------------ # Command: pyproject @@ -127,6 +135,10 @@ def main(): sys.exit(1) elif args.command == "test": + # Run pytest (with optional parallelism if --num-workers > 1) + poetry_ops.run_pytests(test_directory=args.directory, num_workers=args.num_workers) + + elif args.command == "analyze": test_ops.analyze_test_file( file_path=args.file, required_passed=args.required_passed, diff --git a/pkgs/tooling/monorepo_manager/monorepo_manager/poetry_ops.py b/pkgs/tooling/monorepo_manager/monorepo_manager/poetry_ops.py index 71bbb70f..71b87ea3 100644 --- a/pkgs/tooling/monorepo_manager/monorepo_manager/poetry_ops.py +++ b/pkgs/tooling/monorepo_manager/monorepo_manager/poetry_ops.py @@ -135,116 +135,6 @@ def show_pip_freeze(): run_command("pip freeze") -def set_version_in_pyproject(version, directory=None, file=None): - """ - Set the version for the package in the pyproject.toml file(s). - - :param version: The new version string to set. - :param directory: If provided, recursively update all pyproject.toml files found in the directory. - :param file: If provided, update the specific pyproject.toml file. - """ - def update_file(pyproject_file, version): - print(f"Setting version to {version} in {pyproject_file}...") - try: - with open(pyproject_file, "r") as f: - data = toml.load(f) - except Exception as e: - print(f"Error reading {pyproject_file}: {e}", file=sys.stderr) - return - - if "tool" in data and "poetry" in data["tool"]: - data["tool"]["poetry"]["version"] = version - else: - print(f"Invalid pyproject.toml structure in {pyproject_file}.", file=sys.stderr) - return - - try: - with open(pyproject_file, "w") as f: - toml.dump(data, f) - print(f"Version set to {version} in {pyproject_file}.") - except Exception as e: - print(f"Error writing {pyproject_file}: {e}", file=sys.stderr) - - if directory: - for root, _, files in os.walk(directory): - if "pyproject.toml" in files: - pyproject_file = os.path.join(root, "pyproject.toml") - update_file(pyproject_file, version) - elif file: - update_file(file, version) - else: - print("Error: A directory or a file must be specified.", file=sys.stderr) - sys.exit(1) - - -def set_dependency_versions(version, directory=None, file=None): - """ - Update versions for path dependencies found in pyproject.toml files. - - For each dependency that is defined with a "path", update its version to '^'. - Also attempts to update the dependency's own pyproject.toml. - - :param version: The new version string (without the caret). - :param directory: A directory to search for pyproject.toml files. - :param file: A specific pyproject.toml file to update. - """ - def update_file(pyproject_file, version): - print(f"Setting dependency versions to {version} in {pyproject_file}...") - try: - with open(pyproject_file, "r") as f: - data = toml.load(f) - except Exception as e: - print(f"Error reading {pyproject_file}: {e}", file=sys.stderr) - return - - dependencies = data.get("tool", {}).get("poetry", {}).get("dependencies", {}) - updated_dependencies = {} - - for dep_name, dep_value in dependencies.items(): - if isinstance(dep_value, dict) and "path" in dep_value: - # Update the version entry while preserving other keys except 'path' - updated_dep = {"version": f"^{version}"} - for key, val in dep_value.items(): - if key != "path": - updated_dep[key] = val - updated_dependencies[dep_name] = updated_dep - - # Update the dependency's own pyproject.toml file if present. - dependency_path = os.path.join(os.path.dirname(pyproject_file), dep_value["path"]) - dependency_pyproject = os.path.join(dependency_path, "pyproject.toml") - if os.path.isfile(dependency_pyproject): - print(f"Updating version in dependency: {dependency_pyproject} to {version}") - set_version_in_pyproject(version, file=dependency_pyproject) - else: - print(f"Warning: pyproject.toml not found at {dependency_pyproject}") - else: - updated_dependencies[dep_name] = dep_value - - # Write back the updated dependencies. - if "tool" in data and "poetry" in data["tool"]: - data["tool"]["poetry"]["dependencies"] = updated_dependencies - else: - print(f"Error: Invalid pyproject.toml structure in {pyproject_file}.", file=sys.stderr) - return - - try: - with open(pyproject_file, "w") as f: - toml.dump(data, f) - print(f"Dependency versions set to {version} in {pyproject_file}.") - except Exception as e: - print(f"Error writing {pyproject_file}: {e}", file=sys.stderr) - - if directory: - for root, _, files in os.walk(directory): - if "pyproject.toml" in files: - update_file(os.path.join(root, "pyproject.toml"), version) - elif file: - update_file(file, version) - else: - print("Error: A directory or a file must be specified.", file=sys.stderr) - sys.exit(1) - - def publish_package(directory=None, file=None, username=None, password=None): """ Build and publish packages to PyPI. @@ -306,3 +196,23 @@ def publish_from_dependencies(directory=None, file=None, username=None, password ) else: print(f"Skipping {full_path}: not a valid package directory") + +def run_pytests(test_directory=".", num_workers=1): + """ + Run pytest in the specified directory. + + If num_workers is greater than 1, uses pytest‑xdist to run tests in parallel. + + :param test_directory: Directory in which to run tests (default: current directory). + :param num_workers: Number of workers to use (default: 1). Requires pytest-xdist when > 1. + """ + command = "pytest" + try: + workers = int(num_workers) + except ValueError: + print("Error: num_workers must be an integer", file=sys.stderr) + sys.exit(1) + if workers > 1: + command += f" -n {workers}" + print(f"Running tests in '{test_directory}' with command: {command}") + run_command(command, cwd=test_directory) \ No newline at end of file