Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conda build maintenance #188

Merged
merged 9 commits into from
Jan 21, 2025
Merged

Conversation

fabianegli
Copy link
Collaborator

@fabianegli fabianegli commented Jan 21, 2025

User description

This PR fixes the conda package build and includes some code maintenance.

  • The reordering of the conda channels shortens the build time.
  • Setting the package output format fixes the broken build
  • Using conda to get the package path makes the Action more robust.

PR Type

Enhancement, Configuration changes


Description

  • Added default shell configuration for workflow steps.

  • Specified package output format in conda build.

  • Improved indentation for better readability and maintenance.


Changes walkthrough 📝

Relevant files
Configuration changes
conda-build.yml
Enhanced conda build workflow configuration                           

.github/workflows/conda-build.yml

  • Added defaults section to set a default shell.
  • Specified .tar.bz2 as the package output format.
  • Improved indentation for workflow steps.
  • Streamlined workflow configuration for better clarity.
  • +49/-52 

    Need help?
  • Type /help how to ... in the comments thread for any question about Qodo Merge usage.
  • Check out the documentation for more information.
  • Summary by CodeRabbit

    • Chores
      • Updated GitHub Actions workflow configuration for conda build process
      • Modified default shell to bash for workflow steps
      • Simplified dependency installation process
      • Consolidated steps for setting up Miniconda and activating the environment
      • Updated package build command to specify .tar.bz2 package format

    Copy link
    Contributor

    coderabbitai bot commented Jan 21, 2025

    Warning

    Rate limit exceeded

    @fabianegli has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 0 minutes and 17 seconds before requesting another review.

    ⌛ How to resolve this issue?

    After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

    We recommend that you space out your commits to avoid hitting the rate limit.

    🚦 How do rate limits work?

    CodeRabbit enforces hourly rate limits for each developer per organization.

    Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

    Please see our FAQ for further information.

    📥 Commits

    Reviewing files that changed from the base of the PR and between f1bc7af and 6978776.

    📒 Files selected for processing (1)
    • .github/workflows/conda-build.yml (1 hunks)

    Walkthrough

    The pull request modifies the GitHub Actions workflow file for Conda build, introducing a defaults section in the build job to standardize the shell environment to bash -l {0}. The workflow steps have been restructured and renamed, consolidating the setup for Miniconda and simplifying the dependency installation process. The package build command has been updated to specify the .tar.bz2 package format, while the testing steps have been reformatted for clarity. Overall, the changes enhance the organization of the workflow while maintaining the original functionality.

    Changes

    File Change Summary
    .github/workflows/conda-build.yml Added defaults section with bash -l {0} shell configuration
    Restructured and renamed workflow steps
    Updated Miniconda setup with new parameters
    Simplified dependency installation to a single step
    Updated package build command to include --package-format .tar.bz2 and --output flag
    Merged environment activation and package building into a single step
    Reformatted testing steps for clarity

    Poem

    🐰 A Conda Workflow Tale 🐰
    With bash and defaults so neat,
    Our build process now looks sweet
    .tar.bz2 format in sight
    GitHub Actions shining bright
    A rabbit's workflow delight! 🚀


    Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

    ❤️ Share
    🪧 Tips

    Chat

    There are 3 ways to chat with CodeRabbit:

    • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
      • I pushed a fix in commit <commit_id>, please review it.
      • Generate unit testing code for this file.
      • Open a follow-up GitHub issue for this discussion.
    • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
      • @coderabbitai generate unit testing code for this file.
      • @coderabbitai modularize this function.
    • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
      • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
      • @coderabbitai read src/utils.ts and generate unit testing code.
      • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
      • @coderabbitai help me debug CodeRabbit configuration file.

    Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

    CodeRabbit Commands (Invoked using PR comments)

    • @coderabbitai pause to pause the reviews on a PR.
    • @coderabbitai resume to resume the paused reviews.
    • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
    • @coderabbitai full review to do a full review from scratch and review all the files again.
    • @coderabbitai summary to regenerate the summary of the PR.
    • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
    • @coderabbitai resolve resolve all the CodeRabbit review comments.
    • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
    • @coderabbitai help to get help.

    Other keywords and placeholders

    • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
    • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
    • Add @coderabbitai anywhere in the PR title to generate the title automatically.

    CodeRabbit Configuration File (.coderabbit.yaml)

    • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
    • Please see the configuration documentation for more information.
    • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

    Documentation and Community

    • Visit our Documentation for detailed information on how to use CodeRabbit.
    • Join our Discord Community to get help, request features, and share feedback.
    • Follow us on X/Twitter for updates and announcements.

    Copy link

    Qodo Merge was enabled for this repository. To continue using it, please link your Git account with your Qodo account here.

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 1 🔵⚪⚪⚪⚪
    🧪 PR contains tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Package Format

    The explicit package format .tar.bz2 was added. While this is a valid format, verify if this specific format is required or if the default format would be sufficient.

    conda build --package-format .tar.bz2 recipe

    Copy link

    qodo-merge-pro bot commented Jan 21, 2025

    Qodo Merge was enabled for this repository. To continue using it, please link your Git account with your Qodo account here.

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Score
    Possible issue
    ✅ Add error handling for build
    Suggestion Impact:The commit changed the package path search approach entirely by using conda build --output instead of find command, which is a more reliable way to get the package path and eliminates the need for explicit error handling

    code diff:

    -          PACKAGE_PATH=$(find /usr/share/miniconda/envs/sdrf-pipelines/conda-bld/noarch/ -name "sdrf-pipelines-*.tar.bz2" | head -n 1)
    +          PACKAGE_PATH=$(conda build --package-format .tar.bz2 --output recipe)

    Add error handling for the package path search to prevent potential pipeline
    failures if the build output is not found.

    .github/workflows/conda-build.yml [42-43]

     PACKAGE_PATH=$(find /usr/share/miniconda/envs/sdrf-pipelines/conda-bld/noarch/ -name "sdrf-pipelines-*.tar.bz2" | head -n 1)
    +if [ -z "$PACKAGE_PATH" ]; then
    +  echo "Error: Built package not found"
    +  exit 1
    +fi
     conda install --offline "$PACKAGE_PATH"
    • Apply this suggestion
    Suggestion importance[1-10]: 8

    Why: Adding error handling for missing build artifacts is crucial for CI pipeline reliability, as it prevents silent failures and provides clear error messages when builds fail.

    8
    Handle package installation failures

    Add explicit failure condition for the package installation step to ensure the
    workflow fails if installation encounters issues.

    .github/workflows/conda-build.yml [43]

    -conda install --offline "$PACKAGE_PATH"
    +conda install --offline "$PACKAGE_PATH" || exit 1
    • Apply this suggestion
    Suggestion importance[1-10]: 7

    Why: Explicitly handling package installation failures ensures the workflow fails appropriately rather than potentially continuing with an unsuccessful installation, improving pipeline reliability.

    7

    Copy link
    Contributor

    @coderabbitai coderabbitai bot left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Actionable comments posted: 2

    🧹 Nitpick comments (1)
    .github/workflows/conda-build.yml (1)

    23-58: Consider optimizing environment activations.

    The workflow activates the conda environment multiple times. Consider consolidating steps or using a composite action to reduce repetition.

    Example approach:

          - name: Build and test
            run: |
              source "$CONDA/etc/profile.d/conda.sh"
              conda activate sdrf-pipelines
              
              # Install build tools
              conda install conda-build anaconda-client
              
              # Build and install
              conda build --package-format .tar.bz2 recipe
              PACKAGE_PATH=$(find /usr/share/miniconda/envs/sdrf-pipelines/conda-bld/noarch/ -name "sdrf-pipelines-*.tar.bz2" -type f -print -quit)
              conda install --offline "$PACKAGE_PATH"
              
              # Run tests
              parse_sdrf --help
              parse_sdrf validate-sdrf --sdrf_file tests/data/reference/PDC000126/PDC000126.sdrf.tsv
              parse_sdrf validate-sdrf --sdrf_file tests/data/reference/PDC000126/PDC000126.sdrf.tsv --use_ols_cache_only
    🧰 Tools
    🪛 actionlint (1.7.4)

    27-27: shellcheck reported issue in this script: SC2086:info:1:8: Double quote to prevent globbing and word splitting

    (shellcheck)


    32-32: shellcheck reported issue in this script: SC2086:info:1:8: Double quote to prevent globbing and word splitting

    (shellcheck)


    38-38: shellcheck reported issue in this script: SC2086:info:1:8: Double quote to prevent globbing and word splitting

    (shellcheck)

    📜 Review details

    Configuration used: CodeRabbit UI
    Review profile: CHILL
    Plan: Pro

    📥 Commits

    Reviewing files that changed from the base of the PR and between 7eb463e and 82e4618.

    📒 Files selected for processing (1)
    • .github/workflows/conda-build.yml (1 hunks)
    🧰 Additional context used
    🪛 actionlint (1.7.4)
    .github/workflows/conda-build.yml

    27-27: shellcheck reported issue in this script: SC2086:info:1:8: Double quote to prevent globbing and word splitting

    (shellcheck)


    32-32: shellcheck reported issue in this script: SC2086:info:1:8: Double quote to prevent globbing and word splitting

    (shellcheck)


    38-38: shellcheck reported issue in this script: SC2086:info:1:8: Double quote to prevent globbing and word splitting

    (shellcheck)

    ⏰ Context from checks skipped due to timeout of 90000ms (2)
    • GitHub Check: build
    • GitHub Check: Codacy Static Code Analysis
    🔇 Additional comments (3)
    .github/workflows/conda-build.yml (3)

    8-10: LGTM! Good practice using login shell.

    Setting bash -l {0} as the default shell ensures proper conda initialization and environment activation across different runner environments.


    13-22: LGTM! Good configuration of Miniconda setup.

    Using latest action versions and explicit Python version specification with appropriate channel configuration.


    45-58: LGTM! Comprehensive test coverage.

    Good test strategy:

    1. Basic functionality check with --help
    2. Validation with real data
    3. Cache-only validation test

    Copy link
    Contributor

    @coderabbitai coderabbitai bot left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Actionable comments posted: 0

    🧹 Nitpick comments (2)
    .github/workflows/conda-build.yml (2)

    26-28: Consider pinning dependency versions.

    While the essential build tools are included, consider pinning their versions for better reproducibility.

    -          conda install conda-build conda-verify anaconda-client
    +          conda install conda-build=3.27.0 conda-verify=3.3.1 anaconda-client=1.12.0

    36-38: Consider adding more comprehensive package tests.

    The current test only verifies that the command exists. Consider adding tests for basic functionality.

    -          parse_sdrf --help
    +          # Basic command existence test
    +          parse_sdrf --help
    +          # Version check
    +          parse_sdrf --version
    +          # Test basic parsing functionality
    +          parse_sdrf convert-to-json --help
    📜 Review details

    Configuration used: CodeRabbit UI
    Review profile: CHILL
    Plan: Pro

    📥 Commits

    Reviewing files that changed from the base of the PR and between 82e4618 and f1bc7af.

    📒 Files selected for processing (1)
    • .github/workflows/conda-build.yml (1 hunks)
    ⏰ Context from checks skipped due to timeout of 90000ms (2)
    • GitHub Check: build
    • GitHub Check: Codacy Static Code Analysis
    🔇 Additional comments (4)
    .github/workflows/conda-build.yml (4)

    8-10: LGTM! Good shell configuration.

    Setting bash -l {0} as the default shell ensures proper conda initialization across all steps.


    30-34: LGTM! Well-structured build process.

    Good practices implemented:

    • Using .tar.bz2 format for better compatibility
    • Capturing package path efficiently
    • Installing package offline

    40-46: LGTM! Verify test data availability.

    The validation tests are well-structured with both online and offline cache scenarios.

    Let's verify the test data exists:

    ✅ Verification successful

    Test data verification successful

    The referenced SDRF test file exists at the specified location and is accessible for the validation tests.

    🏁 Scripts executed

    The following scripts were executed for the analysis:

    Script:

    #!/bin/bash
    # Verify test data existence
    if [ -f "tests/data/reference/PDC000126/PDC000126.sdrf.tsv" ]; then
        echo "Test data exists"
    else
        echo "Test data not found"
    fi
    

    Length of output: 105


    16-24: LGTM! Verify environment file existence.

    The Miniconda setup is well-configured with explicit versions and correct channel order.

    Let's verify the referenced environment file exists:

    ✅ Verification successful

    Environment file verification successful ✓

    The environment.yml file exists and its configuration aligns perfectly with the workflow settings, including matching environment name, correct channel order, and Python 3.10 specification.

    🏁 Scripts executed

    The following scripts were executed for the analysis:

    Script:

    #!/bin/bash
    # Verify environment.yml exists and contains valid conda configuration
    if [ -f "environment.yml" ]; then
        echo "environment.yml exists"
        cat environment.yml
    else
        echo "environment.yml not found"
    fi
    

    Length of output: 393

    @fabianegli
    Copy link
    Collaborator Author

    @ypriverol Feel free to merge if you think it's good.

    @ypriverol ypriverol self-requested a review January 21, 2025 15:43
    @ypriverol ypriverol merged commit e76d014 into bigbio:main Jan 21, 2025
    12 checks passed
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    2 participants