Re-running review when PR is synchronized

[nataliezeller1]

I'm running the pr-agent as a git action. When a PR is opened, the review is running and a comment is added.
Is there a way to re-run review automatically also when the PR gets synchronized with additional commits?

[mrT23]

This is currently supported only in GitHub App
https://github.com/Codium-ai/pr-agent/blob/main/Usage.md#github-app-automatic-tools-for-new-code-pr-push

[ken109]

@nataliezeller1
If you set up the workflow as below, the workflow will start twice, but PR-Agent can be started for each syncronize event.

name: "PR Agent"

on:
  pull_request:
  issue_comment:

jobs:
  pr_agent_job:
    runs-on: ubuntu-latest
    name: Run pr agent on every pull request, respond to user comments
    permissions:
      issues: write
      pull-requests: write
      contents: write
    steps:
      - name: Review command
        if: github.event_name == 'pull_request' && github.event.action == 'synchronize'
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          gh pr comment ${{ github.event.pull_request.number }} --repo ${{ github.repository }} --body "/review"

      - name: PR Agent action step
        id: pragent
        uses: Codium-ai/pr-agent@main
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          OPENAI_KEY: ${{ secrets.OPENAI_KEY }}

[nataliezeller1]

Thanks @ken109
For some reason it doesn't work for me.
When I add a "/review" comment to the PR manually, the pr-agent runs and provides a feedback.
When it happens from the workflow, the "/review" comment is added but the pr-agent is not triggered...

[ken109]

@nataliezeller1
I wonder why,,
Try using ${{ secrets.YOUR_PAT }} instead of ${{ secrets.GITHUB_TOKEN }} as the token when commenting "/review".

[nataliezeller1]

This is currently supported only in GitHub App https://github.com/Codium-ai/pr-agent/blob/main/Usage.md#github-app-automatic-tools-for-new-code-pr-push

Thanks @mrT23
I've tried to install it using github app, and set the following in a pr_agent.toml file under the root of my repo:

[github_app]
handle_push_trigger=true

The pr-agent is still not triggered when new code is pushed to an open PR. I can see this message in the logs:
{"text": "event or action does not require handling\n", "record": {"elapsed": {"repr": "5:14:48.723584", "seconds": 18888.723584}, "exception": null, "extra": {}, "file": {"name": "github_app.py", "path": "/app/pr_agent/servers/github_app.py"}, "function": "handle_request", "level": {"icon": "ℹ️", "name": "INFO", "no": 20}, "line": 185, "message": "event or action does not require handling", "module": "github_app", "name": "__main__", "process": {"id": 1, "name": "MainProcess"}, "thread": {"id": 139831870700416, "name": "MainThread"}, "time": {"repr": "2023-12-06 08:22:26.614593+00:00", "timestamp": 1701850946.614593}}}

[mrT23]

@nataliezeller1
have you added the push commands ?

for example:

push_commands = [
    "/describe --pr_description.add_original_user_description=true --pr_description.keep_original_user_title=true",
    "/auto_review ",
]

you need to define which tools will run each push.

if it still doesn't work, please send a full og

[nataliezeller1]

@mrT23 yes I included the push_commands and it didn't help.
I could make it work now only after setting handle_push_trigger=true in the pr-agent app itself.
This parameter wasn't overriden when I set it on my repo where I install the Git app - is that expected?

[mrT23]

"I've tried to install it using github app, and set the following in a pr_agent.toml file under the root of my repo:"

.pr_agent.toml
(With dot at the beginning.)

Without a full log, I cannot understand if there are other problems.
A log would show if a .pr_agent.toml file was even loaded, or not

[nataliezeller1]

yes there is a dot at the beginning of the file - .pr_agent.toml
here is the full log. I can tell that the .pr_agent.toml is loaded since it's referring to the other settings I've put in the file, such as extra_instructions.
logs-from-codiumai-in-codiumai-689577d5d8-st4sz.log

[okotek]

@nataliezeller1 did you enable push events in the app settings?

[nataliezeller1]

@okotek we tried to enable push events in the app, but it still ignores them (although we set handle_push_trigger=true in the configuration file in the repo).
The only thing that seems to helps is to set handle_push_trigger=true in the pr-agent app...
This is the content of my .pr-agent.toml in the repo:

[github_app]
handle_push_trigger=true
push_commands=[
    """
    /auto_review --pr_reviewer.remove_previous_review_comment=false --pr_reviewer.require_score_review=true --pr_reviewer.require_tests_review=true --pr_reviewer.persistent_comment=true """
]

[mrT23]

@nataliezeller1
are you able to get any configuration change from .pr-agent.toml ?
try something far more simpler, like:

[pr_reviewer]
num_code_suggestions=1

Make sure the file is in the root of the repo's main branch prior to creating a PR.

[nataliezeller1]

yes the other configurations in this file are reflected, just not the handle_push_trigger.
the file is in the root of the repo's main

[mrT23]

@nataliezeller1
Thanks for the feedback and the patience.

I have identified an issue with 'synchronize' events in GithubApp, and opened a PR to fix it:
#511

[nataliezeller1]

Thanks for handling this so quickly!

[mrT23]

@nataliezeller1 we merged #511
you are welcome to test it and give feedback if it works

[nataliezeller1]

@mrT23 It works, thanks!