Skip to content

Commit

Permalink
Merge pull request #1230 from Codium-ai/tr/dynamic
Browse files Browse the repository at this point in the history
Tr/dynamic
  • Loading branch information
mrT23 authored Sep 15, 2024
2 parents 2102c51 + fdb1ff8 commit 146b882
Show file tree
Hide file tree
Showing 3 changed files with 38 additions and 11 deletions.
39 changes: 33 additions & 6 deletions docs/docs/tools/improve.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,9 +98,12 @@ the tool can automatically approve the PR when the user checks the self-review c

![self_review_2](https://codium.ai/images/pr_agent/self_review_2.png){width=512}

### `Extra instructions` and `best practices`
### 'Extra instructions' and 'best practices'

#### Extra instructions

>`Platforms supported: GitHub, GitLab, Bitbucket`
You can use the `extra_instructions` configuration option to give the AI model additional instructions for the `improve` tool.
Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify relevant aspects that you want the model to focus on.

Expand All @@ -117,25 +120,49 @@ extra_instructions="""\
Use triple quotes to write multi-line instructions. Use bullet points or numbers to make the instructions more readable.

#### Best practices 💎

>`Platforms supported: GitHub, GitLab`
Another option to give additional guidance to the AI model is by creating a dedicated [**wiki page**](https://github.com/Codium-ai/pr-agent/wiki) called `best_practices.md`.
This page can contain a list of best practices, coding standards, and guidelines that are specific to your repo/organization
This page can contain a list of best practices, coding standards, and guidelines that are specific to your repo/organization.

The AI model will use this page as a reference, and in case the PR code violates any of the guidelines, it will suggest improvements accordingly, with a dedicated label: `Organization
The AI model will use this wiki page as a reference, and in case the PR code violates any of the guidelines, it will suggest improvements accordingly, with a dedicated label: `Organization
best practice`.

Example for a `best_practices.md` content can be found [here](https://github.com/Codium-ai/pr-agent/blob/main/docs/docs/usage-guide/EXAMPLE_BEST_PRACTICE.md) (adapted from Google's [pyguide](https://google.github.io/styleguide/pyguide.html)).
This file is only an example. Since it is used as a prompt for an AI model, we want to emphasize the following:

- It should be written in a clear and concise manner
- If needed, it should give short relevant code snippets as examples
- Up to 800 lines are allowed
- Recommended to limit the text to 800 lines or fewer. Here’s why:

1) Extremely long best practices documents may not be fully processed by the AI model.

2) A lengthy file probably represent a more "**generic**" set of guidelines, which the AI model is already familiar with. The objective is to focus on a more targeted set of guidelines tailored to the specific needs of this project.

##### Local and global best practices
By default, PR-Agent will look for a local `best_practices.md` wiki file in the root of the relevant local repo.

If you want to enable also a global `best_practices.md` wiki file, set first in the global configuration file:

```
[best_practices]
enable_global_best_practices = false
```

Example results:
Then, create a `best_practices.md` wiki file in the root of [global](https://pr-agent-docs.codium.ai/usage-guide/configuration_options/#global-configuration-file) configuration repository, `pr-agent-settings`.

##### Example results

![best_practice](https://codium.ai/images/pr_agent/org_best_practice.png){width=512}

Note that while the `extra instructions` are more related to the way the `improve` tool behaves, the `best_practices.md` file is a general guideline for the way code should be written in the repo.

#### How to combine `extra instructions` and `best practices`

The `extra instructions` configuration is more related to the `improve` tool prompt. It can be used, for example, to avoid specific suggestions ("Don't suggest to add try-except block", "Ignore changes in toml files", ...) or to emphasize specific aspects or formats ("Answer in Japanese", "Give only short suggestions", ...)

In contrast, the `best_practices.md` file is a general guideline for the way code should be written in the repo.

Using a combination of both can help the AI model to provide relevant and tailored suggestions.

## Configuration options
Expand Down
8 changes: 4 additions & 4 deletions pr_agent/algo/ai_handlers/litellm_ai_handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ async def chat_completion(self, model: str, system: str, user: str, temperature:
if self.azure:
model = 'azure/' + model
if 'claude' in model and not system:
system = "\n"
system = "No system prompt provided"
get_logger().warning(
"Empty system prompt for claude model. Adding a newline character to prevent OpenAI API error.")
messages = [{"role": "system", "content": system}, {"role": "user", "content": user}]
Expand Down Expand Up @@ -215,13 +215,13 @@ async def chat_completion(self, model: str, system: str, user: str, temperature:

response = await acompletion(**kwargs)
except (openai.APIError, openai.APITimeoutError) as e:
get_logger().warning("Error during OpenAI inference: ", e)
get_logger().warning(f"Error during LLM inference: {e}")
raise
except (openai.RateLimitError) as e:
get_logger().error("Rate limit error during OpenAI inference: ", e)
get_logger().error(f"Rate limit error during LLM inference: {e}")
raise
except (Exception) as e:
get_logger().warning("Unknown error during OpenAI inference: ", e)
get_logger().warning(f"Unknown error during LLM inference: {e}")
raise openai.APIError from e
if response is None or len(response["choices"]) == 0:
raise openai.APIError
Expand Down
2 changes: 1 addition & 1 deletion pr_agent/settings/configuration.toml
Original file line number Diff line number Diff line change
Expand Up @@ -299,7 +299,7 @@ number_of_results = 5

[lancedb]
uri = "./lancedb"

[best_practices]
content = ""
max_lines_allowed = 800
enable_global_best_practices = false

0 comments on commit 146b882

Please sign in to comment.