Merge pull request #1309 from Codium-ai/tr/suggestion_tracking

Tr/suggestion tracking

README.md

@@ -43,6 +43,17 @@ Qode Merge PR-Agent aims to help efficiently review and handle pull requests, by
 
 ## News and Updates
 
+### October 27, 2024
+
+Qodo Merge PR Agent will now automatically document accepted code suggestions in a dedicated wiki page (`.pr_agent_accepted_suggestions`), enabling users to track historical changes, assess the tool's effectiveness, and learn from previously implemented recommendations in the repository.
+
+This dedicated wiki page will also serve as a foundation for future AI model improvements, allowing it to learn from historically implemented suggestions and generate more targeted, contextually relevant recommendations.
+Read more about this novel feature [here](https://qodo-merge-docs.qodo.ai/tools/improve/#suggestion-tracking).
+
+<kbd><img href="https://qodo.ai/images/pr_agent/pr_agent_accepted_suggestions1.png" src="https://qodo.ai/images/pr_agent/pr_agent_accepted_suggestions1.png" width="768"></kbd>
+
+
+
 ### October 21, 2024
 **Disable publishing labels by default:** 
 
@@ -71,14 +82,6 @@ New ability for the `review` tool - **ticket compliance feedback**. If the PR co
 <kbd><img src="https://github.com/user-attachments/assets/4a2a728b-5f47-40fa-80cc-16efd296938c" width="768"></kbd>
 
 
-### September 21, 2024
-Need help with PR-Agent? New feature - simply comment `/help "your question"` in a pull request, and PR-Agent will provide you with the [relevant documentation](https://github.com/Codium-ai/pr-agent/pull/1241#issuecomment-2365259334).
-
-![image](https://github.com/user-attachments/assets/7c214d9c-e7bb-4028-83b6-9515d7310d19)
-<img src="https://www.codium.ai/images/pr_agent/pr_help_chat.png" width="768"></kbd>
-
-
-
 ## Overview
 <div style="text-align:left;">
 

docs/docs/tools/improve.md

@@ -67,6 +67,33 @@ In post-process, Qodo Merge counts the number of suggestions that were implement
 
 ![code_suggestions_asses_impact_stats_2](https://codium.ai/images/pr_agent/code_suggestions_asses_impact_stats_2.png){width=512}
 
+## Suggestion tracking 💎
+`Platforms supported: GitHub, GitLab`
+
+Qodo Merge employs an novel detection system to automatically [identify](https://qodo-merge-docs.qodo.ai/core-abilities/impact_evaluation/) AI code suggestions that PR authors have accepted and implemented.
+
+Accepted suggestions are also automatically documented in a dedicated wiki page called `.pr_agent_accepted_suggestions`, allowing users to track historical changes, assess the tool's effectiveness, and learn from previously implemented recommendations in the repository.
+An example [result](https://github.com/Codium-ai/pr-agent/wiki/.pr_agent_accepted_suggestions):
+
+[![pr_agent_accepted_suggestions1.png](https://qodo.ai/images/pr_agent/pr_agent_accepted_suggestions1.png){width=768}](https://github.com/Codium-ai/pr-agent/wiki/.pr_agent_accepted_suggestions)
+
+This dedicated wiki page will also serve as a foundation for future AI model improvements, allowing it to learn from historically implemented suggestions and generate more targeted, contextually relevant recommendations.
+
+This feature is controlled by a boolean configuration parameter: `pr_code_suggestions.wiki_page_accepted_suggestions` (default is true).
+
+!!! note "Wiki must be enabled"
+    While the aggregation process is automatic, GitHub repositories require a one-time manual wiki setup.
+
+    To initialize the wiki: navigate to `Wiki`, select `Create the first page`, then click `Save page`. 
+
+    ![pr_agent_accepted_suggestions_create_first_page.png](https://qodo.ai/images/pr_agent/pr_agent_accepted_suggestions_create_first_page.png){width=768}
+
+    Once a wiki repo is created, the tool will automatically use this wiki for tracking suggestions.
+
+!!! note "Why a wiki page?"
+    Your code belongs to you, and we respect your privacy. Hence, we won't store any code suggestions in an external database. 
+
+    Instead, we leverage a dedicated private page, within your repository wiki, to track suggestions. This approach offers convenient secure suggestion tracking while avoiding pull requests or any noise to the main repository.
 
 ## Usage Tips
 
@@ -272,6 +299,10 @@ Using a combination of both can help the AI model to provide relevant and tailor
         <td><b>enable_chat_text</b></td>
         <td>If set to true, the tool will display a reference to the PR chat in the comment. Default is true.</td>
       </tr>
+      <tr>
+        <td><b>wiki_page_accepted_suggestions</b></td>
+        <td>If set to true, the tool will automatically track accepted suggestions in a dedicated wiki page called `.pr_agent_accepted_suggestions`. Default is true.</td>
+      </tr>
     </table>
 
 ??? example "Params for number of suggestions and AI calls"

pr_agent/settings/configuration.toml

@@ -135,8 +135,9 @@ demand_code_suggestions_self_review=false # add a checkbox for the author to sel
 code_suggestions_self_review_text= "**Author self-review**: I have reviewed the PR code suggestions, and addressed the relevant ones."
 approve_pr_on_self_review=false # Pro feature. if true, the PR will be auto-approved after the author clicks on the self-review checkbox
 fold_suggestions_on_self_review=true # Pro feature. if true, the code suggestions will be folded after the author clicks on the self-review checkbox
-# Suggestion impact
+# Suggestion impact 💎
 publish_post_process_suggestion_impact=true
+wiki_page_accepted_suggestions=true
 
 [pr_custom_prompt] # /custom_prompt #
 prompt = """\

pr_agent/tools/pr_help_message.py

@@ -138,7 +138,7 @@ async def run(self):
                     for section in relevant_sections:
                         file = section.get('file_name').strip().removesuffix('.md')
                         if str(section['relevant_section_header_string']).strip():
-                            markdown_header = section['relevant_section_header_string'].strip().strip('#').strip().lower().replace(' ', '-')
+                            markdown_header = section['relevant_section_header_string'].strip().strip('#').strip().lower().replace(' ', '-').replace("'", '').replace('(', '').replace(')', '').replace(',', '').replace('.', '').replace('?', '').replace('!', '')
                             answer_str += f"> - {base_path}{file}#{markdown_header}\n"
                         else:
                             answer_str += f"> - {base_path}{file}\n"