Merge pull request #895 from Codium-ai/tr/self_reflect

self-reflection documentation

README.md

@@ -40,6 +40,15 @@ CodiumAI PR-Agent aims to help efficiently review and handle pull requests, by p
 
 ## News and Updates
 
+### May 12, 2024
+Inspired by [AlphaCodium](https://github.com/Codium-ai/AlphaCodium) flow engineering scheme, PR-Agent now performs **self-reflection** on the code suggestions it provides,
+enabling to remove invalid suggestions, and score the valid ones. The suggestions will be presented sorted by their score, enabling to focus on the most important ones first.
+
+<kbd><img src="https://codium.ai/images/pr_agent/self_reflection1.png" width="512"></kbd>
+
+<kbd><img src="https://codium.ai/images/pr_agent/self_reflection2.png" width="512"></kbd>
+
+
 ### May 2, 2024
 Check out the new [PR-Agent Chrome Extension](https://chromewebstore.google.com/detail/pr-agent-chrome-extension/ephlnjeghhogofkifjloamocljapahnl) 🚀🚀🚀
 

docs/docs/tools/improve.md

@@ -86,6 +86,10 @@ Hence, the total number of suggestions is proportional to the number of chunks,
     <td><b>persistent_comment</b></td>
     <td>If set to true, the improve comment will be persistent, meaning that every new improve request will edit the previous one. Default is false.</td>
   </tr>
+  <tr>
+    <td><b>self_reflect_on_suggestions</b></td>
+    <td>If set to true, the improve tool will calculate an importance score for each suggestion [1-10], and sort the suggestion labels group based on this score. Default is true.</td>
+  </tr>
   <tr>
     <td><b>enable_help_text</b></td>
     <td>If set to true, the tool will display a help text in the comment. Default is true.</td>

pr_agent/tools/pr_code_suggestions.py

@@ -351,7 +351,8 @@ async def _prepare_prediction_extended(self, model: str) -> dict:
                                 if score > 0:
                                     data["code_suggestions"].append(prediction)
                                 else:
-                                    get_logger().info(f"Skipping suggestions from call {i + 1}, because score is {score}")
+                                    get_logger().info(f"Removing suggestions {i}, because score is {score}",
+                                                      artifact=prediction)
                             else:
                                 get_logger().error(f"Error getting PR diff, no code suggestions found in call {i + 1}")
                         except Exception as e: