Code capability enhancement & Bot crash fix

[Ninot1Quyi]

Last Modified Time: November 10, 2024, 5:53 PM

Latest changes are as follows:

---

1. Improvement Effects

   • Model: GPT-4o

   • Initial Command: !goal("Your goal is: use only "!newAction" instructions and rely only on code execution to obtain a diamond pickaxe. You must complete this task step by step and by yourself. And can't use another "!command". You should promptly check to see what you have.")

   • Effect: After testing, under the condition of relying solely on generated code, the bot can run stably for at least 30 minutes without crashing (I manually ended the process at 30 minutes), during which it executed over 130 validated code snippets.

   • Remaining Issues:

     1. If illegal commands are executed, such as "attacking a non-existent entity," the server may kick the bot out.
     2. A very small number of tasks may lead to no execution result being obtained, causing code crashes. It is suspected that there may be unconsidered exceptional situations when receiving task results.

   • WARNING: If you use the command above or set a goal that requires a long time to work, please pay attention to the execution status and token consumption, as the LLM may continuously generate code in certain situations. For example, when "an iron pickaxe is available and diamonds need to be mined," it might stand still using its code abilities to search for nearby diamond locations. Since diamonds are rare, it may fail to find them continuously, repeatedly improving the code and getting stuck, leading to substantial token consumption.Please test with caution, it cost me $60 to test with gpt-4o for 60min. But gpt-4o-mini is much cheaper and can be used to test this command

2. Added Features:
   2.1 During code generation, the top select_num relevant skillsDocs related to !newAction("task") will be selected and sent to the LLM in the prompt to help it focus better on the task. Currently, select_num is set to 5.
   2.2 Before running the code, use ESLint to perform syntax and exception checks on the generated code to detect issues in advance, check for undefined functions, and add exceptions to messages.
   2.3 During code execution, detailed error information will be included in messages.

3. Added Files:
   3.1 file path: ./bots/codeCheckTemplate.js
   A template used for performing checks before code execution. ESLint cannot be used for detection in the sandbox.

   3.2 file path: ./eslint.config.js
   Manages the ESLint rules for code syntax and exception detection.

4. Modified Code Content:

   4.1 package.json

   - Added: ESLint dependency.

   4.2 settings.js

   - Set: code_timeout_mins=3, ensuring timely code execution updates and preventing long blocks.

   4.3 coder.js

   - Added: checkCode function to pre-check for syntax and exceptions. First, it checks whether the functions used in the code exist. If they don't, it writes the illegal functions to the message, then proceeds with syntax and exception checks.

   - Modified: Modified the return value of stageCode function from return { main: mainFn }; to return { func: { main: mainFn }, src_check_copy: src_check_copy }; to ensure pre-execution exception detection.

   4.4 action_manager.js

   - Enhanced: catch (err) error detection to include detailed exception content and related code Docs in messages, improving the LLM's ability to fix code.

   4.5 index.js

   - Modified: docHelper and getSkillDocs return values to return the docArray of functions from the skill library for subsequent word embedding vector calculations.

   4.6 prompter.js

   - Added: this.skill_docs_embeddings = {}; to store the docArray word embedding vectors.

   - Added: Parallel initialization of this.skill_docs_embeddings in initExamples.

   - Added: getRelevantSkillDocs function to obtain select_num relevant doc texts based on input messages and select_num. If select_num >= 0, it is meaningful; otherwise, return all content sorted by relevance.

Note: This modification ensures code quality by making minimal changes only where necessary, while also clearing test outputs and comments. If further modifications are needed, please feel free to let me know.

[Ninot1Quyi]

Resolve merge conflicts with the latest code

New additions

1. Added the codeChackTemplate.js file under the bots directory for static syntax and exception detection.
2. Modified the return value of stageCode and the part of generateCodeLoop that runs the code to resolve merge conflict issues.
3. Added the ESLint configuration file eslint.config.js in the project root directory to manage code syntax and exception detection rules.

[JurassikLizard]

Can you try re-running this with a stupider model (not state-of-the-art lol). I'm curious to see if they benefit too, or just advanced ones.

[Ninot1Quyi]

Comparison Experiment on Low-Performance Models

1. Objective

The objective is set using the following command:
!goal("Your goal is: use only "!newAction" instructions and rely only on code execution to obtain a diamondpickaxe. You must complete this task step by step and by yourself. And can't use another "!command". You should promptly check to see what you have")

2. Model Selection

First, I tested the lowest-performance model, gpt-3.5-turbo, but it could not limit itself to using only !newAction and was unable to complete the task. Subsequently, I tested gpt-4o-mini.
All subsequent tests were conducted using gpt-4o-mini.

3. Experimental Process

• Created a world and made two copies to ensure the environment was the same.
• Used both modified and unmodified code to enter the same position, input the goal command, and let the bot execute the task.

4. Experimental Results

4.1 Original

Total run time: 16 minutes 41 seconds.

• 0 min: Start
• 3 min: First crash.
• 4 min: Second crash.
• 13 min: Acquired wooden pickaxe. [The bot was continually collecting wood and only completed the wooden pickaxe after multiple reminders to check existing items.]
• 16 min 24 s: Acquired a stone pickaxe.

4.2 Modified

I didn’t give any reminders to the bot while it was running.
Total run time: 16 minutes 22 seconds.

• 0 min: Start
• 4 min: Acquired a wooden pickaxe.
• 5 min 12 s: First crash.
• 8 min: Acquired a stone pickaxe.
• 11 min: Acquired iron ingots.
• 15 min: The content was obtained.
• 16 min 22s: Second crash.

4.3 Complete Comparison Video

Total duration: 16 minutes 41 seconds.
Watch the full comparison video here

[Ninot1Quyi]

Resolved merge conflict with Action Manager

[Ninot1Quyi]

There is a part that needs improvement

[Ninot1Quyi]

Improve the relevance of docs to !newAction("task")Fix Qwen api concurrency limit issue

[MaxRobinsonTheGreat]

Not sure about this. I like a few things, but not others.

Code linting looks very useful, I would add that by itself. Though codeCheckTemplate.js looks like it overwrites the recent changes to template.js

I don't like selecting only the most relevant skill docs. Wouldn't this strictly reduce performance? Yes it saves on context space, but means the LLM has no knowledge of most of the available functions. I am also skeptical that comparing the latest message to skill doc would reliably select the most relevant docs. Why not do this for commands too? Additionally, the logic for selecting relevant docs should be in its own separate file, like how we do for Examples.

I see your comparison make it looks like it has about the same performance. So what is the benefit?

[Ninot1Quyi]

Explanation and Feedback

1. Purpose of codeCheckTemplate.js

codeCheckTemplate.js haven‘t overwrites the recent changes to template.js.The codeCheckTemplate.js is designed solely for static syntax and error checking. I haven’t found a way to perform syntax checks within the sandbox environment. The final code still uses the modified template.js to run in the sandbox.

2. On Relevant Skill Document Selection

I thought we agree that relevance-based ranking can improve the agent’s performance. The function getRelevantSkillsDocs I designed accepts either no parameters or the parameter '-1', which will return all documents sorted by relevance. I think this approach is at least effective.

In the current implementation, I compare the task extracted from !newAction("task") with the annotations of skill functions (["function name" + "function description"]) to calculate similarity. Regarding the strategy of selecting relevant functions based only on the latest task and determining how many relevant functions to include, I think we should conduct detailed comparative experiments to explore this and determine the best performance.

3. Proposed Experiment Design

I plan to design a comparative experiment to evaluate the agent's performance in acquiring a diamond_pickaxe task using only code generation capabilities.

Initial Experiment Design:

• Model: gpt-4o-mini
• Task: !goal("Generate instructions using only !newAction to acquire a diamond_pickaxe")
• Methodology: Use different codebases to initiate the same task in three scenarios (forest, plains, village) at the same starting location. Each agent operates in an isolated game environment without interference from others.
• Comparison Codes:
  • Base-line: Uses all messages and the default annotation order of skills as defined in the documentation.
  • Code1: Uses all documents, sorted by relevance to the latest task.
  • Code2: Uses 5% of the documents, sorted by relevance to the latest task.
  • Code3: Uses 10% of the documents, sorted by relevance to the latest task.
  • Code4: Uses 25% of the documents, sorted by relevance to the latest task.
  • Code5: Uses 50% of the documents, sorted by relevance to the latest task.
  • Code6: Uses 75% of the documents, sorted by relevance to the latest task.
  • Code7: Uses 90% of the documents, sorted by relevance to the latest task.
• Results: Average the time taken across three scenarios to acquire key items for the diamond_pickaxe task. Plot time taken and crash frequency, with a maximum runtime of 30 minutes.

---

Note

1. I have an exam on the 24th, so I won’t have time to work on this until next week. I will start the experiment at some point next week. If anyone is interested in testing some of these setups, feel free to follow the design above and post your results here. Suggestions for improving this plan are also welcome.

2. If you conduct the tests, please copy and upload the save files for the three scenarios. I am also considering the idea of directly sending the historical information and all documents to the LLM and letting it return documents related to the current task. This might lead to unexpected results, but I will first carry out the experiment designed above.

[MaxRobinsonTheGreat]

Hi @Ninot1Quyi, I've reconsidered and I now agree with you! This will be a very valuable contribution, let's move forward with it. I realized a main benefit is that it would allow the list of skills to grow almost indefinitely. So we should probably do something similar with command docs too, but let's just do skills for now.

You don't need to test so much, and we can expect reduced performance as the cost of fixed context space usage. So long as it is easy to turn off.

A few requests:

• move all skill docs selection functionality from prompter.js and move to a new file like skill_library.js or something like that
• add a param to settings.js to control the number of skill docs to retrieve. If -1, just give the full unsorted skill docs as is currently done.
• why have a separate file codeCheckTemplate.js instead of just using template.js? Preferably we only use template.js
• code_chack_template is misspelled

Take your time, no rush whatsoever. Good luck with your exams!

[Ninot1Quyi]

@MaxRobinsonTheGreat
Hi Max,
I completed my exam last Sunday. Sorry for the delay in replying to your message. Over the next month, I’ll have three more exams, so progress on the revisions might be a bit slow.

I’m glad to hear that you appreciate my work, and I’ll make gradual changes to the code based on your suggestions. I need to find a better way to verify the generated code. Currently, I haven’t figured out how to enable ESLint to check the code in a sandbox environment, which is why I’m using codeCheckTemplate.js to import packages and test them in a real environment. This issue has been bothering me for a while, and I’ll do my best to resolve it since I also dislike the presence of codeCheckTemplate.js.

Thank you again for recognizing my efforts. Wishing you a happy life!

[Ninot1Quyi]

I'm back! Just resolving merge conflicts for now. Code migration and improvements are in progress.

[Ninot1Quyi]

The embedded concurrency limitation issue has been resolved in the latest qwen.js of the current PR, so qwen's own embedded model can be used in qwen.json instead of using openai's embedded model.
I'm currently working on an issue with codeCheckTemplate.js that code evaluation depends on. I'm looking for a way to remove the codeCheckTemplate.js file

[Ninot1Quyi]

@MaxRobinsonTheGreat
Hi Max, I’ve made the following updates as per your suggestions:

1. I separated the skill selection code into 'src/agent/library/skill_library.js'.
2. I added the 'relevant_docs_count' configuration interface in settings.js.
3. The template files are now implemented in a different way. I merged them and placed them in 'bots/codeTemplate.json' for the program to read directly. I searched around but couldn't find a way to implement static syntax checking within the SES sandbox because SES doesn’t allow code with static and dynamic import statements to run inside the sandbox.

Take a look and let me know if any further improvements are needed. Feel free to reach out if you need anything!

[MaxRobinsonTheGreat]

This looks much better and is very close to being done. Thanks for your work. A few small requests:

• separate execTemplate and lintTemplate again
• don't add skill docs in messages, they should always be in context and it will quickly fill up message history
• other little changes

[Ninot1Quyi]

1. Results:

I have made all the code modifications according to your suggestions. All the modified parts have been completed as required. However, I would like to discuss some details with you regarding whether we need to add appropriate error explanations when errors occur during code inspection and execution.

2. Discussion on "Should we add proper error explanation prompts when errors occur during code inspection and execution":

Here are some ideas I've come up with! If you think it's necessary, I can make the changes in the current PR. Alternatively, I can make these improvements to the coder in a new PR after merging. Feel free to share your thoughts by replying to this PR or shoot me an email at [[email protected]]! 😊

2.1 Option 1:

Simply changing the number of "error-related skill_doc" prompts in src/agent/coder.js and src/agent/action_manager.js to 1 will not effectively alleviate the message explosion.

2.2 Option 2:

Add another handling module to replace Assistant's erroneous answers in the coder message log with error-free code from subsequent Assistant responses, and erase the intermediate correction process. This should be a more reasonable approach.

2.3 Option 3:

Modify the history update process in coder.js, separating the debugging conversations between "starting to generate code" and "generated code running correctly," and only add the generated code to the history when there are no errors.

2.4 Option 4:

Extract the code generation process from the complete conversation history and treat it as a separate "small brain" that is solely responsible for code generation, debugging, and execution, much like the cerebellum in the human brain.

3 Summary:

The reason I suggest this is because I want to provide the LLM with correct usage instructions when errors occur, so that it can better correct mistakes, rather than getting stuck in its own delusions and unable to fix them.

Perhaps there is a better way to solve this issue, one that provides the LLM with proper help without causing a message explosion.