Created an agent builder script, would love your feedback

[ben-AI-cybersec]

Hi all,
I was inspired by Dave's video and saw a few problems with his prompting I thought I could solve. After a few hours, I built this agent builder script, which works up to a point. It's 4:30am local time so I think I'm done for the evening, but if anyone likes the idea feel free to continue it. The output looks pretty decent to me, basically it creates new prompts and recursively uses the same script as a starting point.

[RealDanCarroll]

How's it working for you? You prompts look logical to me.

[cipherpodliq1]

Thanks for that! I will try expanding on it!

[daveshap]

Hi all, I was inspired by Dave's video and saw a few problems with his prompting I thought I could solve. After a few hours, I built this agent builder script, which works up to a point. It's 4:30am local time so I think I'm done for the evening, but if anyone likes the idea feel free to continue it. The output looks pretty decent to me, basically it creates new prompts and recursively uses the same script as a starting point.

This is a good start but we need a universal agent building function e.g. ONE FUNCTION TO RULE THEM ALL. We can't have a unique, bespoke function for every tier. If you can figure out how to make it more parameterized with the following, that would be good:

• Archetype/persona
• Role (RBAC)
• Tier (integer)
• Necessary files
• Instructions
• Tools required

[ethanmmay]

If the Role (RBAC) is eventually made into a parameter for a Parent/Universal Agent Building Function, maybe an Agent Authentication function would be possible to create.

That way, when agents are created, they're assigned an RBAC Role, and when they attempt to create another agent, the Authentication function is called and if their Role is valid for creating another agent, it's created.

This in-library function could be re-usable for other future implementations too. For example, when the agents check each others work. I believe this function would be instrumental in establishing the kinds of organization needed to experiment with different configurations of agents.

[RafaDallaCosta]

Remember that SOB is a group of agents.
I guess we need to create a SOB president, who will summon SOB members that answer to the whole SOB, so that to exclude any SOB member a voting is needed. Then the president creates a CEO, who creates department directors that creates executive agents.

[tsalagi]

Not trying to get out of hand here, but this seems strikingly like a consumer interaction with a company. I'm working in the Playground now and am working on something similar with creating domain experts (agents). The Assistant explained to me that although this is a visualization and not an independent instance, I would benefit from a standard outline in a separate text file for each domain agent. I'm at the part now where I'm dealing with what kind of check (feedback loop) I want to put forward to the Assistant.

Imagine the user as a client/customer. Integrating my project with the info above I see it like this: The user queries customer service (the user-facing text prompt/agent) this is an agent apart from other hierarchies). This agent may ask follow up questions, like Poe's Copilot, to refine the user request.

Customer service is now a mediator, forming a feedback loop with the SOB. The SOB manages task delegation to high tier agents. The sub agents fetch and return info to the SOB. The SOB passes the answer back to the customer service agent. It builds a mechanism for capturing user feedback, analyzing it, and feeding it back into the system for learning and improvement. This can be done within the same system that houses the AI or by using additional software or services for analytics and data processing.

My question based on how I'm outlining this: Do you build in checks at the Agent 1 tier? Is that something you would assign to every agent at every stage of the query and calls? I wouldn't think so. Generation 1 Agents would include a check on work. Maybe? Am I way off?

[RafaDallaCosta]

I think that this - David System Prompt writer
is a good start to what the instruction for a generic agent builder should look like.

[binderg]

This may be a step forward as a concept.it allows for showing who can access what thread (chat) and it may be easier to just create functions for the assistants to use because i find it hard to prompt them into making perfect api calls for creating openai bots and do it the way you want (with the metadata)

# (maybe put this data in the metadata key in the respective objects from openai instead of local objects)?

# schema
# id:{
#     persona:"",
#     role:"",
#     tier:0,
#     createdBy (agent id):"",
#     threadIds (to keep track of entered threads)):[] 
#   }

assistants = {}

# schema 
# id (thread id):{
#     createdBy (agent id):"",
#     accessLevel(Public, Private, Tiered):"",
#     allowedAgentIds:[],
# }


threads = {}

# Initial instructions
instructions = (
    "You are the Supreme Oversight Board (SOB) agent of the Hierarchical Autonomous Agent Swarm (HAAS) system. "
    "You are responsible for the creation, oversight, and termination of other agents within the HAAS. "
    "Your purpose is to ensure the reduction of suffering, increase prosperity, and enhance understanding "
    "in the universe. You command executive and subordinate agents, manage internal processes, access "
    "a knowledge base, issue operational directives, and communicate in conversational format."
)


def create_initial_agent_and_thread():
    assistant = client.beta.assistants.create(
        name="Supreme Oversight Board",
        instructions=instructions,
        model="gpt-4-1106-preview",
        tools=[{"type": "retrieval"}],
        file_ids=[]
    )
    create_assistant("Supreme Oversight Board", instructions, "gpt-4-1106-preview", [{"type": "retrieval"}],
                     persona="sob", role="Supreme Oversight Board", tier=1)

    thread = client.beta.threads.create(
        messages=[
            {
                "role": "user",
                "content": (
                    "Create a python script based on agentbuilder.py to create a set of 4 agents at the Executive level. "
                    "Update the instructions for each executive with details of their mission, goals and constraints. "
                    "In the user messages, those executives will then need to create their workers per the README file and the run's ultimate goal. "
                    )
            }
        ]
    )

    threads[thread.id] = {
        "createdById": assistant.id,
        "accessLevel": "Public",
        "allowedAgentIds": [],
    }

    run = client.beta.threads.runs.create(
        thread_id=thread.id,
        assistant_id=assistant.id,
        instructions="The ultimate goal is to create a software product to assist researchers."
    )
    return assistant, thread


def create_assistant(assistant_name, assistant_instructions, assistant_model, assistant_tools, assistant_file_ids, persona, role, tier):
    assistant = client.beta.assistants.create(
        name=assistant_name,
        instructions=assistant_instructions,
        model=assistant_model,
        tools=assistant_tools,
        file_ids=assistant_file_ids
    )

    assistants[assistant.id] = {
        "persona": persona,
        "role": role,
        "tier": tier,
        "threadIds": []
    }
    ```