Propose Milestones, Roadmap, and General Principles

[daveshap]

Moving to discussion as BOUNTY was premature.

For reference:

#66

[BrianP8701]

Continuing where I left off yesterday on memory management for the swarm. Let's keep it simple. LLAMA-Index vector database with metadata embedded rather than raw data. When querying the vector database (performing retrieval) transform the original query to better navigate the latent space.

Agent swarm all share access to the same knowledge base. If knowledge base reaches significant scale (probably won't be a problem for us) we'll want to break the knowledge base into multiple vector databases with metadata because k similarity search is O(N) with respect to size of vector database. Each individual agent in addition can have their own personal data with the naive OpenAI approach (although maybe we should avoid this because OpenAI's retrieval is VERY slow and not as good as LLAMA-Index - for now at least).

I've thought and experimented with this a lot and I briefly explain my reasoning here:
https://youtu.be/1qisOi_1zzY

[BrianP8701]

should i make a fork with a new memory folder or where should this go do you think? I reckon we can put the internal reddit/discord/twitter, RAG, comms etc all in a new memory folder

[daveshap]

There have been some discussion on document best practices #1 (reply in thread)

[BrianP8701]

Thank you thats rlly helpful!

[BrianP8701]

Sparse Priming Representation is exactly what I have on mind! And I just finished watching your video on 2x speed which was helpful: https://www.youtube.com/watch?v=YjdmYCd6y0M&ab_channel=DavidShapiro

I guess there's still I have to learn and catch up on, seems like all of my ideas have already been had and made by others. I'm "information foraging" as you say. I've found your channel and this repo to be a goldmine.

I'm diving into the thread you linked me to and I'm going to try to help! @daveshap

[ThomasSilloway]

Hi all

I started following the project this morning and have become very excited about it.

After reviewing many of the materials and discussion topics, I identified an one element that could be brainstormed more prior to creating a roadmap.

• What is a simple MVP use case for such a swarm of agents?

The best I've come up with so far is that it could be best suited for creating a software product that is continually updated and enhanced, but also something that is never truly complete. It should also be something that at its basic level is very easy for a human to create, test and evaluate, but provides endless possibilities for additional depth after prototyping.

Why software product?

Also, the swarm is by design suited towards continually improving itself and its creations.

Why something that is never truly complete?

This gives the swarm an opportunity to showcase its capabilities and quality of execution over time.

Yes, we could have it build a static project like a webpage for a local business. However, a more interesting application for this tech might be a product where the results are subjective instead. This allows endless possibilities for iteration based off of end user feedback.

Why something easy for human to create, test and iterate?

Well it's an MVP, we want the scope to be as small as possible, so let's create something that is a known quantity and build on top of that for future releases.

It also means that during our testing processes we can easily evaluate to work that the swarm is doing.

This rules out my first idea which was to use the swarm to do something novel like editing live streams into YouTube videos.  This would require audio processing, transcript creation, video analysis, and overall is a very complex project.  This might be a potential long term use case that could be a profitable project to create.

MVP Proposal:

Let's create a game development studio swarm. This checks all of the boxes.

 - Games can start as simple prototypes with small scopes

• Design of games can be complex, interesting with endless possibilities

• Games are often run as a live service with updates

 - Games are simple to review both in their code as well as the end product

• Reviews by end users are subjective, leading to endless possibilities of iteration

• Games take lots of money, time and energy to create while also containing some of the most difficult technical challenges in software. Much of which is well suited towards automation, leaving a very satisfying creative input process from humans.

• Games can be fun to test! This will help incentivize the team to provide feedback to the swarm on how it's doing and potentially include non-technical people in the testing and evaluation process.

• People can fork the project and change the type of game to their own interests. Instead of a fantasy DnD type setting, it could be a Star Trek theme, etc

MVP outline:

One of the simplest games to make and one that's well suited towards GPT based swarm is a text based RPG. This was in fact one of my first solo game development projects. The code was essentially a giant if else statement, so it's easy to understand and the scope can be very small.

I like Dave's idea of using a GitHub repo to manage communication between agents, so everything can live in one place.

I think it could make use of discussions to do game design, scope and project management discussions.

It could use the project management tab to create a roadmap for the development of the game, which can add observability into the swarm's process.

It can create PRs, automated builds and notify human testers when a build is ready for review.

I also like the idea that the agent is fully autonomous. Hopefully it can be given a guideline that after each completed prototype it should wait for user feedback.  This generic rule would allow the swarm to still continue running and evaluate when it's gotten enough feedback. 

A project like this for an MVP could have a few hard coded limitations to reduce costs and development time:

• Use a maximum of 3 room dungeon
• Limit discussions to X minutes (or seconds) in length before executing
• Wait for human feedback for no longer than 1 day
• Pause development and design while waiting for human feedback

[daveshap]

Hey man good to see you!

[BrianP8701]

I think creating a game is a great way to test the swarm!

[piotrjez]

I like Dave's idea of using a GitHub repo to manage communication between agents, so everything can live in one place.

What do you think about using Obsidian.md syntax for docummentation and prompts between agents? With use of tags and links between notes of prompts in Obsidian there's a possibility to visualize connections between data structure as graphs. Obsidian is called "second brain" tool and supports Markdown markup language. There's also a huge amount of plugins provided by open-source community.