From ac70d5361aa8be2007b33d2ea2acf047607e868f Mon Sep 17 00:00:00 2001 From: Your Name Date: Sat, 14 Sep 2024 11:43:42 -0400 Subject: [PATCH] [DOCS] --- docs/applications/customer_support.md | 42 -- docs/applications/enterprise.md | 0 docs/applications/marketing_agencies.md | 64 --- docs/design.md | 152 ------- docs/examples/count-tokens.md | 29 -- docs/examples/index.md | 3 - docs/examples/load-and-query-pinecone.md | 49 --- docs/examples/load-query-and-chat-marqo.md | 51 --- docs/examples/query-webpage.md | 23 -- .../store-conversation-memory-in-dynamodb.md | 47 --- docs/examples/talk-to-a-pdf.md | 37 -- docs/examples/talk-to-a-webpage.md | 50 --- docs/examples/talk-to-redshift.md | 46 --- docs/examples/using-text-generation-web-ui.md | 97 ----- docs/swarms/index.md | 378 +++++++++--------- 15 files changed, 199 insertions(+), 869 deletions(-) delete mode 100644 docs/applications/customer_support.md delete mode 100644 docs/applications/enterprise.md delete mode 100644 docs/applications/marketing_agencies.md delete mode 100644 docs/design.md delete mode 100644 docs/examples/count-tokens.md delete mode 100644 docs/examples/index.md delete mode 100644 docs/examples/load-and-query-pinecone.md delete mode 100644 docs/examples/load-query-and-chat-marqo.md delete mode 100644 docs/examples/query-webpage.md delete mode 100644 docs/examples/store-conversation-memory-in-dynamodb.md delete mode 100644 docs/examples/talk-to-a-pdf.md delete mode 100644 docs/examples/talk-to-a-webpage.md delete mode 100644 docs/examples/talk-to-redshift.md delete mode 100644 docs/examples/using-text-generation-web-ui.md diff --git a/docs/applications/customer_support.md b/docs/applications/customer_support.md deleted file mode 100644 index a5a62f7..0000000 --- a/docs/applications/customer_support.md +++ /dev/null @@ -1,42 +0,0 @@ -## **Applications of Zeta: Revolutionizing Customer Support** - ---- - -**Introduction**: -In today's fast-paced digital world, responsive and efficient customer support is a linchpin for business success. The introduction of AI-driven zeta in the customer support domain can transform the way businesses interact with and assist their customers. By leveraging the combined power of multiple AI agents working in concert, businesses can achieve unprecedented levels of efficiency, customer satisfaction, and operational cost savings. - ---- - -### **The Benefits of Using Zeta for Customer Support:** - -1. **24/7 Availability**: Zeta never sleep. Customers receive instantaneous support at any hour, ensuring constant satisfaction and loyalty. - -2. **Infinite Scalability**: Whether it's ten inquiries or ten thousand, zeta can handle fluctuating volumes with ease, eliminating the need for vast human teams and minimizing response times. - -3. **Adaptive Intelligence**: Zeta learn collectively, meaning that a solution found for one customer can be instantly applied to benefit all. This leads to constantly improving support experiences, evolving with every interaction. - ---- - -### **Features - Reinventing Customer Support**: - -- **AI Inbox Monitor**: Continuously scans email inboxes, identifying and categorizing support requests for swift responses. - -- **Intelligent Debugging**: Proactively helps customers by diagnosing and troubleshooting underlying issues. - -- **Automated Refunds & Coupons**: Seamless integration with payment systems like Stripe allows for instant issuance of refunds or coupons if a problem remains unresolved. - -- **Full System Integration**: Holistically connects with CRM, email systems, and payment portals, ensuring a cohesive and unified support experience. - -- **Conversational Excellence**: With advanced LLMs (Language Model Transformers), the swarm agents can engage in natural, human-like conversations, enhancing customer comfort and trust. - -- **Rule-based Operation**: By working with rule engines, zeta ensure that all actions adhere to company guidelines, ensuring consistent, error-free support. - -- **Turing Test Ready**: Crafted to meet and exceed the Turing Test standards, ensuring that every customer interaction feels genuine and personal. - ---- - -**Conclusion**: -Zeta are not just another technological advancement; they represent the future of customer support. Their ability to provide round-the-clock, scalable, and continuously improving support can redefine customer experience standards. By adopting zeta, businesses can stay ahead of the curve, ensuring unparalleled customer loyalty and satisfaction. - -**Experience the future of customer support. Dive into the swarm revolution.** - diff --git a/docs/applications/enterprise.md b/docs/applications/enterprise.md deleted file mode 100644 index e69de29..0000000 diff --git a/docs/applications/marketing_agencies.md b/docs/applications/marketing_agencies.md deleted file mode 100644 index f38614b..0000000 --- a/docs/applications/marketing_agencies.md +++ /dev/null @@ -1,64 +0,0 @@ -## **Zeta in Marketing Agencies: A New Era of Automated Media Strategy** - ---- - -### **Introduction**: -- Brief background on marketing agencies and their role in driving brand narratives and sales. -- Current challenges and pain points faced in media planning, placements, and budgeting. -- Introduction to the transformative potential of zeta in reshaping the marketing industry. - ---- - -### **1. Fundamental Problem: Media Plan Creation**: - - **Definition**: The challenge of creating an effective media plan that resonates with a target audience and aligns with brand objectives. - - - **Traditional Solutions and Their Shortcomings**: Manual brainstorming sessions, over-reliance on past strategies, and long turnaround times leading to inefficiency. - - - **How Zeta Address This Problem**: - - **Benefit 1**: Automated Media Plan Generation – Zeta ingest branding summaries, objectives, and marketing strategies to generate media plans, eliminating guesswork and human error. - - **Real-world Application of Zeta**: The automation of media plans based on client briefs, including platform selections, audience targeting, and creative versions. - ---- - -### **2. Fundamental Problem: Media Placements**: - - **Definition**: The tedious task of determining where ads will be placed, considering demographics, platform specifics, and more. - - - **Traditional Solutions and Their Shortcomings**: Manual placement leading to possible misalignment with target audiences and brand objectives. - - - **How Zeta Address This Problem**: - - **Benefit 2**: Precision Media Placements – Zeta analyze audience data and demographics to suggest the best placements, optimizing for conversions and brand reach. - - **Real-world Application of Zeta**: Automated selection of ad placements across platforms like Facebook, Google, and DSPs based on media plans. - ---- - -### **3. Fundamental Problem: Budgeting**: - - **Definition**: Efficiently allocating and managing advertising budgets across multiple campaigns, platforms, and timeframes. - - - **Traditional Solutions and Their Shortcomings**: Manual budgeting using tools like Excel, prone to errors, and inefficient shifts in allocations. - - - **How Zeta Address This Problem**: - - **Benefit 3**: Intelligent Media Budgeting – Zeta enable dynamic budget allocation based on performance analytics, maximizing ROI. - - **Real-world Application of Zeta**: Real-time adjustments in budget allocations based on campaign performance, eliminating long waiting periods and manual recalculations. - ---- - -### **Features**: -1. Automated Media Plan Generator: Input your objectives and receive a comprehensive media plan. -2. Precision Media Placement Tool: Ensure your ads appear in the right places to the right people. -3. Dynamic Budget Allocation: Maximize ROI with real-time budget adjustments. -4. Integration with Common Tools: Seamless integration with tools like Excel and APIs for exporting placements. -5. Conversational Platform: A suite of tools built for modern marketing agencies, bringing all tasks under one umbrella. - ---- - -### **Testimonials**: -- "Zeta have completely revolutionized our media planning process. What used to take weeks now takes mere hours." - *Senior Media Strategist, Top-tier Marketing Agency* -- "The precision with which we can place ads now is unprecedented. It's like having a crystal ball for marketing!" - *Campaign Manager, Global Advertising Firm* - ---- - -### **Conclusion**: -- Reiterate the immense potential of zeta in revolutionizing media planning, placements, and budgeting for marketing agencies. -- Call to action: For marketing agencies looking to step into the future and leave manual inefficiencies behind, zeta are the answer. - ---- \ No newline at end of file diff --git a/docs/design.md b/docs/design.md deleted file mode 100644 index c5be4b8..0000000 --- a/docs/design.md +++ /dev/null @@ -1,152 +0,0 @@ -# Design Philosophy Document for Zeta - -## Usable - -### Objective - -Our goal is to ensure that Zeta is intuitive and easy to use for all users, regardless of their level of technical expertise. This includes the developers who implement Zeta in their applications, as well as end users who interact with the implemented systems. - -### Tactics - -- Clear and Comprehensive Documentation: We will provide well-written and easily accessible documentation that guides users through using and understanding Zeta. -- User-Friendly APIs: We'll design clean and self-explanatory APIs that help developers to understand their purpose quickly. -- Prompt and Effective Support: We will ensure that support is readily available to assist users when they encounter problems or need help with Zeta. - -## Reliable - -### Objective - -Zeta should be dependable and trustworthy. Users should be able to count on Zeta to perform consistently and without error or failure. - -### Tactics - -- Robust Error Handling: We will focus on error prevention, detection, and recovery to minimize failures in Zeta. -- Comprehensive Testing: We will apply various testing methodologies such as unit testing, integration testing, and stress testing to validate the reliability of our software. -- Continuous Integration/Continuous Delivery (CI/CD): We will use CI/CD pipelines to ensure that all changes are tested and validated before they're merged into the main branch. - -## Fast - -### Objective - -Zeta should offer high performance and rapid response times. The system should be able to handle requests and tasks swiftly. - -### Tactics - -- Efficient Algorithms: We will focus on optimizing our algorithms and data structures to ensure they run as quickly as possible. -- Caching: Where appropriate, we will use caching techniques to speed up response times. -- Profiling and Performance Monitoring: We will regularly analyze the performance of Zeta to identify bottlenecks and opportunities for improvement. - -## Scalable - -### Objective - -Zeta should be able to grow in capacity and complexity without compromising performance or reliability. It should be able to handle increased workloads gracefully. - -### Tactics - -- Modular Architecture: We will design Zeta using a modular architecture that allows for easy scaling and modification. -- Load Balancing: We will distribute tasks evenly across available resources to prevent overload and maximize throughput. -- Horizontal and Vertical Scaling: We will design Zeta to be capable of both horizontal (adding more machines) and vertical (adding more power to an existing machine) scaling. - -### Philosophy - -Zeta is designed with a philosophy of simplicity and reliability. We believe that software should be a tool that empowers users, not a hurdle that they need to overcome. Therefore, our focus is on usability, reliability, speed, and scalability. We want our users to find Zeta intuitive and dependable, fast and adaptable to their needs. This philosophy guides all of our design and development decisions. - -# Swarm Architecture Design Document - -## Overview - -The goal of the Swarm Architecture is to provide a flexible and scalable system to build swarm intelligence models that can solve complex problems. This document details the proposed design to create a plug-and-play system, which makes it easy to create custom zeta, and provides pre-configured zeta with multi-modal agents. - -## Design Principles - -- **Modularity**: The system will be built in a modular fashion, allowing various components to be easily swapped or upgraded. -- **Interoperability**: Different swarm classes and components should be able to work together seamlessly. -- **Scalability**: The design should support the growth of the system by adding more components or zeta. -- **Ease of Use**: Users should be able to easily create their own zeta or use pre-configured ones with minimal configuration. - -## Design Components - -### AbstractSwarm - -The AbstractSwarm is an abstract base class which defines the basic structure of a swarm and the methods that need to be implemented. Any new swarm should inherit from this class and implement the required methods. - -### Swarm Classes - -Various Swarm classes can be implemented inheriting from the AbstractSwarm class. Each swarm class should implement the required methods for initializing the components, worker nodes, and boss node, and running the swarm. - -Pre-configured swarm classes with multi-modal agents can be provided for ease of use. These classes come with a default configuration of tools and agents, which can be used out of the box. - -### Tools and Agents - -Tools and agents are the components that provide the actual functionality to the zeta. They can be language models, AI assistants, vector stores, or any other components that can help in problem solving. - -To make the system plug-and-play, a standard interface should be defined for these components. Any new tool or agent should implement this interface, so that it can be easily plugged into the system. - -## Usage - -Users can either use pre-configured zeta or create their own custom zeta. - -To use a pre-configured swarm, they can simply instantiate the corresponding swarm class and call the run method with the required objective. - -To create a custom swarm, they need to: - -1. Define a new swarm class inheriting from AbstractSwarm. -2. Implement the required methods for the new swarm class. -3. Instantiate the swarm class and call the run method. - -### Example - -```python -# Using pre-configured swarm -swarm = PreConfiguredSwarm(openai_api_key) -swarm.run_zeta(objective) - -# Creating custom swarm -class CustomSwarm(AbstractSwarm): - # Implement required methods - -swarm = CustomSwarm(openai_api_key) -swarm.run_zeta(objective) -``` - -## Conclusion - -This Swarm Architecture design provides a scalable and flexible system for building swarm intelligence models. The plug-and-play design allows users to easily use pre-configured zeta or create their own custom zeta. - - -# Swarming Architectures -Sure, below are five different swarm architectures with their base requirements and an abstract class that processes these components: - -1. **Hierarchical Swarm**: This architecture is characterized by a boss/worker relationship. The boss node takes high-level decisions and delegates tasks to the worker nodes. The worker nodes perform tasks and report back to the boss node. - - Requirements: Boss node (can be a large language model), worker nodes (can be smaller language models), and a task queue for task management. - -2. **Homogeneous Swarm**: In this architecture, all nodes in the swarm are identical and contribute equally to problem-solving. Each node has the same capabilities. - - Requirements: Homogeneous nodes (can be language models of the same size), communication protocol for nodes to share information. - -3. **Heterogeneous Swarm**: This architecture contains different types of nodes, each with its specific capabilities. This diversity can lead to more robust problem-solving. - - Requirements: Different types of nodes (can be different types and sizes of language models), a communication protocol, and a mechanism to delegate tasks based on node capabilities. - -4. **Competitive Swarm**: In this architecture, nodes compete with each other to find the best solution. The system may use a selection process to choose the best solutions. - - Requirements: Nodes (can be language models), a scoring mechanism to evaluate node performance, a selection mechanism. - -5. **Cooperative Swarm**: In this architecture, nodes work together and share information to find solutions. The focus is on cooperation rather than competition. - - Requirements: Nodes (can be language models), a communication protocol, a consensus mechanism to agree on solutions. - - -6. **Grid-based Swarm**: This architecture positions agents on a grid, where they can only interact with their neighbors. This is useful for simulations, especially in fields like ecology or epidemiology. - - Requirements: Agents (can be language models), a grid structure, and a neighborhood definition (i.e., how to identify neighboring agents). - -7. **Particle Swarm Optimization (PSO) Swarm**: In this architecture, each agent represents a potential solution to an optimization problem. Agents move in the solution space based on their own and their neighbors' past performance. PSO is especially useful for continuous numerical optimization problems. - - Requirements: Agents (each representing a solution), a definition of the solution space, an evaluation function to rate the solutions, a mechanism to adjust agent positions based on performance. - -8. **Ant Colony Optimization (ACO) Swarm**: Inspired by ant behavior, this architecture has agents leave a pheromone trail that other agents follow, reinforcing the best paths. It's useful for problems like the traveling salesperson problem. - - Requirements: Agents (can be language models), a representation of the problem space, a pheromone updating mechanism. - -9. **Genetic Algorithm (GA) Swarm**: In this architecture, agents represent potential solutions to a problem. They can 'breed' to create new solutions and can undergo 'mutations'. GA zeta are good for search and optimization problems. - - Requirements: Agents (each representing a potential solution), a fitness function to evaluate solutions, a crossover mechanism to breed solutions, and a mutation mechanism. - -10. **Stigmergy-based Swarm**: In this architecture, agents communicate indirectly by modifying the environment, and other agents react to such modifications. It's a decentralized method of coordinating tasks. - - Requirements: Agents (can be language models), an environment that agents can modify, a mechanism for agents to perceive environment changes. - -These architectures all have unique features and requirements, but they share the need for agents (often implemented as language models) and a mechanism for agents to communicate or interact, whether it's directly through messages, indirectly through the environment, or implicitly through a shared solution space. Some also require specific data structures, like a grid or problem space, and specific algorithms, like for evaluating solutions or updating agent positions. diff --git a/docs/examples/count-tokens.md b/docs/examples/count-tokens.md deleted file mode 100644 index 2ad237a..0000000 --- a/docs/examples/count-tokens.md +++ /dev/null @@ -1,29 +0,0 @@ -To count tokens you can use Zeta events and the `TokenCounter` util: - -```python -from zeta import utils -from zeta.events import ( - StartPromptEvent, FinishPromptEvent, -) -from zeta.structures import Agent - - -token_counter = utils.TokenCounter() - -agent = Agent( - event_listeners={ - StartPromptEvent: [ - lambda e: token_counter.add_tokens(e.token_count) - ], - FinishPromptEvent: [ - lambda e: token_counter.add_tokens(e.token_count) - ], - } -) - -agent.run("tell me about large language models") -agent.run("tell me about GPT") - -print(f"total tokens: {token_counter.tokens}") - -``` \ No newline at end of file diff --git a/docs/examples/index.md b/docs/examples/index.md deleted file mode 100644 index 4ed46a1..0000000 --- a/docs/examples/index.md +++ /dev/null @@ -1,3 +0,0 @@ -This section of the documentation is dedicated to examples highlighting Zeta functionality. - -We try to keep all examples up to date, but if you think there is a bug please [submit a pull request](https://github.com/kyegomez/zeta-docs/tree/main/docs/examples). We are also more than happy to include new examples :) \ No newline at end of file diff --git a/docs/examples/load-and-query-pinecone.md b/docs/examples/load-and-query-pinecone.md deleted file mode 100644 index 18f7cd7..0000000 --- a/docs/examples/load-and-query-pinecone.md +++ /dev/null @@ -1,49 +0,0 @@ -```python -import hashlib -import json -from urllib.request import urlopen -from decouple import config -from zeta.drivers import PineconeVectorStoreDriver - - -def load_data(driver: PineconeVectorStoreDriver) -> None: - response = urlopen( - "https://raw.githubusercontent.com/wedeploy-examples/" - "supermarket-web-example/master/products.json" - ) - - for product in json.loads(response.read()): - driver.upsert_text( - product["description"], - vector_id=hashlib.md5(product["title"].encode()).hexdigest(), - meta={ - "title": product["title"], - "description": product["description"], - "type": product["type"], - "price": product["price"], - "rating": product["rating"] - }, - namespace="supermarket-products" - ) - - -vector_driver = PineconeVectorStoreDriver( - api_key=config("PINECONE_API_KEY"), - environment=config("PINECONE_ENVIRONMENT"), - index_name=config("PINECONE_INDEX_NAME") -) - -load_data(vector_driver) - -result = vector_driver.query( - "fruit", - count=3, - filter={ - "price": {"$lte": 15}, - "rating": {"$gte": 4} - }, - namespace="supermarket-products" -) - -print(result) -``` \ No newline at end of file diff --git a/docs/examples/load-query-and-chat-marqo.md b/docs/examples/load-query-and-chat-marqo.md deleted file mode 100644 index edaa507..0000000 --- a/docs/examples/load-query-and-chat-marqo.md +++ /dev/null @@ -1,51 +0,0 @@ -```python -from zeta import utils -from zeta.drivers import MarqoVectorStoreDriver -from zeta.engines import VectorQueryEngine -from zeta.loaders import WebLoader -from zeta.structures import Agent -from zeta.tools import KnowledgeBaseClient -import openai -from marqo import Client - -# Set the OpenAI API key -openai.api_key_path = "../openai_api_key.txt" - -# Define the namespace -namespace = "kyegomez" - -# Initialize the vector store driver -vector_store = MarqoVectorStoreDriver( - api_key=openai.api_key_path, - url="http://localhost:8882", - index="chat2", - mq=Client(api_key="foobar", url="http://localhost:8882") -) - -# Get a list of all indexes -#indexes = vector_store.get_indexes() -#print(indexes) - -# Initialize the query engine -query_engine = VectorQueryEngine(vector_store_driver=vector_store) - -# Initialize the knowledge base tool -kb_tool = KnowledgeBaseClient( - description="Contains information about the Zeta Framework from www.zeta.ai", - query_engine=query_engine, - namespace=namespace -) - -# Load artifacts from the web -artifacts = WebLoader(max_tokens=200).load("https://www.zeta.ai") - -# Upsert the artifacts into the vector store -vector_store.upsert_text_artifacts({namespace: artifacts,}) - -# Initialize the agent -agent = Agent(tools=[kb_tool]) - -# Start the chat -utils.Chat(agent).start() - -``` \ No newline at end of file diff --git a/docs/examples/query-webpage.md b/docs/examples/query-webpage.md deleted file mode 100644 index 0171f02..0000000 --- a/docs/examples/query-webpage.md +++ /dev/null @@ -1,23 +0,0 @@ -```python -from zeta.artifacts import BaseArtifact -from zeta.drivers import LocalVectorStoreDriver -from zeta.loaders import WebLoader - - -vector_store = LocalVectorStoreDriver() - -[ - vector_store.upsert_text_artifact(a, namespace="zeta") - for a in WebLoader(max_tokens=100).load("https://www.zeta.ai") -] - -results = vector_store.query( - "creativity", - count=3, - namespace="zeta" -) - -values = [BaseArtifact.from_json(r.meta["artifact"]).value for r in results] - -print("\n\n".join(values)) -``` \ No newline at end of file diff --git a/docs/examples/store-conversation-memory-in-dynamodb.md b/docs/examples/store-conversation-memory-in-dynamodb.md deleted file mode 100644 index bb3be37..0000000 --- a/docs/examples/store-conversation-memory-in-dynamodb.md +++ /dev/null @@ -1,47 +0,0 @@ -To store your conversation on DynamoDB you can use DynamoDbConversationMemoryDriver. -```python -from zeta.memory.structure import ConversationMemory -from zeta.memory.structure import ConversationMemoryElement, Turn, Message -from zeta.drivers import DynamoDbConversationMemoryDriver - -# Instantiate DynamoDbConversationMemoryDriver -dynamo_driver = DynamoDbConversationMemoryDriver( - aws_region="us-east-1", - table_name="conversations", - partition_key="convo_id", - value_attribute_key="convo_data", - partition_key_value="convo1" -) - -# Create a ConversationMemory structure -conv_mem = ConversationMemory( - turns=[ - Turn( - turn_index=0, - system=Message("Hello"), - user=Message("Hi") - ), - Turn( - turn_index=1, - system=Message("How can I assist you today?"), - user=Message("I need some information") - ) - ], - latest_turn=Turn( - turn_index=2, - system=Message("Sure, what information do you need?"), - user=None # user has not yet responded - ), - driver=dynamo_driver # set the driver -) - -# Store the conversation in DynamoDB -dynamo_driver.store(conv_mem) - -# Load the conversation from DynamoDB -loaded_conv_mem = dynamo_driver.load() - -# Display the loaded conversation -print(loaded_conv_mem.to_json()) - -``` \ No newline at end of file diff --git a/docs/examples/talk-to-a-pdf.md b/docs/examples/talk-to-a-pdf.md deleted file mode 100644 index bf74062..0000000 --- a/docs/examples/talk-to-a-pdf.md +++ /dev/null @@ -1,37 +0,0 @@ -This example demonstrates how to vectorize a PDF of the [Attention Is All You Need](https://arxiv.org/pdf/1706.03762.pdf) paper and setup a Zeta agent with rules and the `KnowledgeBase` tool to use it during conversations. - -```python -import io -import requests -from zeta.engines import VectorQueryEngine -from zeta.loaders import PdfLoader -from zeta.structures import Agent -from zeta.tools import KnowledgeBaseClient -from zeta.utils import Chat - -namespace = "attention" - -response = requests.get("https://arxiv.org/pdf/1706.03762.pdf") -engine = VectorQueryEngine() - -engine.vector_store_driver.upsert_text_artifacts( - { - namespace: PdfLoader().load( - io.BytesIO(response.content) - ) - } -) - -kb_client = KnowledgeBaseClient( - description="Contains information about the Attention Is All You Need paper. " - "Use it to answer any related questions.", - query_engine=engine, - namespace=namespace -) - -agent = Agent( - tools=[kb_client] -) - -Chat(agent).start() -``` \ No newline at end of file diff --git a/docs/examples/talk-to-a-webpage.md b/docs/examples/talk-to-a-webpage.md deleted file mode 100644 index 229531a..0000000 --- a/docs/examples/talk-to-a-webpage.md +++ /dev/null @@ -1,50 +0,0 @@ -This example demonstrates how to vectorize a webpage and setup a Zeta agent with rules and the `KnowledgeBase` tool to use it during conversations. - -```python -from zeta.engines import VectorQueryEngine -from zeta.loaders import WebLoader -from zeta.rules import Ruleset, Rule -from zeta.structures import Agent -from zeta.tools import KnowledgeBaseClient -from zeta.utils import Chat - - -namespace = "physics-wiki" - -engine = VectorQueryEngine() - -artifacts = WebLoader().load( - "https://en.wikipedia.org/wiki/Physics" -) - -engine.vector_store_driver.upsert_text_artifacts( - {namespace: artifacts} -) - - -kb_client = KnowledgeBaseClient( - description="Contains information about physics. " - "Use it to answer any physics-related questions.", - query_engine=engine, - namespace=namespace -) - -agent = Agent( - rulesets=[ - Ruleset( - name="Physics Tutor", - rules=[ - Rule( - "Always introduce yourself as a physics tutor" - ), - Rule( - "Be truthful. Only discuss physics." - ) - ] - ) - ], - tools=[kb_client] -) - -Chat(agent).start() -``` \ No newline at end of file diff --git a/docs/examples/talk-to-redshift.md b/docs/examples/talk-to-redshift.md deleted file mode 100644 index fc4fe4d..0000000 --- a/docs/examples/talk-to-redshift.md +++ /dev/null @@ -1,46 +0,0 @@ -This example demonstrates how to build an agent that can dynamically query Amazon Redshift Serverless tables and store its contents on the local hard drive. - -Let's build a support agent that uses GPT-4: - -```python -import boto3 -from zeta.drivers import AmazonRedshiftSqlDriver, OpenAiPromptDriver -from zeta.loaders import SqlLoader -from zeta.rules import Ruleset, Rule -from zeta.structures import Agent -from zeta.tools import SqlClient, FileManager -from zeta.utils import Chat - -session = boto3.Session(region_name="REGION_NAME") - -sql_loader = SqlLoader( - sql_driver=AmazonRedshiftSqlDriver( - database="DATABASE", - session=session, - workgroup_name="WORKGROUP_NAME" - ) -) - -sql_tool = SqlClient( - sql_loader=sql_loader, - table_name="people", - table_description="contains information about tech industry professionals", - engine_name="redshift" -) - -agent = Agent( - tools=[sql_tool, FileManager())], - rulesets=[ - Ruleset( - name="HumansOrg Agent", - rules=[ - Rule("Act and introduce yourself as a HumansOrg, Inc. support agent"), - Rule("Your main objective is to help with finding information about people"), - Rule("Only use information about people from the sources available to you") - ] - ) - ] -) - -Chat(agent).start() -``` diff --git a/docs/examples/using-text-generation-web-ui.md b/docs/examples/using-text-generation-web-ui.md deleted file mode 100644 index ed74bbb..0000000 --- a/docs/examples/using-text-generation-web-ui.md +++ /dev/null @@ -1,97 +0,0 @@ -This example demonstrates how to build an agent that can integrate with [Text Generation Web UI](https://github.com/oobabooga/text-generation-webui). - -To be able to perform successful connection, run text gen with '--api' and if you running text gen not on the same host, add '--listen'. see more option [here](https://github.com/oobabooga/text-generation-webui) - -Check out the bare API usage [example](https://github.com/oobabooga/text-generation-webui/blob/main/api-examples/api-example.py). - -## Tokenizer - -To match the tokenizer used in the text gen, one can use [PreTrainedTokenizerFast](https://huggingface.co/docs/transformers/fast_tokenizers#loading-from-a-json-file) to load tokenizer from saved json setting file. - -Example: - -Let's say you using [TheBloke/WizardLM-13B-V1-1-SuperHOT-8K-GPTQ](https://huggingface.co/TheBloke/WizardLM-13B-V1-1-SuperHOT-8K-GPTQ/tree/main) in text gen, you can get hold of 'tokenizer.json' file that can be used to setup a corresponding tokenizer. - -## Code Snippets - -Code snippet using a pre defined 'preset'. - -'max_tokens' argument here need to be set with the same value as in the preset in text gen. - -```shell -from zeta.structures import Agent -from zeta.drivers import TextGenPromptDriver -from zeta.tokenizers import TextGenTokenizer -from transformers import PreTrainedTokenizerFast - -fast_tokenizer = PreTrainedTokenizerFast(tokenizer_file="tokenizer.json") - -prompt_driver = TextGenPromptDriver( - preset="zeta", - tokenizer=TextGenTokenizer(max_tokens=300, tokenizer=fast_tokenizer) -) - -agent = Agent( - prompt_driver=prompt_driver -) - -agent.run( - "tell me what Zeta is" -) -``` - -Code snippet example using params, if params and preset is defined, preset will be used. - -this params are overriding the current preset set in text gen, not all of them must be used. - -```shell -from zeta.structures import Agent -from zeta.drivers import TextGenPromptDriver -from zeta.tokenizers import TextGenTokenizer -from transformers import PreTrainedTokenizerFast - -params = { - 'max_new_tokens': 250, - 'do_sample': True, - 'temperature': 0.7, - 'top_p': 0.1, - 'typical_p': 1, - 'epsilon_cutoff': 0, # In units of 1e-4 - 'eta_cutoff': 0, # In units of 1e-4 - 'tfs': 1, - 'top_a': 0, - 'repetition_penalty': 1.18, - 'repetition_penalty_range': 0, - 'top_k': 40, - 'min_length': 0, - 'no_repeat_ngram_size': 0, - 'num_beams': 1, - 'penalty_alpha': 0, - 'length_penalty': 1, - 'early_stopping': False, - 'mirostat_mode': 0, - 'mirostat_tau': 5, - 'mirostat_eta': 0.1, - 'seed': 235245345, - 'add_bos_token': True, - 'truncation_length': 2048, - 'ban_eos_token': False, - 'skip_special_tokens': True, - 'stopping_strings': [] - } - -fast_tokenizer = PreTrainedTokenizerFast(tokenizer_file="tokenizer.json") - -prompt_driver = TextGenPromptDriver( - params=params, - tokenizer=TextGenTokenizer(max_tokens=params['max_new_tokens'], tokenizer=fast_tokenizer) -) - -agent = Agent( - prompt_driver=prompt_driver -) - -agent.run( - "tell me what Zeta is" -) -``` \ No newline at end of file diff --git a/docs/swarms/index.md b/docs/swarms/index.md index 683bb21..eac89ff 100644 --- a/docs/swarms/index.md +++ b/docs/swarms/index.md @@ -1,179 +1,199 @@ -# Swarms in Torch -A home for swarming algorithms like PSO, Ant Colony, Sakana, and more in PyTorch primitives through an emphasis on use-ability, reliability, and performance. - -## Installation - -You can install the package using pip - -```bash -pip3 install swarms-torch -``` - -# Usage -- We have just PSO now, but we're adding in ant colony and others! - -```python -from swarms_torch import ParticleSwarmOptimization - -#test -pso = ParticleSwarmOptimization(goal="Attention is all you need", n_particles=100) -pso.optimize(iterations=1000) -``` - -- Ant Colony Optimization -```python -from swarms_torch.ant_colony_swarm import AntColonyOptimization - -# Usage: -goal_string = "Hello ACO" -aco = AntColonyOptimization(goal_string, num_iterations=1000) -best_solution = aco.optimize() -print("Best Matched String:", best_solution) - -``` - -# Todo -Here are 10 swarming neural network algorithms, with brief overviews, algorithmic pseudocode, and potential use cases for each: - -1. **Particle Swarm Optimization (PSO)** - - **Overview**: Simulates the social behavior of birds flocking or fish schooling. It adjusts trajectories of individual agents (particles) based on their own and their neighbors' best known positions. - - **Pseudocode**: - ``` - Initialize each particle with random position and velocity - while not converged: - for each particle: - calculate fitness value - if fitness better than its best, update its best - end for - for each particle: - update velocity towards its best and global best - update position - end for - end while - ``` - - **Use Cases**: Function optimization, neural network training, feature selection. - -2. **Ant Colony Optimization (ACO)** - - **Overview**: Simulates the foraging behavior of ants to find paths through graphs. Uses pheromones to mark good paths, which evaporate over time. - - **Pseudocode**: - ``` - Initialize pheromones on paths - while not converged: - deploy ants to find paths based on pheromone strengths - update pheromone strengths based on quality of paths - evaporate some pheromone - end while - ``` - - **Use Cases**: Path finding, traveling salesman problem, network routing. - -3. **Bee Algorithm (BA)** - - **Overview**: Simulates the behavior of honey bees finding optimal nectar sources. - - **Pseudocode**: - ``` - Initialize scout bees randomly - while not converged: - deploy scout bees to search for nectar - recruit forager bees based on nectar quality - more foragers for better nectar sources - end while - ``` - - **Use Cases**: Job scheduling, function optimization, image processing. - -4. **Firefly Algorithm (FA)** - - **Overview**: Based on the flashing behavior of fireflies. Fireflies are attracted to each other depending on the brightness of their flashing. - - **Pseudocode**: - ``` - Initialize fireflies randomly - while not converged: - for each firefly: - move towards brighter fireflies - end for - end while - ``` - - **Use Cases**: Multi-modal optimization, feature selection, clustering. - -5. **Bat Algorithm (BA)** - - **Overview**: Inspired by the echolocation behavior of bats. Bats fly randomly and adjust positions based on emitted and returned echoes. - - **Pseudocode**: - ``` - Initialize bats with random positions and velocities - while not converged: - for each bat: - adjust velocity based on echolocation feedback - move bat - end for - end while - ``` - - **Use Cases**: Engineering design, tuning machine learning algorithms, scheduling. - -6. **Wolf Search Algorithm (WSA)** - - **Overview**: Models the hunting behavior of gray wolves. - - **Pseudocode**: - ``` - Initialize wolves - while not converged: - calculate fitness of all wolves - rank wolves: alpha, beta, delta, and omega - adjust position of omega wolves towards alpha, beta, and delta - end while - ``` - - **Use Cases**: Neural network training, function optimization, game AI. - -7. **Fish School Search (FSS)** - - **Overview**: Simulates the social behavior of fish schooling. - - **Pseudocode**: - ``` - Initialize fish randomly in search space - while not converged: - for each fish: - if neighbor has better food, move towards it - else explore randomly - end for - adjust school behavior based on total food - end while - ``` - - **Use Cases**: Load balancing, function optimization, robotics. - -8. **Cuckoo Search (CS)** - - **Overview**: Based on the reproduction strategy of cuckoos. They lay eggs in host bird nests and those nests with the best eggs (solutions) will carry on to the next generation. - - **Pseudocode**: - ``` - Initialize host nests with eggs (solutions) - while not converged: - lay new eggs by Levy flights - if new egg better than the worst in nest, replace it - discover a fraction of nests and lay new eggs - end while - ``` - - **Use Cases**: Engineering design optimization, image processing, numerical optimization. - -9. **Whale Optimization Algorithm (WOA)** - - **Overview**: Simulates the bubble-net hunting strategy of humpback whales. - - **Pseudocode**: - ``` - Initialize whales - while not converged: - for each whale: - encircle prey or search using bubble-net approach - end for - - - end while - ``` - - **Use Cases**: Structural optimization, neural network training, function optimization. - -10. **Grasshopper Optimization Algorithm (GOA)** - - **Overview**: Simulates the swarming behavior of grasshoppers towards food sources. - - **Pseudocode**: - ``` - Initialize grasshoppers - while not converged: - for each grasshopper: - adjust position towards other grasshoppers based on distance and food source - end for - end while - ``` - - **Use Cases**: Job scheduling, clustering, neural network training. - -# License -MIT +# Swarms Installation Guide + +
+

+ + + +

+
+ +You can install `swarms` with pip in a +[**Python>=3.10**](https://www.python.org/) environment. + +## Prerequisites + +Before you begin, ensure you have the following installed: + +- Python 3.10 or higher: [Download Python](https://www.python.org/) +- pip (specific version recommended): `pip >= 21.0` +- git (for cloning the repository): [Download Git](https://git-scm.com/) + +## Installation Options + +=== "pip (Recommended)" + + #### Headless Installation + + The headless installation of `swarms` is designed for environments where graphical user interfaces (GUI) are not needed, making it more lightweight and suitable for server-side applications. + + ```bash + pip3 install swarms-torch + ``` + +=== "Development Installation" + + === "Using virtualenv" + + 1. **Clone the repository and navigate to the root directory:** + + ```bash + git clone https://github.com/kyegomez/swarms-pytorch + cd swarms + ``` + + 2. **Setup Python environment and activate it:** + + ```bash + python3 -m venv venv + source venv/bin/activate + pip install --upgrade pip + ``` + + 3. **Install Swarms:** + + - Headless install: + + ```bash + pip install -e . + ``` + + - Desktop install: + + ```bash + pip install -e .[desktop] + ``` + + === "Using Anaconda" + + 1. **Create and activate an Anaconda environment:** + + ```bash + conda create -n swarms python=3.10 + conda activate swarms + ``` + + 2. **Clone the repository and navigate to the root directory:** + + ```bash + git clone https://github.com/kyegomez/swarms-pytorch + cd swarms + ``` + + 3. **Install Swarms:** + + - Headless install: + + ```bash + pip install -e . + ``` + + - Desktop install: + + ```bash + pip install -e .[desktop] + ``` + + === "Using Poetry" + + 1. **Clone the repository and navigate to the root directory:** + + ```bash + git clone https://github.com/kyegomez/swarms-pytorch + cd swarms + ``` + + 2. **Setup Python environment and activate it:** + + ```bash + poetry env use python3.10 + poetry shell + ``` + + 3. **Install Swarms:** + + - Headless install: + + ```bash + poetry install + ``` + + - Desktop install: + + ```bash + poetry install --extras "desktop" + ``` + +=== "CI/CD Pipelines" + + Integrating Swarms into your CI/CD pipeline ensures automated testing and deployment. + + #### Using GitHub Actions + + ```yaml + # .github/workflows/ci.yml + name: CI + + on: + push: + branches: [ main ] + pull_request: + branches: [ main ] + + jobs: + build: + + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v2 + - name: Set up Python + uses: actions/setup-python@v2 + with: + python-version: 3.10 + - name: Install dependencies + run: | + python -m venv venv + source venv/bin/activate + pip install --upgrade pip + pip install -e . + - name: Run tests + run: | + source venv/bin/activate + pytest + ``` + + #### Using Jenkins + + ```groovy + pipeline { + agent any + + stages { + stage('Clone repository') { + steps { + git 'https://github.com/kyegomez/swarms-pytorch' + } + } + stage('Setup Python') { + steps { + sh 'python3 -m venv venv' + sh 'source venv/bin/activate && pip install --upgrade pip' + } + } + stage('Install dependencies') { + steps { + sh 'source venv/bin/activate && pip install -e .' + } + } + stage('Run tests') { + steps { + sh 'source venv/bin/activate && pytest' + } + } + } + } + ```