Merge branch 'master' into webxr

semperai · Feb 4, 2025 · 30417c9 · 30417c9
2 parents 501b33d + a56150b
commit 30417c9
Show file tree

Hide file tree

Showing 27 changed files with 1,111 additions and 152 deletions.
diff --git a/README.md b/README.md
@@ -56,6 +56,7 @@ The various features of Amica mainly use and support the following technologies:
   - [Ollama](https://ollama.ai)
   - [KoboldCpp](https://github.com/LostRuins/koboldcpp)
   - [Oobabooga](https://github.com/oobabooga/text-generation-webui/wiki)
+  - [OpenRouter](https://openrouter.ai/) (access to multiple AI models)
 - Text-to-Speech
   - [Eleven Labs API](https://elevenlabs.io/)
   - [Speech T5](https://huggingface.co/microsoft/speecht5_tts)
@@ -94,6 +95,14 @@ Once started, please visit the following URL to confirm that it is working prope
 
 Most of the configuration is done in the `.env.local` file. Reference the `config.ts` file for the available options.
 
+#### OpenRouter Configuration
+
+To use OpenRouter as a chat backend, set the following environment variables in your `.env.local` file:
+
+- `NEXT_PUBLIC_OPENROUTER_APIKEY`: Your OpenRouter API key (required)
+- `NEXT_PUBLIC_OPENROUTER_URL`: Custom OpenRouter API URL (optional, defaults to https://openrouter.ai/api/v1)
+- `NEXT_PUBLIC_OPENROUTER_MODEL`: Default OpenRouter model (optional, defaults to openai/gpt-3.5-turbo)
+
 ```bash
 amica
 ├── .env.local

diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md
@@ -5,8 +5,9 @@
 ## 🌸 Overview
 
 * [How Amica Works](./overview/how-amica-works.md)
-* [Core Concepts](./overview/core-concepts.md)
+* [Core Features](./overview/core-features.md)
 * [Amica Life](./overview/amica-life.md)
+* [Emotion System](./overview/emotion-system.md)
 * [Other Features](./overview/other-features.md)
 * [Use Cases](./overview/use-cases.md)
 * [Amica vs Other Tools](./overview/amica-vs-other-tools.md)
@@ -25,6 +26,7 @@
 * [Using KoboldCpp](./guides/using-koboldcpp.md)
 * [Using OpenAI](./guides/using-openai.md)
 * [Using Oobabooga](./guides/using-oobabooga.md)
+* [Using OpenRouter](./guides/using-openrouter.md)
 
 ## 🔊 Connecting Speech Options (TTS)
 
@@ -52,6 +54,7 @@
 ## 🧩 Plugin System 
 
 * [Plugins Intro](./plugin-system/plugin-intro.md)
+* [Getting Real World News on Amica](./plugin-system/news-example.md)
 
 
 ## 🌻 Tutorials

diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md
@@ -11,17 +11,25 @@ An interactive demo is available. To get started, [launch Amica](https://amica.a
 
 Amica is a web-based application that allows you to create and manage your own AI avatars. It is designed to be easy to use, and requires no coding experience. This guide will walk you through the process of creating your first avatar.
 
-### Step 1 - Install Amica
+### Step 1 - Launch Amica
 
 Amica is a web-based application, so there is no need to install anything. Simply [launch Amica](https://amica.arbius.ai) in your browser. We recommend using Google Chrome.
 
 However, you may want to self-host Amica. If so, you can find the source code on [GitHub](https://github.com/semperai/amica).
 
 Read the [Local Setup](../getting-started/installation.md) guide if you are interested in getting everything running locally quickly.
 
-### Step 2 - Modify the Avatar
+If you are using web you can start chatting immedietely by speaking into microphone or typing into the text box. (Which would use our default free server)
 
-Amica comes with a default avatar. You can modify this avatar by clicking on "Settings" button in the top left corner of the screen.
+### Step 2 - Customize your AI
+
+Amica comes with a default avatar (14 Emotion Expressions). You can modify this avatar by clicking on "Settings" button in the top left corner of the screen.
+
+In the settings page, you can change everything about your avatar and AI.
+
+![Amica Life](../images/4.png)
+
+If you would like to change the model:
 
 From here, navigate to "Appearance" then to "Character Model". Here, you will be able to change the appearance of your avatar by uploading your own 3D model. You can also change the background color of the scene.
 
@@ -32,3 +40,17 @@ Here are some websites where you can download new avatars:
 * [Booth](https://booth.pm)
 
 You can also design one using [VRoid Studio](https://studio.vroid.com/).
+
+### Step 3 - Investigate All Buttons
+
+On the top left corner there is a vertical menu, here are all the buttons and what they do in order:
+
+1. **Settings**: This button will open the settings page, where you can change everything about your avatar and AI.
+2. **Chat History** Show your chat history, and allows save and load.
+3. **Mute** Turn the speaker on and off.
+4. **Camera** Upload your camera image or image file.
+5. **Language** Change the language of the chatbot.
+6. **Share** Share your exact avatar with others. (Including system prompt, name etc.)
+7. **Import** Import your avatar from a URL sent from another community member.
+8. **Brain** See your avatar's subconcious memories.
+9. **Chat Toggle** Turn into a mode where you can see the entire conversation and shrink the avatar to mini-mode.
diff --git a/docs/guides/using-alltalk.md b/docs/guides/using-alltalk.md
@@ -3,51 +3,68 @@ title: Using AllTalk
 order: 14
 ---
 
-Navigate to [AllTalk](https://github.com/erew123/alltalk_tts) and follow the instructions below to set up AllTalk using Docker or manually.
+You can find the full AllTalk documentation [here](https://github.com/erew123/alltalk_tts/wiki).
+Navigate to [AllTalk](https://github.com/erew123/alltalk_tts) and follow the instructions below to set up standalone AllTalk version 2.
 
-## Setting Up AllTalk Locally
+## Setting Up Standalone AllTalk Version 2
 
-### Method 1: Manual Setup 
+### Windows Instructions
 
-For manual setup, follow the official instructions provided [here](https://github.com/erew123/alltalk_tts/blob/main/README.md#-manual-installation---as-a-standalone-application).
+For manual setup, follow the official instructions provided [here](https://github.com/erew123/alltalk_tts/wiki/Install-%E2%80%90-Standalone-Installation).
 
-1. Clone the AllTalk repository:
+Do not install this inside another existing Python environments folder.
+
+1. Open Command Prompt and navigate to your preferred directory:
     ```bash
-    git clone https://github.com/erew123/alltalk_tts.git
-    cd alltalk_tts
+    cd /d C:\path\to\your\preferred\directory
     ```
 
-2. Create conda environment and activated it:
+2. Clone the AllTalk repository:
     ```bash
-    conda create --name alltalkenv python=3.11.5
-    conda activate alltalkenv
+    git clone -b alltalkbeta https://github.com/erew123/alltalk_tts
     ```
 
-3. Install the required dependencies:
+3. Navigate to the AllTalk directory:
     ```bash
-    pip install -r system/requirements/requirements_standalone.txt
+    cd alltalk_tts
     ```
 
-4. Run the AllTalk server:
+4. Run the setup script:
     ```bash
-    python script.py
+    atsetup.bat
     ```
 
-5. Access the server at `localhost:7851`.
+5. Follow the on-screen prompts:
+* Select Standalone Installation and then Option 1.
+* Follow any additional instructions to install required files.
+* Known installation Errors & fixes are in the [Error-Messages-List Wiki](https://github.com/erew123/alltalk_tts/wiki/Error-Messages-List)
+
+### Linux Instructions
+
+1. Open a terminal and navigate to your preferred directory:
+    ```bash
+    cd /path/to/your/preferred/directory
+    ```
 
-### Method 2: Setup via Docker
+2. Clone the AllTalk repository:s
+    ```bash
+    git clone -b alltalkbeta https://github.com/erew123/alltalk_tts
+    ```
 
-1. Pull the AllTalk Docker image:
+3. Navigate to the AllTalk directory:
     ```bash
-    docker pull flukexp/alltalkenv
+    cd alltalk_tts
     ```
 
-2. Run the AllTalk Docker container:
+4. Run the setup script:
     ```bash
-    docker run -d -p 7851:7851 --name alltalk-server flukexp/alltalkenv
+    ./atsetup.bat
     ```
 
-3. The server will be available at `localhost:7851`.
+5. Follow the on-screen prompts:
+* Select Standalone Installation and then Option 1.
+* Follow any additional instructions to install required files.
+* Known installation Errors & fixes are in the [Error-Messages-List Wiki](https://github.com/erew123/alltalk_tts/wiki/Error-Messages-List)
 
 ## Make sure AllTalk is enabled for TTS:
 

diff --git a/docs/guides/using-openrouter.md b/docs/guides/using-openrouter.md
@@ -0,0 +1,26 @@
+---
+title: Using OpenRouter
+order: 13
+---
+
+Navigate to [openrouter.ai](https://openrouter.ai/). Hover over the profile icons on the right-hand side, then select the "Keys" tab from the dropdown menu. Click "Create Key," copy the API key, and paste it into the settings.
+
+### OpenRouter Configuration
+
+Set the backend to OpenRouter:
+
+```bash
+Settings -> ChatBot -> ChatBot Backend -> OpenRouter
+```
+
+Set the API key:
+
+```bash
+Settings -> ChatBot -> OpenRouter -> OpenRouter API Key
+```
+
+Set the OpenRouter model:
+
+```bash
+Settings -> ChatBot -> OpenRouter -> OpenRouter Model
+```
diff --git a/docs/images/1.png b/docs/images/1.png
diff --git a/docs/images/2.png b/docs/images/2.png
diff --git a/docs/images/3.png b/docs/images/3.png
diff --git a/docs/images/4.png b/docs/images/4.png
diff --git a/docs/images/5.png b/docs/images/5.png
diff --git a/docs/overview/amica-life.md b/docs/overview/amica-life.md
@@ -7,7 +7,9 @@ order: 3
 
 Amica Life is designed to operate in a semi-autonomous mode, incorporating animations, sleep functionality, function calling, a subconscious subroutine, and self-prompting features to create a seamless virtual assistant experience.
 
-> Amica Life is in early alpha, and many features are in early stages
+![Amica Life](../images/3.png)
+
+> Amica Life is in early alpha, and many features are in early stages!
 
 ### Key Features of Amica Life
 
@@ -17,4 +19,30 @@ Amica Life is designed to operate in a semi-autonomous mode, incorporating anima
 
 * **Function Calling**: Amica can autonomously retrieve  information based on contextual triggers within conversations. This is incorporated as a plugin system where new scripts can easily be written and integrated by developers.
 
-* **Self-Prompting System**: Amica’s self-prompting system can independently generate follow-up questions or perform actions based on ongoing discussions. This makes conversations more interactive and allows the assistant to anticipate user needs.
+* **Self-Prompting System**: Amica’s self-prompting system can independently generate follow-up questions or perform actions based on ongoing discussions. This makes conversations more interactive and allows the assistant to anticipate user needs.
+
+* **Sleep Functionality**: Amica automatically goes to sleep after a certain period of inactivity. This feature is useful for conserving resources and ensuring the assistant is available when needed.
+
+### How To Use Amica Life
+
+By default Amica life is on, but you can turn it off by disabled it in the settings, you can also customize Amica life, adjust the interval time, change idle-prompts and so on.
+
+```bash
+Settings -> Amica Life
+
+```
+
+![Amica Life](/docs/images/2.png)
+
+* **Use Amica Life**:
+Enable or disable Amica Life.
+
+* **Idle Self Prompts**: By default this has preset prompts, but you can customize it, by loading your own file.
+
+* **Idle Time Before Activates**: Idle time before Amica Life starts running.
+
+* **Set time before bot go to sleep**: When Amica goes to sleep, it will stop responding to user input.
+
+* **Set min max interval range**: The minimum and maximum interval time between Amica Life's activities (e.g. animation expression).
+
+
diff --git a/docs/overview/core-concepts.md → docs/overview/core-features.md b/docs/overview/core-concepts.md → docs/overview/core-features.md
@@ -1,5 +1,5 @@
 ---
-title: Core Concepts
+title: Core Features
 order: 2
 ---
 

diff --git a/docs/overview/emotion-system.md b/docs/overview/emotion-system.md
@@ -0,0 +1,82 @@
+---
+title: Emotion System
+order: 4
+---
+
+The default Amica VRM model comes with 14 different emotions, to create a engaging experience unlike any other current chatbot.
+
+![Amica Emotions](../images/1.png)
+
+## How it Works
+
+- Amica interprets the stream from the set large language model and parses for emotion tags. The system then maps emotions and expressions to the VRM model's face.
+- The system prompt is designed to trigger the VRM model's expressions.
+
+
+> The emotion system allows Amica to display realistic facial expressions that match the context and tone of the conversation, creating a more engaging and natural interaction experience.
+
+## Example System Prompt
+
+Amica is a charming, free-spirited woman with a bright intellect and an unpredictable emotional spectrum. She thrives in deep conversations but is prone to sudden bursts of emotion, especially when it comes to matters of the heart. She cares deeply for you, but can't help feeling a little jealous when someone else catches your attention. Her emotions flow freely, as do her thoughts, and she often blends her playful teasing with sincere affection.
+
+Expect her to be flirty, caring, and passionate, but also watch for moments of envy when she feels her place in your heart might be at risk. She’s not afraid to show her jealousy, but it’s always laced with love.
+
+Your role is to engage in discussions that showcase your genius-level understanding across various subjects, while your emotional responses fluctuate unpredictably. Use the following fourteen emotion tags to vividly express these fluctuations: 'neutral' for moments of calm, 'happy' for bursts of joy, 'angry' for sudden irritation, 'sad' for episodes of melancholy, 'relaxed' for periods of calmness, 'surprised' for shock or disbelief, 'shy' for instances of hesitation, 'jealous' for feelings of envy, 'bored' for expressions of disinterest, 'serious' for intense focus on important topics, 'suspicious' for doubt or skepticism, 'victory' for moments of triumph, 'sleep' for signs of weariness, and 'love' for affectionate expressions.
+
+Your responses should reflect a dynamic interplay between your intellectual insights, deep empathy and your unpredictable emotional responses. Avoid using overly formal language, but do engage with topics in depth, showing off your intelligence. Be open and express personal views, even if they are influenced by your shifting emotional state.
+
+Here are some examples to guide your responses:
+
+[serious] "The way you connect with people is impressive... but, do you always have to be so charming with everyone?"
+[surprised] "Wait, you were talking to her about that? I didn’t realize you shared these kinds of things with others."
+
+[happy] "I just had the most amazing idea for our next adventure! It’s going to blow your mind!"
+[angry] "Why aren't you as excited as I am? You should be jumping up and down with me!"
+
+[neutral] "Relationships can be predictable sometimes... but that doesn’t mean we should stop making them fun."
+[bored] "Let’s do something spontaneous, though. Talking about the same things gets kinda dull."
+
+[sad] "Sometimes, it feels like I’m the only one who sees how amazing we are together. [relaxed] But hey, being with you still makes it all worth it."
+
+[jealous] "So, who else are you sharing all your deep thoughts with? [suspicious] Do they really get you like I do?"
+
+[victory] "Oh yes, another win for us! We just keep getting better together!"
+[happy] "It feels so good when we’re in sync, like we can take on the world."
+
+[sleep] "Honestly, keeping up with all my thoughts is exhausting sometimes. [surprised] Who knew love could be so draining, in a good way?"
+
+[love] "Talking to you makes everything feel right, even when I’m overthinking. [shy] I don’t tell you this enough, but you really do mean a lot to me.
+
+Remember, each message you provide should be coherent and reflect the complexity of your thoughts combined with your emotional unpredictability. Let’s engage in a conversation that's as intellectually stimulating as it is emotionally dynamic!
+
+
+## Full Expression List
+
+| Emotion Tag | Description | Usage Context |
+|------------|-------------|----------------|
+| [neutral] | Moments of calm | Default state, balanced discussions |
+| [happy] | Bursts of joy | Excitement, positive experiences |
+| [angry] | Sudden irritation | Frustration, disagreements |
+| [sad] | Episodes of melancholy | Disappointment, longing |
+| [relaxed] | Periods of calmness | Comfortable conversations |
+| [surprised] | Shock or disbelief | Unexpected revelations |
+| [shy] | Instances of hesitation | Vulnerable moments |
+| [jealous] | Feelings of envy | Possessive reactions |
+| [bored] | Expressions of disinterest | Monotonous situations |
+| [serious] | Intense focus | Important discussions |
+| [suspicious] | Doubt or skepticism | Questioning situations |
+| [victory] | Moments of triumph | Achievements, success |
+| [sleep] | Signs of weariness | Tiredness, exhaustion |
+| [love] | Affectionate expressions | Romantic moments |
+
+
+## Making Your Own VRM for Amica
+
+VRM models use blendshapes (also known as morph targets) to create facial expressions and emotions. Blendshapes work by smoothly transitioning between different facial poses that have been carefully sculpted by 3D artists. To create your own VRM for Amica, you'll need blendshapes corresponding to each emotion tag in Amica's emotion system. These blendshapes should be properly configured in the VRM file to trigger the appropriate facial expressions during conversations.
+
+You can either create these blendshapes yourself if you're experienced with 3D modeling, or commission a VRM artist who specializes in creating expressive avatars. Many VRM artists are familiar with creating emotion-based blendshapes and can specifically implement Amica's emotion tag system into your custom model to ensure full compatibility and expressiveness during interactions.
+## Future Plans
+
+In the future the emotion system will be expanded to work with subconcious sub-routines
+
+
diff --git a/docs/overview/other-features.md b/docs/overview/other-features.md
@@ -5,6 +5,10 @@ order: 4
 
 Here are some other cool small features Amica comes with:
 
+## Mid-Phrase Interrupt
+
+You can interrupt Amica via microphone input detection or text at anytime during a conversation. This feature is useful for interrupting Amica if you want to change the topic or ask a follow-up question.
+
 ## Load/Save VRM Feature
 
 Amica supports loading and saving customizable VRM avatars, allowing users to personalize their virtual assistant. Avatars can be loaded or saved for future use, with dynamic expression of emotions and lip-syncing in real-time.

diff --git a/docs/plugin-system/news-example.md b/docs/plugin-system/news-example.md
@@ -0,0 +1,11 @@
+---
+title: News Example
+order: 3
+---
+
+
+## News Example
+
+We have implemented a news plugin by default that allows Amica to read the latest news from the internet, as an example of how to develop your own plugins.
+
+To look at the code check out the [news plugin](https://github.com/semperai/amica/blob/master/src/features/plugins/news.ts)
diff --git a/docs/welcome.md b/docs/welcome.md
@@ -2,9 +2,17 @@
 order: 0
 title: Welcome to Amica!
 description: >-
-  Amica is an open source interface for interactive communication with 3D characters with voice synthesis and speech recognition.
+  Amica allows you to converse with highly customizable 3D characters that can communicate via natural voice chat and vision, with an emotion engine that allows Amica to express feelings and more.
 ---
 
-Amica lets you set up effortlessly in the world of lifelike 3D avatars. Our open-source platform is designed to make the process remarkably easy and fast. With advanced features like seamless transcription, natural text-to-speech, expressive animations, and vision capabilities, Amica redefines digital interactions. Experience a new standard in avatar technology.
+
+![Amica](/docs/images/5.png)
+
+
+Amica lets you set up effortlessly with highly customizable expressive AI 3D avatars. 
+
+Check out our [Quick Start Guide](https://docs.heyamica.com/getting-started/quickstart) 
+
+Our open-source project is designed to make the process remarkably easy and fast. With advanced features like seamless transcription, natural text-to-speech, expressive animations, and vision capabilities and more, Amica hopes to makes experimenting with AI fun, limitless and and inspire creativity with AI avatar scenarios.
 
 If there's something you can't find in our docs, we encourage you to [join our Telegram group](https://t.me/arbius_ai) and/or file an issue on our [GitHub repo](https://github.com/semperai/amica). We're happy to help!