diff --git a/docs/architecture.md b/docs/architecture.md index 81c9735..ee5e5de 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,12 +1,12 @@ # Architecture -Cube AI is built on top of Magistrala, Ollama and a custom proxy server. All these host the Cube AI API that hosts the AI models and protects prompts and user data securely. +Cube AI is built on top of SuperMQ, Ollama and a custom proxy server. All these host the Cube AI API that hosts the AI models and protects prompts and user data securely. ![Architecture Image](./img/architecture.png) -## Magistrala +## SuperMQ -Cube AI uses Magistrala Users and Auth Service to manage users and authentication. Magistrala is an IoT platform that provides a secure and scalable way to manage IoT devices. Since Magistrala is based on micro-service architecture, auth and users are separated from the core application. This allows for better scalability and security. Magistrala is responsible for users authentication, authorization and user management. It also provides a secure way to store user data. +Cube AI uses SuperMQ Users and Auth Service to manage users and authentication. SuperMQ is an IoT platform that provides a secure and scalable way to manage IoT devices. Since SuperMQ is based on micro-service architecture, auth and users are separated from the core application. This allows for better scalability and security. SuperMQ is responsible for users authentication, authorization and user management. It also provides a secure way to store user data. ## Ollama @@ -14,4 +14,4 @@ Ollama is a framework that provides a unified interface for interacting with dif ## Proxy Server -The proxy server is responsible for handling requests to ollama. Once a user is registered on Magistrala and issued with an access token, the user can interact with Cube AI using the issued token. The proxy server will verify the token and ensure that the user has the necessary permissions to access the Cube AI API by checking it with the Magistrala Auth Service. Once the request is verified, the proxy server will forward the request to the appropriate Ollama instance. The proxy server also handles authentication and authorization for the user. It ensures that only authorized users can access the Cube AI API. +The proxy server is responsible for handling requests to ollama. Once a user is registered on SuperMQ and issued with an access token, the user can interact with Cube AI using the issued token. The proxy server will verify the token and ensure that the user has the necessary permissions to access the Cube AI API by checking it with the SuperMQ Auth Service. Once the request is verified, the proxy server will forward the request to the appropriate Ollama instance. The proxy server also handles authentication and authorization for the user. It ensures that only authorized users can access the Cube AI API. diff --git a/docs/developer-guide.md b/docs/developer-guide.md index 1f1518e..b8e0ac1 100644 --- a/docs/developer-guide.md +++ b/docs/developer-guide.md @@ -1,9 +1,11 @@ # Developer Guide -This guide will help you get started with developing, deploying, and running Cube AI. +This guide will help you get started with developing, testing, deploying, and running Cube AI. ## Cloning the Repository +Clone Cube AI repository by running: + ```bash git clone https://github.com/ultravioletrs/cube.git cd cube @@ -11,6 +13,8 @@ cd cube ## Pulling Docker Images +Pull the Docker images by running: + ```bash cd cube/docker/ docker compose pull @@ -18,23 +22,82 @@ docker compose pull ## Running Services with Docker Compose -You can run/start Cube AI services using Docker Compose as described in [this guide](https://github.com/ultravioletrs/cube/blob/main/hal/ubuntu/README.md). +If you are running Cube AI on your local machine: -To properly access Cube AI UI deployed on a different server, update the IP address entries in `docker/.env` as described in the above guide to point to your server IP address. The Cube AI UI can then be accessed through your browser at: +1. **Update the Environment File** + Open the `docker/.env` file and set the `UV_CUBE_NEXTAUTH_URL` to point to `localhost` using the specified UI port (default is `6193`): -```bash -http://:3001 -``` + ```bash + UV_CUBE_NEXTAUTH_URL=http://localhost:${UI_PORT} + ``` -For example, if you have deployed locally, use: +2. **Start the Docker Containers** + Run the following command to start the services: -```bash -http://localhost:3001 -``` + ```bash + docker compose up -d + ``` + +3. **Access the UI** + Open your browser and navigate to: + + ```bash + http://localhost:6193 + ``` + +If you are running Cube AI on a different server (e.g., a virtual machine or cloud instance): + +1. **Update the Environment File** + Open the `docker/.env` file and replace the placeholders with your server's IP address and UI port. For example, if your server's IP address is `209.18.187.53` and you are using the default port `6193`, update the file as follows: + + ```bash + UI_PORT=6193 + UV_CUBE_NEXTAUTH_URL=http://209.18.187.53:${UI_PORT} + ``` + + You can set a custom port if `6193` is unavailable or conflicts with other services. For instance, if you want to use port `8080`, update the `docker/.env` file as follows: + + ```bash + UI_PORT=8080 + UV_CUBE_NEXTAUTH_URL=http://209.18.187.53:${UI_PORT} + ``` + + Ensure the chosen port is open and not restricted by firewalls. + +2. **Start the Docker Containers** + Run the following command on the server: + + ```bash + docker compose up -d + ``` + +3. **Access the UI** + Open your browser and navigate to: + + - For the default port: + + ```bash + http://209.18.187.53:6193 + ``` + + - For a custom port (e.g., `8080`): + + ```bash + http://209.18.187.53:8080 + ``` + +After successfully deploying Cube AI, you can perform various tasks to set up and manage your instance. These include: + +- Managing domains +- Managing users +- Generating authentication tokens (using curl command) +- Intergrating your `Cube AI` instance with `Continue` Visual Studio Code extension to leverage large language models (LLMs) directly within VS Code to assist with coding task + +For detailed instructions on the above tasks and more, refer to the [Getting Started Guide](https://github.com/ultravioletrs/cube-docs/blob/main/docs/getting-started.md). ## Open Web UI Integration -Open Web UI is integrated into Cube AI to help in debugging and monitoring key performance metrics of the models, including response token/s, prompt token/s, total duration, and load duration. For more detailed setup and configuration instructions, refer to the [Open Web UI documentation](https://docs.openwebui.com/). +Open Web UI is integrated into Cube AI to help in debugging and monitoring key performance metrics of the LLMs, including response token/s, prompt token/s, total duration, and load duration. For more detailed setup and configuration instructions, refer to the [Open Web UI documentation](https://docs.openwebui.com/). To access Open Web UI, once Cube AI services are up and running, open your browser and navigate to: @@ -108,12 +171,12 @@ For detailed instructions on setting up and building Cube HAL, please refer to [ Once SSH is set up, use `scp` to securely copy the `.tar.gz` file from step 1 to the CVM. For detailed steps, refer to the [SSH guide](https://github.com/ultravioletrs/cube/blob/main/hal/buildroot/ssh.md). 5. **Decompress and Extract the Model Files on the CVM** - After transferring the archive to the CVM, log in to the CVM and use `gunzip` to decompress the `.tar.gz` file, then extract the `.tar` file: + After transferring the archive to the CVM, log in to the CVM and use `gunzip` to decompress the `.tar.gz` file, then extract the `.tar` file: - ```bash - gunzip model-name.tar.gz - tar -xvf model-name.tar - ``` + ```bash + gunzip model-name.tar.gz + tar -xvf model-name.tar + ``` 6. **Copy the Extracted Model Files to the `ollama` Container** diff --git a/docs/getting-started.md b/docs/getting-started.md index 92ce631..6e9a9fb 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -1,36 +1,36 @@ # Getting Started -Before logging in or interacting with Cube AI, you need to make sure that Cube AI has been deployed and is running. Deployment can be done by the instance administrator or by you if you have the necessary access permissions. Therefore, to Connect: +Before logging in or interacting with Cube AI, ensure that Cube AI has been properly deployed and is running. Deployment can be performed by the instance administrator or by you, provided you have the necessary access permissions. For a detailed guide on how to deploy Cube AI, refer to the [developer guide](https://github.com/ultravioletrs/cube-docs/blob/main/docs/developer-guide.md). + +Therefore, to connect: 1. Ensure that Cube AI is deployed and running on your instance. 2. Connect to the deployed Cube AI instance on port `6193` using your login credentials (username and password). -Port `6193` is our default port for accessing Cube AI deployment. It is configurable through the `.env` file, meaning you can change it to any port that suits your deployment needs as described in [this guide](https://github.com/ultravioletrs/cube/blob/main/hal/ubuntu/README.md) +Port `6193` is the default port for accessing Cube AI deployment. It is configurable through the `docker/.env` file, meaning you can change it to any port that suits your deployment needs as described in the [developer guide](https://github.com/ultravioletrs/cube-docs/blob/main/docs/developer-guide.md). -For example, to customize the port: +## Administrator and User Accounts -```bash -UV_CUBE_UI_BASE_URL=http://: -``` +### Administrator Access -If you are running Cube AI on a different port, make sure to update the environment variable accordingly. +If you are the instance administrator, you do not need to create a separate account for yourself. The platform is preconfigured with an administrator account that you can log into directly using your admin credentials. As the administrator, you can also generate authentication tokens for API access. Non-admin users, however, will need accounts created for them by the administrator. -## Create a new user +### Creating a New User Account -If you are the instance administrator (or have administrator credentials), follow [this demonstration](https://jam.dev/c/f8d3fa47-7505-4201-b8ca-c0f724826237) to create a new user. Below are the summarized steps: +As an administrator, you have the ability to create accounts for non-admin users and grant them access to Cube AI. Follow [this demonstration](https://jam.dev/c/f8d3fa47-7505-4201-b8ca-c0f724826237) to see the process in action. Here’s a summary of the steps: -1. Log in with admin credentials -2. Create a new domain (if needed) -3. Login to the newly created domain (or already existing domain) -4. Click profile icon and navigate to `Manage Users` -5. Click create -6. Fill in the form -7. Click `Create` -8. Share the login credentials (username and password) with the new user +1. **Log in** using your **administrator credentials**. +2. **Create a new domain** (if one is needed). +3. **Log in** to the newly created domain (or an existing domain). +4. Click on your **profile icon** and select **`Manage Users`**. +5. Click **`Create`** to start creating a new user. +6. Fill out the user details in the form. +7. Click **`Create`** to finalize the user creation. +8. **Share** the username and password with the newly created user so they can log in. -## Login +### Non-Admin User Login -Once the administrator creates your account and shares your login details, use the credentials to log in to Cube AI and obtain an authentication token as shown below. +Once the administrator has created your account and shared the login details with you, use those credentials to log in to Cube AI. After logging in, you can obtain an authentication token for API interactions as shown below: ```bash curl -ksSiX POST https:///users/tokens/issue -H "Content-Type: application/json" -d @- << EOF @@ -41,7 +41,9 @@ curl -ksSiX POST https:///users/tokens/issue -H "Content-Type: EOF ``` -Be sure to replace `` and `` with your actual login details. You'll receive a response similar to this: +Replace `` and `` with the credentials provided by the administrator. + +You will receive a response similar to the following: ```bash HTTP/2 201 @@ -54,43 +56,110 @@ content-length: 591 {"access_token":"","refresh_token":""} ``` -The `access_token` field in the response is your API token, which will be used for authentication in future API calls. The `refresh_token` field is a token you can use to obtain a new access token once the current one expires. +The `access_token` field contains your API token, which is required for making authenticated API calls. The `refresh_token` can be used to obtain a new access token when the current one expires. -## VS Code Setup +## Setting Up VS Code for Cube AI Integration -1. Download and install [VS Code](https://code.visualstudio.com/). -2. In VS Code, download and install the [Continue extension](https://www.continue.dev/). This extension connects Cube AI models to your coding environment for assistance. -3. Open the Continue extension, click the settings icon (`Configure Continue`), and open the `.continue/config.json` file (alternatively, you can navigate to the `.continue` folder in your project’s root directory via File Explorer or press `Ctrl+Shift+P` and type "Continue: Open config.json" in the Command Palette). -4. Edit the `.continue/config.json` file to look like this: +To maximize Cube AI’s potential within your development environment, you’ll need to integrate it with Visual Studio Code (VS Code) using the **Continue extension**. This extension enables you to directly interact with LLMs in TEE inside VS Code, providing intelligent code completion, code suggestions, and contextual insights. + +### Steps for Setting Up + +1. **Download and install** [Visual Studio Code (VS Code)](https://code.visualstudio.com/). +2. In VS Code, **download and install** the [Continue extension](https://www.continue.dev/), which connects Cube AI models to your development environment for enhanced coding assistance. +3. **Open the Continue extension** by clicking the settings icon (gear icon), then select **`Configure Continue`**. This will open the `.continue/config.json` file. Alternatively: + - You can navigate to the `.continue` folder in your project’s root directory using File Explorer. + - Press `Ctrl+Shift+P` to open the Command Palette and search for **"Continue: Open config.json"**. +4. **Edit** the `.continue/config.json` file to include the following configuration: ```json { "models": [ { - "title": "Ollama", + "title": "tinyllama", "provider": "ollama", - "model": "llama3:8b", + "model": "tinyllama:1.1b", "apiKey": "", - "apiBase": "/ollama" + "apiBase": "http:///ollama" } ], "tabAutocompleteModel": { "title": "Starcoder 2 3b", "provider": "ollama", - "model": "starcoder2:3b", + "model": "starcoder2:7b", "apiKey": "", - "apiBase": "/ollama" + "apiBase": "http:///ollama" }, "embeddingsProvider": { "provider": "ollama", "model": "nomic-embed-text", "apiKey": "", - "apiBase": "/ollama" + "apiBase": "http:///ollama" }, "requestOptions": { "verifySsl": false - } + }, + "customCommands": [ + { + "name": "test", + "prompt": "{{{ input }}}\n\nWrite a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.", + "description": "Write unit tests for highlighted code" + } + ], + "contextProviders": [ + { + "name": "code", + "params": {} + }, + { + "name": "docs", + "params": {} + }, + { + "name": "diff", + "params": {} + }, + { + "name": "terminal", + "params": {} + }, + { + "name": "problems", + "params": {} + }, + { + "name": "folder", + "params": {} + }, + { + "name": "codebase", + "params": {} + } + ], + "slashCommands": [ + { + "name": "edit", + "description": "Edit selected code" + }, + { + "name": "comment", + "description": "Write comments for the selected code" + }, + { + "name": "share", + "description": "Export the current chat session to markdown" + }, + { + "name": "cmd", + "description": "Generate a shell command" + }, + { + "name": "commit", + "description": "Generate a git commit message" + } + ] } ``` +Update the `apiKey` with your `access token` and the `apiBase` with the URL of your Cube AI instance (if different from the default one). These values should reflect the actual deployment settings you're working with. + For a more detailed explanation of how to connect to Cube AI with the continue extension, check out [this video demonstration](https://www.youtube.com/watch?v=BGpv_iTB2NE). diff --git a/docs/img/architecture.png b/docs/img/architecture.png index 9327157..c6d73bf 100644 Binary files a/docs/img/architecture.png and b/docs/img/architecture.png differ