Skip to content

Commit

Permalink
Merge pull request #13 from JeffMboya/updateuiport
Browse files Browse the repository at this point in the history
NOISSUE - Enhance docs
  • Loading branch information
drasko authored Nov 29, 2024
2 parents 3f2d6fb + d0a27c1 commit 66cbd9d
Show file tree
Hide file tree
Showing 4 changed files with 185 additions and 53 deletions.
8 changes: 4 additions & 4 deletions docs/architecture.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# Architecture

Cube AI is built on top of Magistrala, Ollama and a custom proxy server. All these host the Cube AI API that hosts the AI models and protects prompts and user data securely.
Cube AI is built on top of SuperMQ, Ollama and a custom proxy server. All these host the Cube AI API that hosts the AI models and protects prompts and user data securely.

![Architecture Image](./img/architecture.png)

## Magistrala
## SuperMQ

Cube AI uses Magistrala Users and Auth Service to manage users and authentication. Magistrala is an IoT platform that provides a secure and scalable way to manage IoT devices. Since Magistrala is based on micro-service architecture, auth and users are separated from the core application. This allows for better scalability and security. Magistrala is responsible for users authentication, authorization and user management. It also provides a secure way to store user data.
Cube AI uses SuperMQ Users and Auth Service to manage users and authentication. SuperMQ is an IoT platform that provides a secure and scalable way to manage IoT devices. Since SuperMQ is based on micro-service architecture, auth and users are separated from the core application. This allows for better scalability and security. SuperMQ is responsible for users authentication, authorization and user management. It also provides a secure way to store user data.

## Ollama

Ollama is a framework that provides a unified interface for interacting with different LLMs. Ollama is responsible for managing LLMs and their configurations. It provides a unified interface for interacting with different LLMs, allowing developers to easily switch between different LLMs without having to change their code. Ollama also provides a way to manage LLMs, including configuring LLMs, managing prompts, and managing models.

## Proxy Server

The proxy server is responsible for handling requests to ollama. Once a user is registered on Magistrala and issued with an access token, the user can interact with Cube AI using the issued token. The proxy server will verify the token and ensure that the user has the necessary permissions to access the Cube AI API by checking it with the Magistrala Auth Service. Once the request is verified, the proxy server will forward the request to the appropriate Ollama instance. The proxy server also handles authentication and authorization for the user. It ensures that only authorized users can access the Cube AI API.
The proxy server is responsible for handling requests to ollama. Once a user is registered on SuperMQ and issued with an access token, the user can interact with Cube AI using the issued token. The proxy server will verify the token and ensure that the user has the necessary permissions to access the Cube AI API by checking it with the SuperMQ Auth Service. Once the request is verified, the proxy server will forward the request to the appropriate Ollama instance. The proxy server also handles authentication and authorization for the user. It ensures that only authorized users can access the Cube AI API.
95 changes: 79 additions & 16 deletions docs/developer-guide.md
Original file line number Diff line number Diff line change
@@ -1,40 +1,103 @@
# Developer Guide

This guide will help you get started with developing, deploying, and running Cube AI.
This guide will help you get started with developing, testing, deploying, and running Cube AI.

## Cloning the Repository

Clone Cube AI repository by running:

```bash
git clone https://github.com/ultravioletrs/cube.git
cd cube
```

## Pulling Docker Images

Pull the Docker images by running:

```bash
cd cube/docker/
docker compose pull
```

## Running Services with Docker Compose

You can run/start Cube AI services using Docker Compose as described in [this guide](https://github.com/ultravioletrs/cube/blob/main/hal/ubuntu/README.md).
If you are running Cube AI on your local machine:

To properly access Cube AI UI deployed on a different server, update the IP address entries in `docker/.env` as described in the above guide to point to your server IP address. The Cube AI UI can then be accessed through your browser at:
1. **Update the Environment File**
Open the `docker/.env` file and set the `UV_CUBE_NEXTAUTH_URL` to point to `localhost` using the specified UI port (default is `6193`):

```bash
http://<your-server-ip-address>:3001
```
```bash
UV_CUBE_NEXTAUTH_URL=http://localhost:${UI_PORT}
```

For example, if you have deployed locally, use:
2. **Start the Docker Containers**
Run the following command to start the services:

```bash
http://localhost:3001
```
```bash
docker compose up -d
```

3. **Access the UI**
Open your browser and navigate to:

```bash
http://localhost:6193
```

If you are running Cube AI on a different server (e.g., a virtual machine or cloud instance):

1. **Update the Environment File**
Open the `docker/.env` file and replace the placeholders with your server's IP address and UI port. For example, if your server's IP address is `209.18.187.53` and you are using the default port `6193`, update the file as follows:

```bash
UI_PORT=6193
UV_CUBE_NEXTAUTH_URL=http://209.18.187.53:${UI_PORT}
```

You can set a custom port if `6193` is unavailable or conflicts with other services. For instance, if you want to use port `8080`, update the `docker/.env` file as follows:

```bash
UI_PORT=8080
UV_CUBE_NEXTAUTH_URL=http://209.18.187.53:${UI_PORT}
```

Ensure the chosen port is open and not restricted by firewalls.

2. **Start the Docker Containers**
Run the following command on the server:

```bash
docker compose up -d
```

3. **Access the UI**
Open your browser and navigate to:

- For the default port:

```bash
http://209.18.187.53:6193
```

- For a custom port (e.g., `8080`):

```bash
http://209.18.187.53:8080
```

After successfully deploying Cube AI, you can perform various tasks to set up and manage your instance. These include:

- Managing domains
- Managing users
- Generating authentication tokens (using curl command)
- Intergrating your `Cube AI` instance with `Continue` Visual Studio Code extension to leverage large language models (LLMs) directly within VS Code to assist with coding task

For detailed instructions on the above tasks and more, refer to the [Getting Started Guide](https://github.com/ultravioletrs/cube-docs/blob/main/docs/getting-started.md).

## Open Web UI Integration

Open Web UI is integrated into Cube AI to help in debugging and monitoring key performance metrics of the models, including response token/s, prompt token/s, total duration, and load duration. For more detailed setup and configuration instructions, refer to the [Open Web UI documentation](https://docs.openwebui.com/).
Open Web UI is integrated into Cube AI to help in debugging and monitoring key performance metrics of the LLMs, including response token/s, prompt token/s, total duration, and load duration. For more detailed setup and configuration instructions, refer to the [Open Web UI documentation](https://docs.openwebui.com/).

To access Open Web UI, once Cube AI services are up and running, open your browser and navigate to:

Expand Down Expand Up @@ -108,12 +171,12 @@ For detailed instructions on setting up and building Cube HAL, please refer to [
Once SSH is set up, use `scp` to securely copy the `.tar.gz` file from step 1 to the CVM. For detailed steps, refer to the [SSH guide](https://github.com/ultravioletrs/cube/blob/main/hal/buildroot/ssh.md).
5. **Decompress and Extract the Model Files on the CVM**
After transferring the archive to the CVM, log in to the CVM and use `gunzip` to decompress the `.tar.gz` file, then extract the `.tar` file:
After transferring the archive to the CVM, log in to the CVM and use `gunzip` to decompress the `.tar.gz` file, then extract the `.tar` file:
```bash
gunzip model-name.tar.gz
tar -xvf model-name.tar
```
```bash
gunzip model-name.tar.gz
tar -xvf model-name.tar
```
6. **Copy the Extracted Model Files to the `ollama` Container**
Expand Down
135 changes: 102 additions & 33 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
@@ -1,36 +1,36 @@
# Getting Started

Before logging in or interacting with Cube AI, you need to make sure that Cube AI has been deployed and is running. Deployment can be done by the instance administrator or by you if you have the necessary access permissions. Therefore, to Connect:
Before logging in or interacting with Cube AI, ensure that Cube AI has been properly deployed and is running. Deployment can be performed by the instance administrator or by you, provided you have the necessary access permissions. For a detailed guide on how to deploy Cube AI, refer to the [developer guide](https://github.com/ultravioletrs/cube-docs/blob/main/docs/developer-guide.md).

Therefore, to connect:

1. Ensure that Cube AI is deployed and running on your instance.
2. Connect to the deployed Cube AI instance on port `6193` using your login credentials (username and password).

Port `6193` is our default port for accessing Cube AI deployment. It is configurable through the `.env` file, meaning you can change it to any port that suits your deployment needs as described in [this guide](https://github.com/ultravioletrs/cube/blob/main/hal/ubuntu/README.md)
Port `6193` is the default port for accessing Cube AI deployment. It is configurable through the `docker/.env` file, meaning you can change it to any port that suits your deployment needs as described in the [developer guide](https://github.com/ultravioletrs/cube-docs/blob/main/docs/developer-guide.md).

For example, to customize the port:
## Administrator and User Accounts

```bash
UV_CUBE_UI_BASE_URL=http://<your-instance-ip>:<your-preferred-port>
```
### Administrator Access

If you are running Cube AI on a different port, make sure to update the environment variable accordingly.
If you are the instance administrator, you do not need to create a separate account for yourself. The platform is preconfigured with an administrator account that you can log into directly using your admin credentials. As the administrator, you can also generate authentication tokens for API access. Non-admin users, however, will need accounts created for them by the administrator.

## Create a new user
### Creating a New User Account

If you are the instance administrator (or have administrator credentials), follow [this demonstration](https://jam.dev/c/f8d3fa47-7505-4201-b8ca-c0f724826237) to create a new user. Below are the summarized steps:
As an administrator, you have the ability to create accounts for non-admin users and grant them access to Cube AI. Follow [this demonstration](https://jam.dev/c/f8d3fa47-7505-4201-b8ca-c0f724826237) to see the process in action. Here’s a summary of the steps:

1. Log in with admin credentials
2. Create a new domain (if needed)
3. Login to the newly created domain (or already existing domain)
4. Click profile icon and navigate to `Manage Users`
5. Click create
6. Fill in the form
7. Click `Create`
8. Share the login credentials (username and password) with the new user
1. **Log in** using your **administrator credentials**.
2. **Create a new domain** (if one is needed).
3. **Log in** to the newly created domain (or an existing domain).
4. Click on your **profile icon** and select **`Manage Users`**.
5. Click **`Create`** to start creating a new user.
6. Fill out the user details in the form.
7. Click **`Create`** to finalize the user creation.
8. **Share** the username and password with the newly created user so they can log in.

## Login
### Non-Admin User Login

Once the administrator creates your account and shares your login details, use the credentials to log in to Cube AI and obtain an authentication token as shown below.
Once the administrator has created your account and shared the login details with you, use those credentials to log in to Cube AI. After logging in, you can obtain an authentication token for API interactions as shown below:

```bash
curl -ksSiX POST https://<cube-ai-instance>/users/tokens/issue -H "Content-Type: application/json" -d @- << EOF
Expand All @@ -41,7 +41,9 @@ curl -ksSiX POST https://<cube-ai-instance>/users/tokens/issue -H "Content-Type:
EOF
```

Be sure to replace `<your_email>` and `<your_password>` with your actual login details. You'll receive a response similar to this:
Replace `<your_email>` and `<your_password>` with the credentials provided by the administrator.

You will receive a response similar to the following:

```bash
HTTP/2 201
Expand All @@ -54,43 +56,110 @@ content-length: 591
{"access_token":"<access_token>","refresh_token":"<refresh_token>"}
```

The `access_token` field in the response is your API token, which will be used for authentication in future API calls. The `refresh_token` field is a token you can use to obtain a new access token once the current one expires.
The `access_token` field contains your API token, which is required for making authenticated API calls. The `refresh_token` can be used to obtain a new access token when the current one expires.

## VS Code Setup
## Setting Up VS Code for Cube AI Integration

1. Download and install [VS Code](https://code.visualstudio.com/).
2. In VS Code, download and install the [Continue extension](https://www.continue.dev/). This extension connects Cube AI models to your coding environment for assistance.
3. Open the Continue extension, click the settings icon (`Configure Continue`), and open the `.continue/config.json` file (alternatively, you can navigate to the `.continue` folder in your project’s root directory via File Explorer or press `Ctrl+Shift+P` and type "Continue: Open config.json" in the Command Palette).
4. Edit the `.continue/config.json` file to look like this:
To maximize Cube AI’s potential within your development environment, you’ll need to integrate it with Visual Studio Code (VS Code) using the **Continue extension**. This extension enables you to directly interact with LLMs in TEE inside VS Code, providing intelligent code completion, code suggestions, and contextual insights.

### Steps for Setting Up

1. **Download and install** [Visual Studio Code (VS Code)](https://code.visualstudio.com/).
2. In VS Code, **download and install** the [Continue extension](https://www.continue.dev/), which connects Cube AI models to your development environment for enhanced coding assistance.
3. **Open the Continue extension** by clicking the settings icon (gear icon), then select **`Configure Continue`**. This will open the `.continue/config.json` file. Alternatively:
- You can navigate to the `.continue` folder in your project’s root directory using File Explorer.
- Press `Ctrl+Shift+P` to open the Command Palette and search for **"Continue: Open config.json"**.
4. **Edit** the `.continue/config.json` file to include the following configuration:

```json
{
"models": [
{
"title": "Ollama",
"title": "tinyllama",
"provider": "ollama",
"model": "llama3:8b",
"model": "tinyllama:1.1b",
"apiKey": "<access_token>",
"apiBase": "<cube-ai-instance>/ollama"
"apiBase": "http://<your-ollama-instance>/ollama"
}
],
"tabAutocompleteModel": {
"title": "Starcoder 2 3b",
"provider": "ollama",
"model": "starcoder2:3b",
"model": "starcoder2:7b",
"apiKey": "<access_token>",
"apiBase": "<cube-ai-instance>/ollama"
"apiBase": "http://<your-ollama-instance>/ollama"
},
"embeddingsProvider": {
"provider": "ollama",
"model": "nomic-embed-text",
"apiKey": "<access_token>",
"apiBase": "<cube-ai-instance>/ollama"
"apiBase": "http://<your-ollama-instance>/ollama"
},
"requestOptions": {
"verifySsl": false
}
},
"customCommands": [
{
"name": "test",
"prompt": "{{{ input }}}\n\nWrite a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.",
"description": "Write unit tests for highlighted code"
}
],
"contextProviders": [
{
"name": "code",
"params": {}
},
{
"name": "docs",
"params": {}
},
{
"name": "diff",
"params": {}
},
{
"name": "terminal",
"params": {}
},
{
"name": "problems",
"params": {}
},
{
"name": "folder",
"params": {}
},
{
"name": "codebase",
"params": {}
}
],
"slashCommands": [
{
"name": "edit",
"description": "Edit selected code"
},
{
"name": "comment",
"description": "Write comments for the selected code"
},
{
"name": "share",
"description": "Export the current chat session to markdown"
},
{
"name": "cmd",
"description": "Generate a shell command"
},
{
"name": "commit",
"description": "Generate a git commit message"
}
]
}
```

Update the `apiKey` with your `access token` and the `apiBase` with the URL of your Cube AI instance (if different from the default one). These values should reflect the actual deployment settings you're working with.

For a more detailed explanation of how to connect to Cube AI with the continue extension, check out [this video demonstration](https://www.youtube.com/watch?v=BGpv_iTB2NE).
Binary file modified docs/img/architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 66cbd9d

Please sign in to comment.