Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/main'
Browse files Browse the repository at this point in the history
  • Loading branch information
bentwnghk committed Feb 13, 2025
2 parents 76e7f27 + 5a012e4 commit 3b4df8e
Show file tree
Hide file tree
Showing 3 changed files with 140 additions and 14 deletions.
16 changes: 9 additions & 7 deletions docs/self-hosting/advanced/model-list.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Mr.🆖 AI supports customizing the model list during deployment. This configura
You can use `+` to add a model, `-` to hide a model, and use `model name=display name<extension configuration>` to customize the display name of a model, separated by English commas. The basic syntax is as follows:

```text
id=displayName<maxToken:vision:fc:file>,model2,model3
id=displayName<maxToken:vision:reasoning:fc:file>,model2,model3
```

For example: `+qwen-7b-chat,+glm-6b,-gpt-3.5-turbo,gpt-4-0125-preview=gpt-4-turbo`
Expand All @@ -29,7 +29,7 @@ In the above example, it adds `qwen-7b-chat` and `glm-6b` to the model list, rem
Considering the diversity of model capabilities, we started to add extension configuration in version `0.147.8`, with the following rules:

```shell
id=displayName<maxToken:vision:fc:file>
id=displayName<maxToken:vision:reasoning:fc:file>
```

The first value in angle brackets is designated as the `maxToken` for this model. The second value and beyond are the model's extension capabilities, separated by colons `:`, and the order is not important.
Expand All @@ -39,12 +39,14 @@ Examples are as follows:
- `chatglm-6b=ChatGLM 6B<4096>`: ChatGLM 6B, maximum context of 4k, no advanced capabilities;
- `spark-v3.5=讯飞星火 v3.5<8192:fc>`: Xunfei Spark 3.5 model, maximum context of 8k, supports Function Call;
- `gemini-1.5-flash-latest=Gemini 1.5 Flash<16000:vision>`: Google Vision model, maximum context of 16k, supports image recognition;
- `o3-mini=OpenAI o3-mini<200000:reasoning:fc>`: OpenAI o3-mini model, maximum context of 200k, supports reasoning and Function Call;
- `gpt-4-all=ChatGPT Plus<128000:fc:vision:file>`, hacked version of ChatGPT Plus web, context of 128k, supports image recognition, Function Call, file upload.

Currently supported extension capabilities are:

| --- | Description |
| -------- | -------------------------------------------------------- |
| `fc` | Function Calling |
| `vision` | Image Recognition |
| `file` | File Upload (a bit hacky, not recommended for daily use) |
| --- | Description |
| ----------- | -------------------------------------------------------- |
| `fc` | Function Calling |
| `vision` | Image Recognition |
| `reasoning` | Support Reasoning |
| `file` | File Upload (a bit hacky, not recommended for daily use) |
16 changes: 9 additions & 7 deletions docs/self-hosting/advanced/model-list.zh-CN.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Mr.🆖 AI 支持在部署时自定义模型列表,详情请参考 [模型提
你可以使用 `+` 增加一个模型,使用 `-` 来隐藏一个模型,使用 `模型名=展示名<扩展配置>` 来自定义模型的展示名,用英文逗号隔开。通过 `<>` 来添加扩展配置。基本语法如下:

```text
id=displayName<maxToken:vision:fc:file>,model2,model3
id=displayName<maxToken:vision:reasoning:fc:file>,model2,model3
```

例如: `+qwen-7b-chat,+glm-6b,-gpt-3.5-turbo,gpt-4-0125-preview=gpt-4-turbo`
Expand All @@ -28,7 +28,7 @@ id=displayName<maxToken:vision:fc:file>,model2,model3
考虑到模型的能力多样性,我们在 `0.147.8` 版本开始增加扩展性配置,它的规则如下:

```shell
id=displayName<maxToken:vision:fc:file>
id=displayName<maxToken:vision:reasoning:fc:file>
```

尖括号第一个值约定为这个模型的 `maxToken` 。第二个及以后作为模型的扩展能力,能力与能力之间用冒号 `:` 作为分隔符,顺序不重要。
Expand All @@ -38,12 +38,14 @@ id=displayName<maxToken:vision:fc:file>
- `chatglm-6b=ChatGLM 6B<4096>`:ChatGLM 6B,最大上下文 4k,没有高阶能力;
- `spark-v3.5=讯飞星火 v3.5<8192:fc>`:讯飞星火 3.5 模型,最大上下文 8k,支持 Function Call;
- `gemini-1.5-flash-latest=Gemini 1.5 Flash<16000:vision>`:Google 视觉模型,最大上下文 16k,支持图像识别;
- `o3-mini=OpenAI o3-mini<200000:reasoning:fc>`:OpenAI o3-mini 模型,最大上下文 200k,支持推理及 Function Call;
- `gpt-4-all=ChatGPT Plus<128000:fc:vision:file>`,hack 的 ChatGPT Plus 网页版,上下 128k ,支持图像识别、Function Call、文件上传

目前支持的扩展能力有:

| --- | 描述 |
| -------- | ---------------------- |
| `fc` | 函数调用(function calling) |
| `vision` | 视觉识别 |
| `file` | 文件上传(比较 hack,不建议日常使用) |
| --- | 描述 |
| ----------- | ------------------------------------- |
| `fc` | 函数调用(function calling) |
| `vision` | 视觉识别 |
| `reasoning` | 支持推理 |
| `file` | 文件上传(比较 hack,不建议日常使用) |
122 changes: 122 additions & 0 deletions src/services/aiModel/server.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
import { describe, expect, it, vi } from 'vitest';

import { lambdaClient } from '@/libs/trpc/client';
import { AiProviderModelListItem } from '@/types/aiModel';

import { ServerService } from './server';

vi.mock('@/libs/trpc/client', () => ({
lambdaClient: {
aiModel: {
createAiModel: { mutate: vi.fn() },
getAiProviderModelList: { query: vi.fn() },
getAiModelById: { query: vi.fn() },
toggleModelEnabled: { mutate: vi.fn() },
updateAiModel: { mutate: vi.fn() },
batchUpdateAiModels: { mutate: vi.fn() },
batchToggleAiModels: { mutate: vi.fn() },
clearModelsByProvider: { mutate: vi.fn() },
clearRemoteModels: { mutate: vi.fn() },
updateAiModelOrder: { mutate: vi.fn() },
removeAiModel: { mutate: vi.fn() },
},
},
}));

describe('ServerService', () => {
const service = new ServerService();

it('should create AI model', async () => {
const params = {
id: 'test-id',
providerId: 'test-provider',
displayName: 'Test Model',
};
await service.createAiModel(params);
expect(vi.mocked(lambdaClient.aiModel.createAiModel.mutate)).toHaveBeenCalledWith(params);
});

it('should get AI provider model list', async () => {
await service.getAiProviderModelList('123');
expect(vi.mocked(lambdaClient.aiModel.getAiProviderModelList.query)).toHaveBeenCalledWith({
id: '123',
});
});

it('should get AI model by id', async () => {
await service.getAiModelById('123');
expect(vi.mocked(lambdaClient.aiModel.getAiModelById.query)).toHaveBeenCalledWith({
id: '123',
});
});

it('should toggle model enabled', async () => {
const params = { id: '123', providerId: 'test', enabled: true };
await service.toggleModelEnabled(params);
expect(vi.mocked(lambdaClient.aiModel.toggleModelEnabled.mutate)).toHaveBeenCalledWith(params);
});

it('should update AI model', async () => {
const value = { contextWindowTokens: 4000, displayName: 'Updated Model' };
await service.updateAiModel('123', 'openai', value);
expect(vi.mocked(lambdaClient.aiModel.updateAiModel.mutate)).toHaveBeenCalledWith({
id: '123',
providerId: 'openai',
value,
});
});

it('should batch update AI models', async () => {
const models: AiProviderModelListItem[] = [
{
id: '123',
enabled: true,
type: 'chat',
},
];
await service.batchUpdateAiModels('provider1', models);
expect(vi.mocked(lambdaClient.aiModel.batchUpdateAiModels.mutate)).toHaveBeenCalledWith({
id: 'provider1',
models,
});
});

it('should batch toggle AI models', async () => {
const models = ['123', '456'];
await service.batchToggleAiModels('provider1', models, true);
expect(vi.mocked(lambdaClient.aiModel.batchToggleAiModels.mutate)).toHaveBeenCalledWith({
id: 'provider1',
models,
enabled: true,
});
});

it('should clear models by provider', async () => {
await service.clearModelsByProvider('provider1');
expect(vi.mocked(lambdaClient.aiModel.clearModelsByProvider.mutate)).toHaveBeenCalledWith({
providerId: 'provider1',
});
});

it('should clear remote models', async () => {
await service.clearRemoteModels('provider1');
expect(vi.mocked(lambdaClient.aiModel.clearRemoteModels.mutate)).toHaveBeenCalledWith({
providerId: 'provider1',
});
});

it('should update AI model order', async () => {
const items = [{ id: '123', sort: 1 }];
await service.updateAiModelOrder('provider1', items);
expect(vi.mocked(lambdaClient.aiModel.updateAiModelOrder.mutate)).toHaveBeenCalledWith({
providerId: 'provider1',
sortMap: items,
});
});

it('should delete AI model', async () => {
const params = { id: '123', providerId: 'openai' };
await service.deleteAiModel(params);
expect(vi.mocked(lambdaClient.aiModel.removeAiModel.mutate)).toHaveBeenCalledWith(params);
});
});

0 comments on commit 3b4df8e

Please sign in to comment.