runGeneration implementation and associated Dust app
#1302
Conversation
spolu
changed the title
runGeneration implementation and associated Dust apprunGeneration implementation and associated Dust app
spolu
changed the title
runGeneration implementation and associated Dust apprunGeneration implementation and associated Dust app
| @@ -33,6 +33,8 @@ import { | |||
| UserMessageType, | |||
| } from "@app/types/assistant/conversation"; | |||
|
|
|||
| import { ModelMessageType } from "../generation"; | |||
| @@ -507,7 +509,7 @@ export async function* runRetrieval( | |||
| id: 0, // dummy pending database insertion | |||
| dataSourceId: d.data_source_id, | |||
| documentId: d.document_id, | |||
| reference: "", | |||
| reference: new_id().slice(0, 3), | |||
There was a problem hiding this comment.
NIT: In the type this field is described as is: "Short random string so that the model can refer to the document."
As "reference" is not self-explanatory, what do you think about calling sId and have it sliced to 10 char?
There was a problem hiding this comment.
So we want it to be much shorter. The idea is for the model (with proper prompting) to generate references using eg [a32] or [ef2]. We want it short so that the generation is snappy and we can render it as a nice reference in the UI.
This is quite WIP of course.
front/lib/api/assistant/agent.ts
Outdated
| created: number; | ||
| configurationId: string; | ||
| messageId: string; | ||
| text: string; | ||
| }; | ||
|
|
||
| // Event sent once the message is completed and successful. | ||
| export type AgentGenerationSuccessEvent = { | ||
| type: "agent_generation_success"; | ||
| export type AgentSuccessEvent = { |
There was a problem hiding this comment.
NIT: Looking at the comment, should it be "AgentMessageSuccessEvent"? (asking as "once the message is completed and successful" means it is attached to the generation, right?)
Or will we use this event also after a successful action and it's the comment that needs to be updated?
There was a problem hiding this comment.
Happy to call it AgentMessageSuccessEvent 👍 The idea is for this one to be the final success event of the agent meaning that everything is ready and done. Since we have succeeded on AgentMessage, let's add Message in the event name here 👍
| messages: selected, | ||
| }); | ||
| } | ||
| import { GenerationTokensEvent } from "./generation"; |
| */ | ||
|
|
||
| export type ModelMessageType = { | ||
| role: "action" | "agent" | "user"; |
|
|
||
| const now = Date.now(); | ||
|
|
||
| // This is a bit aggressive but fuck it. |
|
Addressed || commented on your comments |
Fixes #1300
Dust app: https://dust.tt/w/78bda07b39/a/6a27050429
The Dust app is on purpose extremely simple. It is possible that we may want in the future a meta prompt to properly handle "references" but this is for future us.