Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements to renderPrompt #112

Open
TylerLeonhardt opened this issue Oct 16, 2024 · 3 comments
Open

Improvements to renderPrompt #112

TylerLeonhardt opened this issue Oct 16, 2024 · 3 comments
Assignees
Labels
feature-request Request for new features or functionality

Comments

@TylerLeonhardt
Copy link
Member

renderPrompt uses an IChatEndpointInfo:

export interface IChatEndpointInfo {
    /**
     * The maximum number of tokens allowed in the model prompt.
     */
    readonly modelMaxPromptTokens: number;
}

but in VS Code API when we get a model, it looks like this:

	/**
	 * Represents a language model for making chat requests.
	 *
	 * @see {@link lm.selectChatModels}
	 */
	export interface LanguageModelChat {

		// ...

		/**
		 * The maximum number of tokens that can be sent to the model in a single request.
		 */
		readonly maxInputTokens: number;

		// ...
        }

So naturally in several places I have to make this silly object:

{ modelMaxPromptTokens: model.maxInputTokens }

and also pass in model for:

renderPrompt<P extends BasePromptElementProps>(endpoint: IChatEndpointInfo, ctor: PromptElementCtor<P, any>, props: P, model: LanguageModelChat, token: CancellationToken)

so I pass in the same information twice... and I have to massage things to work but we own both APIs.

I think this should be cleaned up to take in only a LanguageModelChat.

This also applies to PromptRenderer.

@connor4312
Copy link
Member

@joyceerhl @roblourens I think it makes sense to tweak this to match VS Code's API (such that interfaces are just assignable in most cases) now that it's all finalized. Any reason not to?

@roblourens
Copy link
Member

Sounds good

@joyceerhl
Copy link
Contributor

SGTM

@connor4312 connor4312 added the feature-request Request for new features or functionality label Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-request Request for new features or functionality
Projects
None yet
Development

No branches or pull requests

4 participants