Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Context manager improvement #285

Open
2 tasks done
evison opened this issue Oct 26, 2024 · 0 comments
Open
2 tasks done

[Feature] Context manager improvement #285

evison opened this issue Oct 26, 2024 · 0 comments
Labels
feature request New feature or request

Comments

@evison
Copy link
Collaborator

evison commented Oct 26, 2024 •
edited by dongyuanjushi
Loading

Checked other resources

The feature, motivation and pitch

Current context management will interrupt the current generated tokens and save that into disks. A better way is to save context using RAM or GPU memory, which is required to be implemented.

Alternatives

No response

Additional context

No response
@evison evison added the feature request New feature or request label Oct 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature request New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@evison