Support for speculative decoding #689
BowenBao
started this conversation in
New features / APIs
Replies: 1 comment 1 reply
-
|
Hi @BowenBao, yes we do have speculative decoding on our roadmap but do not have a definite schedule for that yet |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Thank you @natke, we are happy to contribute an initial version :) |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi team, is there plan to support speculative decoding?
cc @spandantiwari-amd
Beta Was this translation helpful? Give feedback.
All reactions