Replies: 1 comment
-
Since v0.4.0, the pipeline can output logprobs and token ids. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
想集成lmdeploy做推理,用于rlhf过程,这个过程需要得到一次generate的output ids,但是调用lmdeploy的pipeline,只能得到decode后的output str,请问有什么推荐的解决办法吗?
vllm有generate接口,可以一次处理一个batch的prompts,直接得到output_ids,lmdeploy有没有类似的接口呢?
Beta Was this translation helpful? Give feedback.
All reactions