add pts to llm full response end frame #583
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aconchillo
commented
Oct 14, 2024
| while True: | ||
| try: | ||
| (word, timestamp) = await self._words_queue.get() | ||
| if word == "LLMFullResponseEndFrame" and timestamp == 0: | ||
| await self.push_frame(LLMFullResponseEndFrame()) | ||
| frame = LLMFullResponseEndFrame() | ||
| frame.pts = last_pts |
There was a problem hiding this comment.
Before this frame would be added after all the audio and since it didn't have a pts it would just be processed after the audio is spoken which should coincide with the text timestamps. But to make it more accurate we make sure it has a pts and its processed just after the text.
aconchillo
commented
Oct 14, 2024
| @@ -210,7 +210,6 @@ def _get_websocket(self): | |||
| async def _handle_interruption(self, frame: StartInterruptionFrame, direction: FrameDirection): | |||
| await super()._handle_interruption(frame, direction) | |||
| await self.stop_all_metrics() | |||
| await self.push_frame(LLMFullResponseEndFrame()) | |||
There was a problem hiding this comment.
I already mentioned it in the commit messages but this is not needed. The context aggregator interruption already handle interruption properly. We don't want to pollute the code with push_frame() otherwise it will become very hard to maintain.
aconchillo
commented
Oct 14, 2024
| # audio and video as well. | ||
| if isinstance(frame, TextFrame): | ||
| await self.push_frame(frame) | ||
|
|
There was a problem hiding this comment.
we already have _sink_frame_handler() so this function is not really needed.
aconchillo
force-pushed
the
aleix/add-pts-to-llm-full-response-end-frame
branch
from
22c4752 to
e6b3ca3
Compare
Interruptions are already handled by context aggregators.
aconchillo
force-pushed
the
aleix/add-pts-to-llm-full-response-end-frame
branch
from
e6b3ca3 to
7bbaf4d
Compare
markbackman
approved these changes
Oct 15, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.