Align translated_batch and result.segments

[Louisfrn]

Hello, I’m facing an issue with the following code:

text_batch = get_text_batch(result.segments)
translated_batch = translates(translate_to=translate_to, text_batch=text_batch)
result.segments = replace_text_batch(segments=result.segments, translated_batch=translated_batch)

def replace_text_batch(segments, translated_batch):
    if len(segments) != len(translated_batch):
        raise ValueError("Le nombre de segments et de textes traduits doit être identique.")
    
    for i, segment in enumerate(segments):
        segment.text = translated_batch[i]
    
    return segments

This code takes the text content of result.segments, sends it to a Llama 2 model for translation into the desired language, and then replaces the text in each segment with the translated text using the replace_text_batch function.

The replacement works as expected: result.segments correctly contains the translated text for each segment. However, when I try to apply subsequent operations on result, such as:

result.split_by_length(max_words=10)
result.split_by_duration(max_dur=0.85)
result.to_srt_vtt(
    filepath=srt_path,
    segment_level=True,  # Include segment-level timestamps
    word_level=True,     # Include word-level timestamps
    min_dur=0.2,         # Minimum duration for segments/words
    strip=True           # Remove spaces around text
)

these operations fail. It seems that my replace_text_batch function might be removing or modifying some attributes of result or result.segments, which prevents methods like split_by_length, split_by_duration, and to_srt_vtt from functioning properly.

Request for Help

Does anyone know why this might be happening? I suspect the replace_text_batch function might inadvertently delete or alter some required attributes of result or its segments. Any guidance on how to debug or fix this would be appreciated.

[jianfch]

Which version of Stable-ts are you using?

This line should raise an AttributeError in the latest version:

        segment.text = translated_batch[i]

This is because text is a property of the all the words in the segment for results with word-timestamps.

stable-ts/stable_whisper/result.py

Lines 440 to 444 in c176ecd

	@property
	def text(self) -> str:
	if self.has_words:
	return ''.join(word.word for word in self.words)
	return self._default_text

You have to replace the individual words of the segment in order to change the text of the segment itself:

def replace_text_batch(segments, translated_batch):
    if len(segments) != len(translated_batch):
        raise ValueError("Le nombre de segments et de textes traduits doit être identique.")

    for i, segment in enumerate(segments):
        translated_words = translated_batch[i].split(' ')
        assert len(segment.words) == len(translated_words)
        for word, new_word in zip(segment, translated_words):
            word.word = new_word
    
    return segments

[Louisfrn]

Thank you !! Using your guidance, I updated my function to handle apostrophes and extra spaces in words, leading to the solution below:

def replace_text_batch(segments, translated_batch):
    if len(segments) != len(translated_batch):
        raise ValueError("Le nombre de segments et de textes traduits doit être identique.")
    
    for i, segment in enumerate(segments):
        translated_words = translated_batch[i].replace("'", "-'").split("-") # handling apostrophes in segments
        translated_words = [word for item in translated_words for word in item.split()]
                
        for word, new_word in zip(segment.words, translated_words):
            word.word = " " + new_word
    
    return segments

This implementation works perfectly with Stable-ts Version: 2.17.5 , installed using pip install -U stable-ts[fw], (I am using large-v3-turbo model).
Thank you for maintaining such a robust library!