Regrouping method for Japanese

[dgoryeo]

Hi @jianfch , thanks for so many new post processing features. In your view which regrouping method or custom combination might be best for Japanese language, considering the new capabilites?
Thanks

[jianfch]

I'm not familiar enough with Japanese to come up with a good combination.
But the new changes to include_lock might be useful at preventing splits before or/and after certain Japanese characters.

result.lock(endswith='よ', left=True, right=False).split_by_length(20, include_lock=True)

[dgoryeo]

Wow, there are so many features now baked in stable-ts that we probably need a YouTube guide or something. Would it be an idea to pull together some fundraising / donations to assist with creating more advanced howto material or videos? Just a thought.

…

On Tue, 10 Dec 2024, 20:02 jian, ***@***.***> wrote: I'm not familiar enough with Japanese to come up with a good combination. But the new changes to include_lock might be useful at preventing splits before or/and after certain Japanese characters. result.lock(endswith='よ', left=True, right=False).split_by_length(20, include_lock=True) — Reply to this email directly, view it on GitHub <#422 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3DK35W5R63AYJTWR6H4BRL2E5CEXAVCNFSM6AAAAABTJVI4QKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNJSGUZTGNY> . You are receiving this because you authored the thread.Message ID: ***@***.***>