Kyujitai / Shinjitai Text preprocessors

[Casheeew]

related: #620, #1356

This PR adds a Kyujitai (旧字体) to Shinjitai (新字体) text preprocessor, which is useful when reading older texts.

Based on https://github.com/DrTurnon/kyujipy/blob/master/kyujipy/basic_converter.py

This PR does not include transformations caused by the 同音による書き換え reform
it does not include 俗字, 別体, 誤字 or other uncommon forms/variants.

[Kuuuube]

Mentioned in discord that this probably shouldnt use regex but after testing it looks like this is the right way to go. Any other way I could think of handling this benchmarked much slower. Probably hitting a sweet spot in browser optimization there for the number of possible replaces that are required here.

[djahandarie]

In the case there is a direct match on the kyuujitai prior to conversion, it shows that first, right?

[Casheeew]

In the case there is a direct match on the kyuujitai prior to conversion, it shows that first, right?

Yes, thats right. That is true for preprocessors in general.
(This PR is currently waiting for @Lyroxide to process more data and move the entire kyuji-shinji converter into a separate library)

[codspeed-hq]

CodSpeed Performance Report

Merging #1357 will not alter performance

_{Comparing Casheeew:shinji-preprocessor (ccd0225) with master (6496b68)}

Summary

✅ 5 untouched benchmarks