Remove iconv for decision search highlighting
#1766
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Unfortunately, by using `iconv` there is a greater chance that the
resulting (converted) text is either short or longer. For example,
by using the euro symbol (€) we artifically increase the length of
the texts we are comparing:
```php
iconv('UTF-8', 'ASCII//TRANSLIT//IGNORE', '€');
// 'EUR'
```
This is problematic, as it will result in incorrectly alignments
of `<mark>`. While this can be mitigated by carefully calculating
offsets for the offsets this quickly makes it more difficult to
keep maintaining this functionality. Especially when there need
to be more of these exceptions.
Only using the transliterator with `Any-Latin; Latin-ASCII` seems
to preserve the length of the comparing elements and allow for
searching accented/special characters. There are characters that
are not part of/exist in `Latin-ASCII`, however, these characters
are probably never used in the setting of the association.
|
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Unfortunately, by using
iconvthere is a greater chance that the resulting (converted) text is either short or longer. For example, by using the euro symbol (€) we artifically increase the length of the texts we are comparing:This is problematic, as it will result in incorrectly alignments of
<mark>. While this can be mitigated by carefully calculating offsets for the offsets this quickly makes it more difficult to keep maintaining this functionality. Especially when there need to be more of these exceptions.Only using the transliterator with
Any-Latin; Latin-ASCIIseems to preserve the length of the comparing elements and allow for searching accented/special characters. There are characters that are not part of/exist inLatin-ASCII, however, these characters are probably never used in the setting of the association.This is a bug fix for GH-1764.