Support RTL/bidi text, some font and harfbuzz fixes #309

@pkb

Skip elements among siblings that are not list items. By @pkb from buggins/coolreader#105

No logical changes: just tab and indentation cleanups, and comments.

We'll need more flags for char classification. HyphMan hyphenate() (used by some other parts of the code) then needs to be tweaked so it can work on a table of lUInt8, as well as on our upgraded table of lUInt16 flags.

Use Freetype embolden API instead of LVFontBoldTransform to make fake bold (for fonts that do not provide a bold face). This allows Harfbuzz to work with them (even if they won't look as nice as if they came with a real bold font). (LVFontBoldTransform is a wrapper that enlarge glyphs and advances, so killing any Harfbuzz measurement and glyph replacements.) See comments for more details.

Even if the original text is italic and/or bold, when requesting the fallback font, we always got the regular unbold variant - so part of a sentence or word would jump out as strange. With this and previous commit (Freetype embolden), we'll get many things nicer when using our FreeSerif (which has no italic nor bold variant) as the fallback font. Also loosen fonts charset detection by no more looking for "azAZ09" in a font (NotoSansMyanmar.ttf, for example, comes without them and could then not even be used as a fallback font). Also switch symbol fonts to the FT_ENCODING_MS_SYMBOL charmap immediately on load - otherwise Harfbuzz would not see/use the symbol glyphs.

measureText(): accepts an added "hints" parameter, thru which we can pass text direction and additional hints for Harfbuzz (begin/end of paragraph for now). For freetype and harbuzz light, draw chars in the reverse order when direction is RTL: this allows individual RTL words to be drawn correctly, even when not using Harfbuzz (and to witness the additional Harfbuzz magic when switching to it). DrawTextString(): returns advance, so we can draw subsegments of the text with the fallback font, instead of only individual chars are previously. This is needed to get correct shaping with Harfbuzz with the fallback font. Harbuzz, in both measureText() and DrawTextString(): Don't use filterChar() with the main font, as the not-found chars might be found in the fallback font; no need to replace them thar early. Rework glyph/cluster/text walking and drawing to be more generic: should work with "one char > multiple glyphs" situations (we previously handled correctly only "multiple chars merged into a single glyph", like ligatures), and with RTL text. Also accumulate not-found glyphs so we can draw them as a single segment with the fallback font, with harfbuzz (instead of with Freetype previously): this is needed when using a latin font as the main font, and be able to see nice arabic drawn with the fallback font. Drop letterspacing when the detected script is cursive. Harfbuzz light: hbCalcCharWidth(): skip triplet when any of the 3 chars is not found, as it would mess up the result (and cached values) when some char is combined with mutiple rare diacritic marks (eg. Hebrew). This could mess the drawing of the whole text. Fallback to more robust Freetype measurement in such cases.

@pkb

Shouldn't have any impact on pure LTR text. Additions to the usual LTR processing: copyText(): detect if we have RTL chars (arabic, hebrew, unicode bidi chars...) in a paragraph text, and use fribidi only when we have some (fribidi processing is quite expensive, so avoid it when not needed). When we do, compute bidi levels, to be used for visual reordering in addLine(). measureText(): Split measuring on bidi level change (and also on letter spacing change - upstream fix by @pkb). Provide fribidi segment direction and harfbuzz hints to font->measureText() for correct Harfbuzz measurements. Allow for ignoring some chars when measuring and drawing (for now: the set of unicode bidirectionality chars that can be used to tweak the unicode bidi algorithm, as some font have glyphs for them). When splitting the paragraph into lines, we continue to process chars in the logical order. Only in addLine(), with bidi paragraphs, we reorder the chars (and flags, widths...) from the logical line segment, into their visual order, and we make words from the result the usual way, with some additional flags (to help later with drawing, createXPointer() and getRect()). alignLine(): simple tweaks for RTL paragraphs: - put text-indent on the right - for justified text, align last (or single) line to the right More work needs to be done on the block rendering code for proper RTL layout (list items bullets on the right, table columns ordered from right to left...) lvrend.cpp: handle <bdi> and <bdo> elements, used to tweak the unicode bidirectional algorithm. Also add simple support for <q> by using a hardcoded single set of quotes.

To allow links/text to be selected and highlighted in bidi/RTL text. Text selection should work fine in pure RTL text, but may get bogus in bidi text when selection cross bidi levels (where a single selection suddenly becomes 2 as we are steping over a segment of the opposite direction, and moving back towards the previous segment...)

In EPUB, the <html> node of each embedded HTML file is not included in the generated single DOM. We now parse its attributes and forward them to be included as attribute of the followup <docFragment> element, so they are part of the DOM.

In-page footnotes are now vertically stacked in the order they are met by the reading direction in a same line.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support RTL/bidi text, some font and harfbuzz fixes #309

Support RTL/bidi text, some font and harfbuzz fixes #309

Commits on Sep 14, 2019