Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates from koreader/crengine #299

Merged
merged 88 commits into from
Jun 12, 2021
Merged

Updates from koreader/crengine #299

merged 88 commits into from
Jun 12, 2021

Conversation

virxkane
Copy link
Collaborator

@virxkane virxkane commented Jun 11, 2021

Merged new stuff from koreader/crengine:

Thirdparty library/dependencies added:

  • zstd (for cache compression)
  • utf8proc (improvements to unicode support).

Other changes:

  • Android: optimized pixel format conversion from crengine BGRX to android RGBA. Conversion time was reduced by about 50%.
  • Use the correct font glyph cache item size & glyph image offset. Based on Allocation & alignment trickeries koreader/crengine#441
  • Fixed global-buffer-overflow.
  • Fixed coolreader/crengine/src/lvstring.cpp:127:42: runtime error: signed integer overflow: 1775760480 * 31 cannot be represented in type 'int'.
  • lvtinydom.cpp: bool hasInvisibleParent(ldomNode* node): null pointer dereference fixed.
  • Memory leaks fixed.

poire-z and others added 30 commits May 27, 2021 20:29
"<div>abc <![CDATA[d<e&>f]]> ghi</div>" will now add into
the DOM 3 text nodes: "abc " , "d<e&>f" and " ghi", as it's
supposed to happen in XML/XHTML.
(In HTML, browsers would not do any specific parsing, with
possibly strange results. We do it too in plain HTML.)
…aks & cleanups' by NiLuJe

* Mark getter functions as const
* Simplify LVGrayDrawBuf::Invert
* LVColorDrawBuf::Clear @ 32bpp: Cache the color value
Helps avoid noise on debug builds. (Because I sure hope an optimized build would have hoisted that).
…& cleanups' by NiLuJe

* Mark getter functions as const
* We don't need stbiw outside of lvimg
(Probably requires a recent stbiw, the one shipped w/ NanoSVG is
hilariously old. Harmless if unsupported, though).
And make it private & inlined, as only lvtinydom uses it.
* Move ldomPack/ldomUnpack to the CacheFile class, as that the only
  class that uses it.
* Which implies duplicating (zlib) unpack in PDBFile for pdbfmt...
* Lazy init ZSTD ressources, and keep 'em around as long as sensible.
* crengine: allow to set global cache compression type (none, zlib, zstd).
* crengine: allow use zlib cache compression even if zstd available
  (selected by setCacheCompressionType() global functions).
  In the future, can be implemented selection in the settings dialog.
* Android: add static library zstd to dependencies.
It turns out it's not just LVImg that wants inverted alpha: it's *all* CRe...
So, we'll be handling the pixel format conversion entirely in KOReader instead, as dealing with the alpha here is kind of hairy without basically rewiring all the things.

:(
…oid RGBA.

Conversion time was reduced by about 50%:
Nokia 8: 18 ms => 9 ms
x86 emu: 1.6 ms => 0.6 ms
Handle content as text, but don't stop just at any '<',
only at '</script>', as <script> may contain tags.
Not doing that might put a lot of javascript as
document text content. Note that <script> content
is "display: none" by default.
Upstream update of the German patterns (roughly 800 changes)
(from cr3gui/data/hyph/german_hyphen.pattern).
It was hardcoded'ly removed as part of 9214bfc when
implementing support for CSS width & height for images,
so they don't get in the way.
Removed the hardcoding so the original CoolReader
settings can work again.
We can still disable that from frontend code.
Increase nb of bitmaps from 2 to 3 lUInt32 as we're
going over 64 properties.
Compute image size according to CSS2 specs, ensuring
CSS width, height, min-width, min-height, max-width
and max-height.
Used in lvtextfm when drawing images, and when
estimating block widths in getRenderedWidths()
instead of the previous wrong computation.
Limitation: heights in % are not supported and ignored.
Ensure min-width, min-height, max-width (but not max-height)
on block elements, inline-block/table, block floats and
floating images (min-height in % not supported).
Ensure min-width on the table element itself (but not
supported on the table inner elements like cells and cols).
So frontends can know if the rendering or the number
of pages have changed (instead of relying of the
document height, which is not a strictly accurate
indicator of changes).
Ensure "the rules above are applied again" from the specs
when ensuring max/min-height, meaning some bits of the width
computations have to be ensured again.
Also handle <image> just as <img> in the few places we
did not yet: when handling floats.
poire-z and others added 25 commits June 9, 2021 11:54
Everything wrapped in #if MATHML_SUPPORT==1 / #endif,
mostly for documentation purpose, to make it explicite
a reader can skip it when not interested.

lvtinydom.cpp DOM writers: add flag when entering <math>,
so that MathMLHelper can complete and tweak the DOM as it
is being built.

lvrend.cpp: as most of the non trivial MathML elements
(mfrac, msub, mover, msqrt...) are handled as inline-table,
plug a few specific handler in the table rendering code,
and in getRenderedWidths() to estimate inline-block/table
elements min/max widths.
Also have setNodeStyle() call setMathMLElementNodeStyle()
on MathML elements to complete their styling.

lvtextfm.cpp: alignLine() position inline-block boxes
may call a specific method on MathML elements to tweak
their position and size to ensure vertical stretching
of MathML operators, so they adjust to the line box
vertical space, adjusting the line height and baseline.

lvrend.cpp renderFinalBlock() and lvtextfm.cpp: pass
LTEXT_MATH_TRANSFORM flag to have stretchy final
blocks drawn stretched.
Simple (and ugly) way to draw MathML stretchy operators,
for fences, arrows, ... to adjust to their neighbours.
A more correct rendering would require more changes
here and in lvtextfm.cpp, to use OT Math fonts glyph
variants and glyph construction features.
==536==ERROR: AddressSanitizer: global-buffer-overflow on address 0x555559aa5784 at pc 0x5555576f898a bp 0x7fffffffadb0 sp 0x7fffffffada0
READ of size 4 at 0x555559aa5784 thread T0
    00 0x5555576f8989 in LVFreeTypeFace::measureText(char32_t const*, int, unsigned short*, unsigned char*, int, char32_t, TextLangCfg*, int, bool, unsigned int, unsigned int) coolreader/crengine/src/lvfont/lvfreetypeface.cpp:1756
    01 0x5555570c6ab4 in ldomDocument::createXPointer(lvPoint, int, bool, ldomNode*) coolreader/crengine/src/lvtinydom.cpp:9059
    02 0x5555570c5fbe in ldomDocument::createXPointer(lvPoint, int, bool, ldomNode*) coolreader/crengine/src/lvtinydom.cpp:9014
    03 0x5555572fd8e3 in LVDocView::getNodeByPoint(lvPoint, bool) coolreader/crengine/src/lvdocview.cpp:2623
    04 0x555556e154cb in CR3View::mouseMoveEvent(QMouseEvent*) coolreader/cr3qt/src/cr3widget.cpp:999
    05 0x7ffff6c17947 in QWidget::event(QEvent*) (/usr/lib64/libQt5Widgets.so.5+0x1ad947)
    06 0x7ffff6bd092e in QApplicationPrivate::notify_helper(QObject*, QEvent*) (/usr/lib64/libQt5Widgets.so.5+0x16692e)
    07 0x7ffff6bd84cc in QApplication::notify(QObject*, QEvent*) (/usr/lib64/libQt5Widgets.so.5+0x16e4cc)
    08 0x7ffff611c657 in QCoreApplication::notifyInternal2(QObject*, QEvent*) (/usr/lib64/libQt5Core.so.5+0x2b1657)
    09 0x7ffff6bd7660 in QApplicationPrivate::sendMouseEvent(QWidget*, QMouseEvent*, QWidget*, QWidget*, QWidget**, QPointer<QWidget>&, bool, bool) (/usr/lib64/libQt5Widgets.so.5+0x16d660)
    10 0x7ffff6c33a3a  (/usr/lib64/libQt5Widgets.so.5+0x1c9a3a)
    11 0x7ffff6c36d66  (/usr/lib64/libQt5Widgets.so.5+0x1ccd66)
    12 0x7ffff6bd092e in QApplicationPrivate::notify_helper(QObject*, QEvent*) (/usr/lib64/libQt5Widgets.so.5+0x16692e)
    13 0x7ffff6bd81f7 in QApplication::notify(QObject*, QEvent*) (/usr/lib64/libQt5Widgets.so.5+0x16e1f7)
    14 0x7ffff611c657 in QCoreApplication::notifyInternal2(QObject*, QEvent*) (/usr/lib64/libQt5Core.so.5+0x2b1657)
    15 0x7ffff6533ce7 in QGuiApplicationPrivate::processMouseEvent(QWindowSystemInterfacePrivate::MouseEvent*) (/usr/lib64/libQt5Gui.so.5+0x125ce7)
    16 0x7ffff65352d4 in QGuiApplicationPrivate::processWindowSystemEvent(QWindowSystemInterfacePrivate::WindowSystemEvent*) (/usr/lib64/libQt5Gui.so.5+0x1272d4)
    17 0x7ffff650e56a in QWindowSystemInterface::sendWindowSystemEvents(QFlags<QEventLoop::ProcessEventsFlag>) (/usr/lib64/libQt5Gui.so.5+0x10056a)
    18 0x7fffee994d99  (/usr/lib64/libQt5XcbQpa.so.5+0x77d99)
    19 0x7ffff4eb8d66 in g_main_context_dispatch (/usr/lib64/libglib-2.0.so.0+0x58d66)
    20 0x7ffff4eb8fff  (/usr/lib64/libglib-2.0.so.0+0x58fff)
    21 0x7ffff4eb908e in g_main_context_iteration (/usr/lib64/libglib-2.0.so.0+0x5908e)
    22 0x7ffff617766c in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) (/usr/lib64/libQt5Core.so.5+0x30c66c)
    23 0x7ffff611b212 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) (/usr/lib64/libQt5Core.so.5+0x2b0212)
    24 0x7ffff61234cf in QCoreApplication::exec() (/usr/lib64/libQt5Core.so.5+0x2b84cf)
    25 0x555556dc571f in main coolreader/cr3qt/src/main.cpp:212
    26 0x7ffff503fe39 in __libc_start_main (/lib64/libc.so.6+0x23e39)
    27 0x555556d937d9 in _start (coolreader-debug-build/cr3qt/cr3+0x183f7d9)

0x555559aa5784 is located 0 bytes to the right of global variable 'empty_str_32' defined in 'coolreader/crengine/src/lvstring.cpp:59:16' (0x555559aa5780) of size 4
0x555559aa5784 is located 60 bytes to the left of global variable 'empty_chunk_32' defined in 'coolreader/crengine/src/lvstring.cpp:60:26' (0x555559aa57c0) of size 24
SUMMARY: AddressSanitizer: global-buffer-overflow coolreader/crengine/src/lvfont/lvfreetypeface.cpp:1756 in LVFreeTypeFace::measureText(char32_t const*, int, unsigned short*, unsigned char*, int, char32_t, TextLangCfg*, int, bool, unsigned int, unsigned int)
Shadow bytes around the buggy address:
  0x0aab2b34caa0: 00 f9 f9 f9 f9 f9 f9 f9 01 f9 f9 f9 f9 f9 f9 f9
  0x0aab2b34cab0: 01 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9
  0x0aab2b34cac0: 00 00 00 00 01 f9 f9 f9 f9 f9 f9 f9 00 00 00 00
  0x0aab2b34cad0: 01 f9 f9 f9 f9 f9 f9 f9 00 00 00 f9 f9 f9 f9 f9
  0x0aab2b34cae0: 02 f9 f9 f9 f9 f9 f9 f9 00 00 00 f9 f9 f9 f9 f9
=>0x0aab2b34caf0:[04]f9 f9 f9 f9 f9 f9 f9 00 00 00 f9 f9 f9 f9 f9
  0x0aab2b34cb00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0aab2b34cb10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0aab2b34cb20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0aab2b34cb30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0aab2b34cb40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
…ned integer overflow: 1775760480 * 31 cannot be represented in type 'int'
Extend 2504a40 to keep doing it when multiple such
initial quotation marks/dashes separated by spaces.
New patterns based on dehyph-exptl 0.7.
from cr3gui/data/hyph/german_hyphen.pattern
from 38d3aec
Fix a TeX font name, and avoid some clang warning.
(And fix 2 other compiler warnings.)
DrawStretchedGlyph(): fix synthesized bold italic drawing
similarly to c24628d with regular glyph drawing.
getCharProp() is nearly no longer used when using libunibreak,
but it is still used by lStr_findWordBounds() to detect words
to try to hyphenate.
It knowing only about latin, greek and cyrillic letters made
hyphenation not detect any candidate word in Armenian and
Georgian text.
So, fallback to use utf8proc to detect letters in any other
alphabet so hyphenation (including soft-hyphens) can work.
Also detect RTL chars, as we don't want to hyphenate on
soft-hyphens in Arabic or Hebrew, as the text rendering code
currently can't draw hyphens on the left.
* Android: add static library utf8proc to dependencies.
In case footnotes' <body> don't come with a proper <title>
that would have ensured this page break.
Rework 89b0650: ignore the space even if followed by
another text node; just don't ignore it for the first
line where it could have some purpose (eg. to add some
indentation).
Direct leak of 64 byte(s) in 4 object(s) allocated from:
    00 0x7fe9ec7f4087 in operator new(unsigned long) (/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/libasan.so.6+0xb0087)
    01 0x55b88bf8ac27 in OpcPart::readRelations() coolreader/crengine/src/lvopc.cpp:73
    02 0x55b88bf89a55 in OpcPart::getRelatedPartName(char32_t const*, lString32) coolreader/crengine/src/lvopc.cpp:33
    03 0x55b88bc814b3 in fb3ImportContext::openBook() coolreader/crengine/src/fb3fmt.cpp:142
    04 0x55b88bc7f3b0 in ImportFb3Document(LVFastRef<LVStream>, ldomDocument*, LVDocViewCallback*, CacheLoadingCallback*) coolreader/crengine/src/fb3fmt.cpp:100
    05 0x55b88b98f789 in LVDocView::loadDocumentInt(LVFastRef<LVStream>, bool) coolreader/crengine/src/lvdocview.cpp:4445
    06 0x55b88b97ec85 in LVDocView::LoadDocument(char32_t const*, bool) coolreader/crengine/src/lvdocview.cpp:4127
    07 0x55b88b452554 in CR3View::loadDocument(QString) coolreader/cr3qt/src/cr3widget.cpp:475
    08 0x55b88b427628 in RecentBooksDlg::openBook(int) coolreader/cr3qt/src/recentdlg.cpp:131
    09 0x55b88b4262c0 in RecentBooksDlg::on_buttonBox_accepted() coolreader/cr3qt/src/recentdlg.cpp:112
    10 0x55b88b5fac7d in RecentBooksDlg::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) coolreader-debug-build/cr3qt/src/moc_recentdlg.cpp:100
    11 0x55b88b5fb38d in RecentBooksDlg::qt_metacall(QMetaObject::Call, int, void**) coolreader-debug-build/cr3qt/src/moc_recentdlg.cpp:136
    12 0x7fe9eb28968f  (/usr/lib64/libQt5Core.so.5+0x2e768f)
    13 0x7fe9ebec1d2d  (/usr/lib64/libQt5Widgets.so.5+0x320d2d)

Indirect leak of 512 byte(s) in 4 object(s) allocated from:
    00 0x7fe9ec7f41f7 in operator new[](unsigned long) (/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/libasan.so.6+0xb01f7)
    01 0x55b88bbb24e8 in LVHashTable<lString32, lString32>::LVHashTable(int) coolreader/crengine/src/../include/lvhashtable.h:107
    02 0x55b88bf8ac59 in OpcPart::readRelations() coolreader/crengine/src/lvopc.cpp:73
    03 0x55b88bf89a55 in OpcPart::getRelatedPartName(char32_t const*, lString32) coolreader/crengine/src/lvopc.cpp:33
    04 0x55b88bc814b3 in fb3ImportContext::openBook() coolreader/crengine/src/fb3fmt.cpp:142
    05 0x55b88bc7f3b0 in ImportFb3Document(LVFastRef<LVStream>, ldomDocument*, LVDocViewCallback*, CacheLoadingCallback*) coolreader/crengine/src/fb3fmt.cpp:100
    06 0x55b88b98f789 in LVDocView::loadDocumentInt(LVFastRef<LVStream>, bool) coolreader/crengine/src/lvdocview.cpp:4445
    07 0x55b88b97ec85 in LVDocView::LoadDocument(char32_t const*, bool) coolreader/crengine/src/lvdocview.cpp:4127
    08 0x55b88b452554 in CR3View::loadDocument(QString) coolreader/cr3qt/src/cr3widget.cpp:475
    09 0x55b88b427628 in RecentBooksDlg::openBook(int) coolreader/cr3qt/src/recentdlg.cpp:131
    10 0x55b88b4262c0 in RecentBooksDlg::on_buttonBox_accepted() coolreader/cr3qt/src/recentdlg.cpp:112
    11 0x55b88b5fac7d in RecentBooksDlg::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) coolreader-debug-build/cr3qt/src/moc_recentdlg.cpp:100
    12 0x55b88b5fb38d in RecentBooksDlg::qt_metacall(QMetaObject::Call, int, void**) coolreader-debug-build/cr3qt/src/moc_recentdlg.cpp:136
    13 0x7fe9eb28968f  (/usr/lib64/libQt5Core.so.5+0x2e768f)
    14 0x7fe9ebec1d2d  (/usr/lib64/libQt5Widgets.so.5+0x320d2d)

Indirect leak of 96 byte(s) in 4 object(s) allocated from:
    00 0x7fe9ec7f4087 in operator new(unsigned long) (/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/libasan.so.6+0xb0087)
    01 0x55b88bbb2273 in LVHashTable<lString32, lString32>::set(lString32 const&, lString32) coolreader/crengine/src/../include/lvhashtable.h:174
    02 0x55b88bf8aec9 in OpcPart::readRelations() coolreader/crengine/src/lvopc.cpp:77
    03 0x55b88bf89a55 in OpcPart::getRelatedPartName(char32_t const*, lString32) coolreader/crengine/src/lvopc.cpp:33
    04 0x55b88bc814b3 in fb3ImportContext::openBook() coolreader/crengine/src/fb3fmt.cpp:142
    05 0x55b88bc7f3b0 in ImportFb3Document(LVFastRef<LVStream>, ldomDocument*, LVDocViewCallback*, CacheLoadingCallback*) coolreader/crengine/src/fb3fmt.cpp:100
    06 0x55b88b98f789 in LVDocView::loadDocumentInt(LVFastRef<LVStream>, bool) coolreader/crengine/src/lvdocview.cpp:4445
    07 0x55b88b97ec85 in LVDocView::LoadDocument(char32_t const*, bool) coolreader/crengine/src/lvdocview.cpp:4127
    08 0x55b88b452554 in CR3View::loadDocument(QString) coolreader/cr3qt/src/cr3widget.cpp:475
    09 0x55b88b427628 in RecentBooksDlg::openBook(int) coolreader/cr3qt/src/recentdlg.cpp:131
    10 0x55b88b4262c0 in RecentBooksDlg::on_buttonBox_accepted() coolreader/cr3qt/src/recentdlg.cpp:112
    11 0x55b88b5fac7d in RecentBooksDlg::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) coolreader-debug-build/cr3qt/src/moc_recentdlg.cpp:100
    12 0x55b88b5fb38d in RecentBooksDlg::qt_metacall(QMetaObject::Call, int, void**) coolreader-debug-build/cr3qt/src/moc_recentdlg.cpp:136
    13 0x7fe9eb28968f  (/usr/lib64/libQt5Core.so.5+0x2e768f)
    14 0x7fe9ebec1d2d  (/usr/lib64/libQt5Widgets.so.5+0x320d2d)
Direct leak of 64 byte(s) in 1 object(s) allocated from:
    00 0x7ff64a52c087 in operator new(unsigned long) (/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/libasan.so.6+0xb0087)
    01 0x564d4a6375f6 in tinyNodeCollection::allocTinyElement(ldomNode*, unsigned short, unsigned short) coolreader/crengine/src/lvtinydom.cpp:16430
    02 0x564d4a5573aa in ldomDocument::ldomDocument() coolreader/crengine/src/lvtinydom.cpp:4136
    03 0x564d4a828137 in LVDocView::createEmptyDocument() coolreader/crengine/src/lvdocview.cpp:4797
    04 0x564d4a81461b in LVDocView::loadDocumentInt(LVFastRef<LVStream>, bool) coolreader/crengine/src/lvdocview.cpp:4438
    05 0x564d4a804bc5 in LVDocView::LoadDocument(char32_t const*, bool) coolreader/crengine/src/lvdocview.cpp:4127
    06 0x564d4a2d8554 in CR3View::loadDocument(QString) coolreader/cr3qt/src/cr3widget.cpp:475
    07 0x564d4a2d7d24 in CR3View::loadLastDocument() coolreader/cr3qt/src/cr3widget.cpp:468
    08 0x564d4a33e8a8 in MainWindow::showEvent(QShowEvent*) coolreader/cr3qt/src/mainwindow.cpp:467
    09 0x7ff649a86947 in QWidget::event(QEvent*) (/usr/lib64/libQt5Widgets.so.5+0x1ad947)
    10 0x6150000223ff  (<unknown module>)
Direct leak of 128 byte(s) in 2 object(s) allocated from:
    0 0x7f084fbb2087 in operator new(unsigned long) (/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/libasan.so.6+0xb0087)
    1 0x563f6643c10f in ldomNode::modify() coolreader/crengine/src/lvtinydom.cpp:19148
    2 0x563f66430684 in ldomNode::removeChild(unsigned int) coolreader/crengine/src/lvtinydom.cpp:18643
    3 0x563f6635598a in ldomNode::removeChildren(int, int) coolreader/crengine/src/lvtinydom.cpp:5845
    4 0x563f663567a4 in ldomNode::autoboxChildren(int, int, bool) coolreader/crengine/src/lvtinydom.cpp:5995
    5 0x563f6635d3b0 in ldomNode::initNodeRendMethod() coolreader/crengine/src/lvtinydom.cpp:6842
    6 0x563f663632aa in ldomElementWriter::onBodyExit() coolreader/crengine/src/lvtinydom.cpp:7815
    7 0x563f66364b37 in ldomElementWriter::~ldomElementWriter() coolreader/crengine/src/lvtinydom.cpp:7955
    8 0x563f663d87ca in ldomDocumentWriterFilter::popUpTo(ldomElementWriter*, unsigned short, int) coolreader/crengine/src/lvtinydom.cpp:13895
    9 0x563f663de026 in ldomDocumentWriterFilter::AutoOpenClosePop(int, unsigned short) coolreader/crengine/src/lvtinydom.cpp:14126
    10 0x563f663ed679 in ldomDocumentWriterFilter::OnTagClose(char32_t const*, char32_t const*, bool) coolreader/crengine/src/lvtinydom.cpp:14657
    11 0x563f6652b2e9 in LVXMLParser::Parse() coolreader/crengine/src/lvxml/lvxmlparser.cpp:278
    12 0x563f6653df6f in LVHTMLParser::Parse() coolreader/crengine/src/lvxml/lvhtmlparser.cpp:157
    13 0x563f66612696 in LVDocView::ParseDocument() coolreader/crengine/src/lvdocview.cpp:5023
    14 0x563f66604181 in LVDocView::loadDocumentInt(LVFastRef<LVStream>, bool) coolreader/crengine/src/lvdocview.cpp:4744
    15 0x563f665e2bf7 in LVDocView::LoadDocument(char32_t const*, bool) coolreader/crengine/src/lvdocview.cpp:4127
    16 0x563f660b6554 in CR3View::loadDocument(QString) coolreader/cr3qt/src/cr3widget.cpp:475
    17 0x563f66111b27 in MainWindow::on_actionOpen_triggered() coolreader/cr3qt/src/mainwindow.cpp:249
    18 0x563f66250407 in MainWindow::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) coolreader-debug-build/cr3qt/src/moc_mainwindow.cpp:253
    19 0x563f66250f69 in MainWindow::qt_metacall(QMetaObject::Call, int, void**) coolreader-debug-build/cr3qt/src/moc_mainwindow.cpp:295
    20 0x7f084e64768f  (/usr/lib64/libQt5Core.so.5+0x2e768f)
    21 0x7f084f0bdff1 in QAction::triggered(bool) (/usr/lib64/libQt5Widgets.so.5+0x15eff1)

Direct leak of 64 byte(s) in 1 object(s) allocated from:
    0 0x7f084fbb2087 in operator new(unsigned long) (/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/libasan.so.6+0xb0087)
    1 0x563f6643c10f in ldomNode::modify() coolreader/crengine/src/lvtinydom.cpp:19148
    2 0x563f6642e7e5 in ldomNode::insertChildElement(unsigned int, unsigned short, unsigned short) coolreader/crengine/src/lvtinydom.cpp:18534
    3 0x563f6635636f in ldomNode::autoboxChildren(int, int, bool) coolreader/crengine/src/lvtinydom.cpp:5974
    4 0x563f6635d3b0 in ldomNode::initNodeRendMethod() coolreader/crengine/src/lvtinydom.cpp:6842
    5 0x563f663632aa in ldomElementWriter::onBodyExit() coolreader/crengine/src/lvtinydom.cpp:7815
    6 0x563f66364b37 in ldomElementWriter::~ldomElementWriter() coolreader/crengine/src/lvtinydom.cpp:7955
    7 0x563f663d87ca in ldomDocumentWriterFilter::popUpTo(ldomElementWriter*, unsigned short, int) coolreader/crengine/src/lvtinydom.cpp:13895
    8 0x563f663de026 in ldomDocumentWriterFilter::AutoOpenClosePop(int, unsigned short) coolreader/crengine/src/lvtinydom.cpp:14126
    9 0x563f663ed679 in ldomDocumentWriterFilter::OnTagClose(char32_t const*, char32_t const*, bool) coolreader/crengine/src/lvtinydom.cpp:14657
    10 0x563f6652b2e9 in LVXMLParser::Parse() coolreader/crengine/src/lvxml/lvxmlparser.cpp:278
    11 0x563f6653df6f in LVHTMLParser::Parse() coolreader/crengine/src/lvxml/lvhtmlparser.cpp:157
    12 0x563f66612696 in LVDocView::ParseDocument() coolreader/crengine/src/lvdocview.cpp:5023
    13 0x563f66604181 in LVDocView::loadDocumentInt(LVFastRef<LVStream>, bool) coolreader/crengine/src/lvdocview.cpp:4744
    14 0x563f665e2bf7 in LVDocView::LoadDocument(char32_t const*, bool) coolreader/crengine/src/lvdocview.cpp:4127
    15 0x563f660b6554 in CR3View::loadDocument(QString) coolreader/cr3qt/src/cr3widget.cpp:475
    16 0x563f66111b27 in MainWindow::on_actionOpen_triggered() coolreader/cr3qt/src/mainwindow.cpp:249
    17 0x563f66250407 in MainWindow::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) coolreader-debug-build/cr3qt/src/moc_mainwindow.cpp:253
    18 0x563f66250f69 in MainWindow::qt_metacall(QMetaObject::Call, int, void**) coolreader-debug-build/cr3qt/src/moc_mainwindow.cpp:295
    19 0x7f084e64768f  (/usr/lib64/libQt5Core.so.5+0x2e768f)
    20 0x7f084f0bdff1 in QAction::triggered(bool) (/usr/lib64/libQt5Widgets.so.5+0x15eff1)

Direct leak of 64 byte(s) in 1 object(s) allocated from:
    0 0x7f084fbb2087 in operator new(unsigned long) (/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/libasan.so.6+0xb0087)
    1 0x563f66415628 in tinyNodeCollection::allocTinyElement(ldomNode*, unsigned short, unsigned short) coolreader/crengine/src/lvtinydom.cpp:16424
    2 0x563f6642eaa9 in ldomNode::insertChildElement(unsigned int, unsigned short, unsigned short) coolreader/crengine/src/lvtinydom.cpp:18538
    3 0x563f6635636f in ldomNode::autoboxChildren(int, int, bool) coolreader/crengine/src/lvtinydom.cpp:5974
    4 0x563f6635d3b0 in ldomNode::initNodeRendMethod() coolreader/crengine/src/lvtinydom.cpp:6842
    5 0x563f663632aa in ldomElementWriter::onBodyExit() coolreader/crengine/src/lvtinydom.cpp:7815
    6 0x563f66364b37 in ldomElementWriter::~ldomElementWriter() coolreader/crengine/src/lvtinydom.cpp:7955
    7 0x563f663d87ca in ldomDocumentWriterFilter::popUpTo(ldomElementWriter*, unsigned short, int) coolreader/crengine/src/lvtinydom.cpp:13895
    8 0x563f663de026 in ldomDocumentWriterFilter::AutoOpenClosePop(int, unsigned short) coolreader/crengine/src/lvtinydom.cpp:14126
    9 0x563f663ed679 in ldomDocumentWriterFilter::OnTagClose(char32_t const*, char32_t const*, bool) coolreader/crengine/src/lvtinydom.cpp:14657
    10 0x563f6652b2e9 in LVXMLParser::Parse() coolreader/crengine/src/lvxml/lvxmlparser.cpp:278
    11 0x563f6653df6f in LVHTMLParser::Parse() coolreader/crengine/src/lvxml/lvhtmlparser.cpp:157
    12 0x563f66612696 in LVDocView::ParseDocument() coolreader/crengine/src/lvdocview.cpp:5023
    13 0x563f66604181 in LVDocView::loadDocumentInt(LVFastRef<LVStream>, bool) coolreader/crengine/src/lvdocview.cpp:4744
    14 0x563f665e2bf7 in LVDocView::LoadDocument(char32_t const*, bool) coolreader/crengine/src/lvdocview.cpp:4127
    15 0x563f660b6554 in CR3View::loadDocument(QString) coolreader/cr3qt/src/cr3widget.cpp:475
    16 0x563f66111b27 in MainWindow::on_actionOpen_triggered() coolreader/cr3qt/src/mainwindow.cpp:249
    17 0x563f66250407 in MainWindow::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) coolreader-debug-build/cr3qt/src/moc_mainwindow.cpp:253
    18 0x563f66250f69 in MainWindow::qt_metacall(QMetaObject::Call, int, void**) coolreader-debug-build/cr3qt/src/moc_mainwindow.cpp:295
    19 0x7f084e64768f  (/usr/lib64/libQt5Core.so.5+0x2e768f)
    20 0x7f084f0bdff1 in QAction::triggered(bool) (/usr/lib64/libQt5Widgets.so.5+0x15eff1)

Direct leak of 144 byte(s) in 2 object(s) allocated from:
    0 0x7f63aca6a087 in operator new(unsigned long) (/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/libasan.so.6+0xb0087)
    1 0x55f5f438981a in CCRTable::LookupElem(ldomNode*, int, int) (coolreader-debug-build/cr3qt/cr3+0x1f6681a)
    2 0x55f5f4388cb3 in CCRTable::LookupElem(ldomNode*, int, int) (coolreader-debug-build/cr3qt/cr3+0x1f65cb3)
    3 0x55f5f43af7f9 in CCRTable::CCRTable(ldomNode*, int, bool, int, int, bool, bool, int, bool) coolreader/crengine/src/lvrend.cpp:2243
    4 0x55f5f4310333 in renderTable(LVRendPageContext&, ldomNode*, int, int, int, bool, int, int&, int, bool, bool, bool) coolreader/crengine/src/lvrend.cpp:2276
    5 0x55f5f433ff0c in renderBlockElementEnhanced(FlowState*, ldomNode*, int, int, unsigned int) coolreader/crengine/src/lvrend.cpp:7639
    6 0x55f5f4341c7b in renderBlockElementEnhanced(FlowState*, ldomNode*, int, int, unsigned int) coolreader/crengine/src/lvrend.cpp:7860
    7 0x55f5f43462bc in renderBlockElement(LVRendPageContext&, ldomNode*, int, int, int, int, int, int, int*, unsigned int) coolreader/crengine/src/lvrend.cpp:8393
    8 0x55f5f43465cb in renderBlockElement(LVRendPageContext&, ldomNode*, int, int, int, int, int, int, int*) coolreader/crengine/src/lvrend.cpp:8409
    9 0x55f5f42d05b0 in LVFormatter::measureText() coolreader/crengine/src/lvtextfm.cpp:2045
    10 0x55f5f42eeb05 in LVFormatter::processParagraph(int, int, bool) coolreader/crengine/src/lvtextfm.cpp:3809
    11 0x55f5f42f7062 in LVFormatter::splitParagraphs() coolreader/crengine/src/lvtextfm.cpp:4461
    12 0x55f5f42f7aad in LVFormatter::format() coolreader/crengine/src/lvtextfm.cpp:4508
    13 0x55f5f42aca7d in LFormattedText::Format(unsigned short, unsigned short, int, int, int, bool, BlockFloatFootprint*) coolreader/crengine/src/lvtextfm.cpp:4612
    14 0x55f5f4051aa8 in ldomNode::renderFinalBlock(LVRef<LFormattedText>&, RenderRectAccessor*, int, BlockFloatFootprint*) coolreader/crengine/src/lvtinydom.cpp:19074
    15 0x55f5f43432c8 in renderBlockElementEnhanced(FlowState*, ldomNode*, int, int, unsigned int) coolreader/crengine/src/lvrend.cpp:8167
    16 0x55f5f4341c7b in renderBlockElementEnhanced(FlowState*, ldomNode*, int, int, unsigned int) coolreader/crengine/src/lvrend.cpp:7860
    17 0x55f5f4341c7b in renderBlockElementEnhanced(FlowState*, ldomNode*, int, int, unsigned int) coolreader/crengine/src/lvrend.cpp:7860
    18 0x55f5f4341c7b in renderBlockElementEnhanced(FlowState*, ldomNode*, int, int, unsigned int) coolreader/crengine/src/lvrend.cpp:7860
    19 0x55f5f43462bc in renderBlockElement(LVRendPageContext&, ldomNode*, int, int, int, int, int, int, int*, unsigned int) coolreader/crengine/src/lvrend.cpp:8393
    20 0x55f5f43465cb in renderBlockElement(LVRendPageContext&, ldomNode*, int, int, int, int, int, int, int*) coolreader/crengine/src/lvrend.cpp:8409
    21 0x55f5f3f627eb in ldomDocument::render(LVRendPageList*, LVDocViewCallback*, int, int, bool, int, LVProtectedFastRef<LVFont>, int, LVFastRef<CRPropAccessor>, int, int) coolreader/crengine/src/lvtinydom.cpp:5259
    22 0x55f5f41d7d98 in LVDocView::Render(int, int, LVRendPageList*) coolreader/crengine/src/lvdocview.cpp:2897
    23 0x55f5f4189a0f in LVDocView::checkRender() coolreader/crengine/src/lvdocview.cpp:636
    24 0x55f5f41e707f in LVDocView::updateBookMarksRanges() coolreader/crengine/src/lvdocview.cpp:3396
    25 0x55f5f41f40f5 in LVDocView::restorePosition() coolreader/crengine/src/lvdocview.cpp:3967
    26 0x55f5f3ccf365 in CR3View::loadDocument(QString) coolreader/cr3qt/src/cr3widget.cpp:486
    27 0x55f5f3ccdd24 in CR3View::loadLastDocument() coolreader/cr3qt/src/cr3widget.cpp:468
    28 0x55f5f3d348a8 in MainWindow::showEvent(QShowEvent*) coolreader/cr3qt/src/mainwindow.cpp:467
    29 0x7f63abfc4947 in QWidget::event(QEvent*) (/usr/lib64/libQt5Widgets.so.5+0x1ad947)

Indirect leak of 896 byte(s) in 28 object(s) allocated from:
    0 0x7fd0a64a4a88 in __interceptor_realloc (/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/libasan.so.6+0xaea88)
    1 0x560452fd10d6 in lxmlAttribute* cr_realloc<lxmlAttribute>(lxmlAttribute*, unsigned long) (coolreader-debug-build/cr3qt/cr3+0x1c890d6)
    2 0x560452fab09e in ldomAttributeCollection::set(unsigned short, unsigned short, unsigned int) coolreader/crengine/src/lvtinydom.cpp:3933
    3 0x560452f597b9 in ldomNode::setAttributeValue(unsigned short, unsigned short, char32_t const*) coolreader/crengine/src/lvtinydom.cpp:17001
    4 0x560453498b0a in fixupMathML coolreader/crengine/src/mathml.cpp:2157
    5 0x56045349635c in fixupMathMLRecursive coolreader/crengine/src/mathml.cpp:1737
    6 0x560453496321 in fixupMathMLRecursive coolreader/crengine/src/mathml.cpp:1734
    7 0x560453496321 in fixupMathMLRecursive coolreader/crengine/src/mathml.cpp:1734
    8 0x560453496321 in fixupMathMLRecursive coolreader/crengine/src/mathml.cpp:1734
    9 0x560453496321 in fixupMathMLRecursive coolreader/crengine/src/mathml.cpp:1734
    10 0x560453496321 in fixupMathMLRecursive coolreader/crengine/src/mathml.cpp:1734
    11 0x560453496321 in fixupMathMLRecursive coolreader/crengine/src/mathml.cpp:1734
    12 0x560453496321 in fixupMathMLRecursive coolreader/crengine/src/mathml.cpp:1734
    13 0x560453496321 in fixupMathMLRecursive coolreader/crengine/src/mathml.cpp:1734
    14 0x560453496321 in fixupMathMLRecursive coolreader/crengine/src/mathml.cpp:1734
    15 0x56045349612f in fixupMathMLMathElement(ldomNode*) coolreader/crengine/src/mathml.cpp:1708
    16 0x560452e9f555 in ldomNode::initNodeRendMethod() coolreader/crengine/src/lvtinydom.cpp:7777
    17 0x560452ea023e in ldomElementWriter::onBodyExit() coolreader/crengine/src/lvtinydom.cpp:7819
    18 0x560452ea1acb in ldomElementWriter::~ldomElementWriter() coolreader/crengine/src/lvtinydom.cpp:7959
    19 0x560452f1575e in ldomDocumentWriterFilter::popUpTo(ldomElementWriter*, unsigned short, int) coolreader/crengine/src/lvtinydom.cpp:13899
    20 0x560452f1afba in ldomDocumentWriterFilter::AutoOpenClosePop(int, unsigned short) coolreader/crengine/src/lvtinydom.cpp:14130
    21 0x560452f2a60d in ldomDocumentWriterFilter::OnTagClose(char32_t const*, char32_t const*, bool) coolreader/crengine/src/lvtinydom.cpp:14661
    22 0x56045306835f in LVXMLParser::Parse() coolreader/crengine/src/lvxml/lvxmlparser.cpp:278
    23 0x56045307afe5 in LVHTMLParser::Parse() coolreader/crengine/src/lvxml/lvhtmlparser.cpp:157
    24 0x56045314f70c in LVDocView::ParseDocument() coolreader/crengine/src/lvdocview.cpp:5023
    25 0x5604531411f7 in LVDocView::loadDocumentInt(LVFastRef<LVStream>, bool) coolreader/crengine/src/lvdocview.cpp:4744
    26 0x56045311fc6d in LVDocView::LoadDocument(char32_t const*, bool) coolreader/crengine/src/lvdocview.cpp:4127
    27 0x560452bf3554 in CR3View::loadDocument(QString) coolreader/cr3qt/src/cr3widget.cpp:475
    28 0x560452bf2d24 in CR3View::loadLastDocument() coolreader/cr3qt/src/cr3widget.cpp:468
    29 0x560452c598a8 in MainWindow::showEvent(QShowEvent*) coolreader/cr3qt/src/mainwindow.cpp:467
Comment on lines +9059 to +9062
if (str.empty() && word->t.len > 0) {
// Don't know that the fuck up, but it happens
for (int i = 0; i < word->t.len; i++)
width[i] = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strange indeed.
Can you reproduce this? Specific book? Specific action that uses createXPointer() ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you reproduce this? Specific book?

Yes, of cource, book: http://fred-wang.github.io/mathml.css/

I don't know how to reproduce this in KOReader, but in the desktop version of CoolReader, just move the mouse over the text and the program will crash sooner or later.
Just see backtrace in commit comment.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, can't reproduce. Anyway, it feels that if this happens, the bug might be elsewhere, and it feels awkward handling that here.
If you have the time, could you try to printf the xpointer.toStringV1() when this happens, and try to find out what bit of HTML/MathML is involved in this document?

Just curious: why is there such a CR3View::mouseMoveEvent() handler ? Does it really get the xpointer at each mouse move ?! Feels it might feel really slow.

Copy link
Collaborator Author

@virxkane virxkane Jun 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious: why is there such a CR3View::mouseMoveEvent() handler ? Does it really get the xpointer at each mouse move ?! Feels it might feel really slow.

Sorry, I have zero interest in digging this deeper.

Comment on lines -6409 to +6411
for ( ; !node->isRoot(); node = node->getParentNode() )
if ( node->getStyle()->display==css_d_none )
for ( ; node && !node->isRoot(); node = node->getParentNode() ) {
css_style_ref_t style = node->getStyle();
if (!style.isNull() && style->display == css_d_none)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got a crash? Or just some reports from code analysis tools?
Because unless I missed something with the call from the added MathML bit, the only other call is from initNodeRendMethod, where there is always a style defined at this point.
And when walking nodes up, we should always meet the root node (so, !node->isRoot() will get us out of this loop), and so we should never meet a NULL node.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got a crash?

Yes, of cource.

Because unless I missed something with the call from the added MathML bit, the only other call is from initNodeRendMethod, where there is always a style defined at this point.

Crashed when style is null ref.

And when walking nodes up, we should always meet the root node (so, !node->isRoot() will get us out of this loop), and so we should never meet a NULL node.

I just added it just in case, it costs absolutely nothing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Crashed when style is null ref.

Can you get a backtrace? I'd like to see from which of the 2 hasInvisibleParent() calls this happen.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you get a backtrace? I'd like to see from which of the 2 hasInvisibleParent() calls this happen.

Sorry, I lost it. Document the same: http://fred-wang.github.io/mathml.css/

@@ -7774,6 +7775,8 @@ void ldomNode::initNodeRendMethod()
}
}
#endif

persist();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Asking, as you must have dig into that to understand it - and you added more in this and previous commit:
Is there a simple rule of thumb as to when we should use/add it?
No slowness added with doing it for each node here?

Copy link
Collaborator Author

@virxkane virxkane Jun 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Asking, as you must have dig into that to understand it - and you added more in this and previous commit:
Is there a simple rule of thumb as to when we should use/add it?

I think yes. If you call ldomNode::modify() to perform any node manipulation, then after that you must call ldomNode::persist().

No slowness added with doing it for each node here?

I don't know, not tested.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just look at the ldomNode::setAttributeValue() function where ldomNode::modify() is called. I think it would be silly to call persist() right there, it can be done later.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See ldomDocumentWriter::ElementCloseHandler().

Also you can try tinyNodeCollection::persist() instead of this.

Comment on lines -4141 to +4137
// possibly because it's not anchored anywhere.
// Attempt at anchoring into a _nullNode, and calling ->detroy()
// in ~ldomDocument(), did not prevent this report, and caused other ones...
ldomNode* node = allocTinyElement(NULL, 0, 0);
node->persist();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like when my questions get such simple answers ;)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In tinyNodeCollection::allocTinyElement() we alloc new object, therefore, somewhere we have to delete it. ldomNode::persist() converts the mutable object to persistent and removes the tinyElement instance that would otherwise be lost.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants