Smart quotes sometimes too aggressive

[Omikhleia]

Running with the smart extension enabled.

It's clear it shouldn't happen. Aujourd'hui n'est pas demain.

With Pandoc: <p>It’s clear it shouldn’t happen. Aujourd’hui n’est pas demain.</p>
(with right quotes = apostrophes everywhere)

But Lunamark generates <p>It‘s clear it shouldn’t happen. Aujourd‘hui n’est pas demain.</p>
(with left quotes in "It's" and "Aujourd'hui")

[Witiko]

It seems to me that Pandoc only considers quotes that begin at word boundaries:

https://github.com/jgm/pandoc/blob/139f9c064d056989f3a7071341c7b15bfc1850a7/src/Text/Pandoc/Parsing/Smart.hs#L142-L151

By contrast, we are quite happy to parse quotes anywhere:

lunamark/lunamark/reader/markdown.lua

Lines 1068 to 1073 in 1eeea41

	larsers.DoubleQuoted = parsers.dquote * Ct((parsers.Inline - parsers.dquote)^1)
	* parsers.dquote / writer.doublequoted
	larsers.SingleQuoted = parsers.squote_start
	* Ct((parsers.Inline - parsers.squote_end)^1)
	* parsers.squote_end / writer.singlequoted

[Omikhleia]

An additional case to consider:

D&D was a big thing in the '80s.

Lunamark currently gives:

<p>D&amp;D was a big thing in the &#39;80s.</p>

(with a straight apostrophe).

Expected (as with Pandoc):

<p>D&amp;D was a big thing in the ’80s.</p>

[Witiko]

Seems like single quotes get special additional treatment by Pandoc. I will need to delve into Pandoc's implementation a bit more. As illustrated by the fenced divs, it may be more time-efficient to study Pandoc's code over treating it like a black box and reverse-engineering the code from its outputs.