Lua filter's Inlines callback is not getting called for a simple markdown file with a loan citation element

[kjambunathan]

This bug is a result of retorquere/zotero-better-bibtex#1581. In @retorquere's machines the Inlines function does work. But it doesn't work on my machine.

(You can take a look at the above issue if you want further details)

Pandoc version

~$ pandoc --version
pandoc 2.9.1.1
Compiled with pandoc-types 1.20, texmath 0.12.0.2, skylighting 0.8.4
Default user data directory: /home/kjambunathan/.local/share/pandoc or /home/kjambunathan/.pandoc
Copyright (C) 2006-2019 John MacFarlane
Web:  https://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

~$ dpkg -l | grep pandoc
ii  libghc-pandoc-citeproc-data           0.16.4.1-1                         all          Pandoc support for Citation Style Language - data files
ii  pandoc                                2.9.1.1-2                          amd64        general markup converter
ii  pandoc-citeproc                       0.16.4.1-1+b1                      amd64        Pandoc support for Citation Style Language - tools
ii  pandoc-data                           2.9.1.1-2                          all          general markup converter - data files

~$ uname -a
Linux debian 5.7.0-1-amd64 #1 SMP Debian 5.7.6-1 (2020-06-24) x86_64 GNU/Linux

Download test.lua

~$ curl -s https://gist.githubusercontent.com/retorquere/fe514aa0344c65d28445d8dceb899de1/raw/6ea5fdad0764c002d51ff409c854e40f5a777a64/mwe.lua -o test.lua

Look at test.lua. It registers for Inlines

~$ cat test.lua
function Inlines_holler(inlines)
  print('inlines')
  return inlines
end

return {
  { Inlines = Inlines_holler },
}

Create an md file with a loan citaiton keys

~$ echo "[@wrigstad2017mastery]" > ww.md
~$ cat ww.md
[@wrigstad2017mastery]

Run the pandoc with the above test.lua

~$ pandoc -s --lua-filter=./test.lua ww.md -t plain
[@wrigstad2017mastery]

Expected output

inlines in the output stream

Actual behaviour

No inlines in the output stream. This implies Inlines_holler is not getting called.

[kjambunathan]

Other Lua filter (like pandoc-zotxt does work on my machine)

~$ cat ww.md
[@wrigstad2017mastery]
~$ pandoc -s --lua-filter=pandoc-zotxt.lua -F pandoc-citeproc ww.md -t plain


(Wrigstad and Castegren 2017)

Wrigstad, Tobias, and Elias Castegren. 2017. “Mastery Learning-Like
Teaching with Achievements.” In SPLASH-E.

Here is the AST for the test input

~$ echo 'See [@wrigstad2017mastery].' | pandoc -f markdown -t native ww.md

[Para [Cite [Citation {citationId = "wrigstad2017mastery", citationPrefix = [], citationSuffix = [], citationMode = NormalCitation, citationNoteNum = 0, citationHash = 0}] [Str "[@wrigstad2017mastery]"]]]

[kjambunathan]

In @retorquere's machines the Inlines function does work. But it doesn't work on my machine.

I want to reitrate this observation.

This issue is very subtle.

@retorquere was kind enough to help me /in real-time/. Otherwise, I wouldn't have progressed this far.

[mb21]

can you try with the newest pandoc release?

[kjambunathan]

Lua filter's Inlines callback is not getting called

can you try with the newest pandoc release?

Instead of compiling from source, I would prefer to use the binaries at https://github.com/jgm/pandoc/releases/latest

I see following remark:

Note: because of the static linking, the pandoc binary from this package cannot use lua filters that require external lua modules written in C.

I cannot make out the implications of that remark. Can I use that version for current purposes ...

[kjambunathan]

Ok ... I should have asked this.

Do you want me to compile and install from git. Or, does testing with released tar work for you

[retorquere]

The MWE filter posted here doesn't use any external modules at all.

[mb21]

huh, are you people having a project-specific discussion in the pandoc issue tracker, or..?

[kjambunathan]

The bug goes away once I upgraded.

~$ dpkg -l | grep pandoc
ii  libghc-pandoc-citeproc-data           0.16.4.1-1                         all          Pandoc support for Citation Style Language - data files
ii  pandoc                                2.10-1                             amd64        general markup converter
ii  pandoc-citeproc                       0.16.4.1-1+b1                      amd64        Pandoc support for Citation Style Language - tools
ii  pandoc-data                           2.9.1.1-2                          all          general markup converter - data files


~$ pandoc --version
pandoc 2.10
Compiled with pandoc-types 1.21, texmath 0.12.0.2, skylighting 0.8.5
Default user data directory: /home/kjambunathan/.local/share/pandoc or /home/kjambunathan/.pandoc
Copyright (C) 2006-2020 John MacFarlane
Web:  https://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.
~$ cat test.lua
function Inlines_holler(inlines)
  print('inlines')
  return inlines
end

return {
  { Inlines = Inlines_holler },
}
~$ cat ww.md
[@wrigstad2017mastery]
~$ pandoc -s --lua-filter=./test.lua ww.md -t plain
inlines
inlines
inlines
inlines
[@wrigstad2017mastery]

[mb21]

The bug goes away once I upgraded.

great, closing this then...

[retorquere]

huh, are you people having a project-specific discussion in the pandoc issue tracker, or..?

Not really -- the discussion originated in the other repo but the MWE showed the problem to be reproducible with a simple lua filter that used only pandoc facilities. In any case, 2.10 seems to have solved it.

[tarleb]

The facility to filter on Inlines was added in 2.9.2.