Searching front-matter of all docs

[bradumbaugh]

Can a macro inspect the front-matter of all the Markdown files in my project's docs folder? I'm trying to write a link_to macro that receives a key, finds a doc that has a matching key in its front-matter, and returns an absolute link to that file.

This is a great library, thanks! Enjoying playing around with it.

[fralau]

Thanks. 👍

Yes you probably could write such a macro! Here are a few pointers.

You could write a main.py program (in the top directory of your project), with a function that scan sthe yaml files in the docs directory (env.conf['docs_dir'] should contain its (relative) name; unless you are sure it's called docs)

It could then read all YAML headers .

define_env(env):
    """
    This is the hook for defining variables, macros and filters

    - variables: the dictionary that contains the environment variables
    - macro: a decorator function, to declare a macro.
    """

    @env.macro
    def link_to(key:str) -> str:
          """Finds a doc that has a matching key in its front-matter, and returns an absolute link to that file"""
          ...
          return link

Here are the basic instructions for writing macros.

To inspect the YAML front matters, you can check some good Python libraries, such as python-frontmatter.

Let me know if that's sufficient to get you started?

[fralau]

@bradumbaugh Thanks again. Your answers are helfpful, as they give me new insights. If I understand well, you want to implement a sort of interwiki system, where the definitions of the key/value pairs (prefixes) are specific to each page?

Interwiki links are links to pages of other projects, using a prefixed internal link style.
A link like [[Wikipedia:Main Page]] will appear like this: Wikipedia:Main Page and redirect you to Wikipedia's Main Page.
(Mediawiki's documentation)

In that case, we would be back to my first answer, but with a macro that would combine at least two elements: the prefix (key) and the suffix (specific for each call). For simplicity, I would also add a third argument: the text that will be highlighted in the hyperlink.

In that case, you would have, in each page's frontmatter, a 'page variable' declaration like this:

---
foo: https://foo.bar.baz/
bar: ....
---

What I might do is would be declare the macro like this (super-simplified):

    @env.macro
    def interwiki(text:str, prefix:str, suffix:str) -> str:
          """Create a markdown hyperlink, """
          full_url = f"{prefix}/{suffix}" # join the two
          return f"[{text}]({full_url})"

And call it in this way:

And this calls {{ interwiki("a special page", foo, 'bar/baz.html') }}

(I called it 'interwiki' because that's the way I think about it; but you could name it the way you want, of course).

Is my understanding correct? Would that answer your use case?

[bradumbaugh]

Thank you for the fast response! This is generally the right idea, yes.

But, for example, if I have these documents:

document-a.md:

---
key: doc-a
url: ...
---

Some text

How could I create a link to document-a.md, by its key, from another document?

document-b.md:

---
key: doc-b
url: ...
---

{{ link_to('My Link Text', 'doc-a') }}

Since the implementation of link_to, when called from document-b.md, doesn't have access to the front-matter in document-a.md, it seems like I can't do this, unless I have extras to use for looking up a URL by key.

I'm thinking I need to do something like this in mkdocs.yml:

extra:
  doc-keys:
    doc-a:
       url: path/to/document-a.md
    doc-b:
       url: path/to/document-b.md

In this way, the macro can access the URLs for all documents, by key.

I think this might work, I'm just trying to find a way to do it with out the extras, and without having to manually parse the front-matter of all docs each time I invoke the macro.

Please don't feel any pressure to continue with this conversation, you've been extraordinarily helpful and I hate to take your time.

[fralau]

Ah OK 🤔 Just curious, why would you adopt this form:

---
key: doc-a
url: path/to/document-a.md
---

Instead of:

---
doc-a: path/to/document-a.md
---

But even if it is wordy, I don't see your choice (key/url) as problem!

In any case, if you need to share those keys between documents, you are left with two solutions:

1. Gather them manually in the extra section of your config file.
2. Write a Python function, as explained, to read all the frontmatters and collect them in a variable, which will then be accessible to the macros in the markdown pages.

I don't find anything shocking in doing 2. 🤔

First of all, having to re-scan source code to extract data (but differently) is something that is often necessary and happens quite often time in compilers and interpreters (see, two-pass).

Secondly, it should take a very short time, unless your website has hundreds of pages; especially if your read the markdown pages from the env object directly (in-memory), instead of the file system.

But perhaps you may want to start with 1, since it is simpler? Then, once you feel more comfortable, you could venture into 2? How you think it is best. 🤷‍♂️

[bradumbaugh]

Thanks @fralau! This all makes sense to me, and I think you've got me going in a good direction, here. I'll probably try #2! Thank you very much for the interesting discussion, good ideas, and great library that adds so much value to MkDocs.

[fralau]

You're very welcome.