Build documentation with mkdocs or sphinx for monorepos?

[jgarte]

Hi, does pants have a way to build documentation for monorepos with sphinx or mkdocs?

Should that feature/functionality be handled by pants?

I think maybe?

What do you think?

The projects that I've come across that try to provide monorepo support for sphinx or mkdocs seem very experimental.

I wonder if pants can help here...

[benjyw]

There is no sphinx plugin at the moment, but we would love for there to be one, not least because it's a future option for generating Pants's own docs (we currently use readme.com, but it's pretty limited).

There is some concurrent discussion of support for Jupyter Book: #17789 which is related

[jgarte]

@benjyw

Here is some info I found that could be useful to consult in thinking about how to solve this issue:

https://sphinx-collections.readthedocs.io/en/latest/

referenced in the following blog post:

https://wrla.ch/blog/2022/09/using-sphinx-in-a-monorepo/

[jgarte]

@benjyw or any other pants dev/contributor:

Do you have any hypotheticals/ideas for how pants would support building documentation?

Some open questions

Does it make sense for sphinx-collections to be integrated into pants or is the "driver" functionality in sphinx-collections something that pants can take care of itself?

We are asking similar questions here:

useblocks/sphinx-collections#8

I am not an expert yet in either system, their feature parities, or how people have managed documentation for monorepos in the past so I'm looking for guidance from the community on this matter. Any help is much appreciated as it is a topic that I'm currently very interested in helping out with.

[benjyw]

Hi! I'm no expert on sphinx either, and I'm not sure anyone in the Pants community is. Possibly @chrisjrn might have a bit more context than I do. So your ideas/recommendations are very welcome!

I'll read up a bit on the resources you linked to and come back with thoughts.

[benjyw]

OK, having skimmed those resources. I think, and could be wrong, that Pants should be the "driver" and should not use sphinx-collections.

The reason is that Pants is already really good at gathering files from all over the codebase and nesting them into a single tree. It does this a lot with code - e.g., creating a single package hierarchy from a bunch of different source roots. So doing so with documentation files seems directly analogous?

[jgarte]

Thanks for the reply!

OK, having skimmed those resources. I think, and could be wrong, that Pants should be the "driver" and should not use sphinx-collections.

I actually had an intuition about this being the very approach that pants should probably take given the little that I know about pants and sphinx-collections so I'm glad that you are confirming that intuition :)

I even described pants as the driver here in this reply on the sphinx-collections thread:

useblocks/sphinx-collections#8 (reply in thread)

That said, I was merely hypothesizing at a high level without knowing the details of pants architecture or how it really works :)

[jgarte]

@benjyw

that Pants should be the "driver" and should not use sphinx-collections.

If pants should be the "driver" what would a documentation layout for a large monorepo look like that follows PEP 420?

See this issue thread for discussions to possibilities of proposed monorepo directory layouts:

useblocks/sphinx-collections#10 (comment)

[benjyw]

That I'm not sure about, but Pants generally likes to be agnostic to layout choices, even if it is "gently opinionated" about them in some cases (e.g., we do recommend a single source root or a small number of them, not one per subproject, but you don't have to do that).

[benjyw]

I think the idea would be that, no matter how you lay out the sources, Pants strips source roots and merges everything into a single directory for processing. So the relevant concept is source root: https://www.pantsbuild.org/docs/source-roots