Running doccmd over a directory

[my1e5]

Found your tool from your comment on the "Apply ruff to markdown code blocks" issue. It's really nice! Very interested in adopting it into my workflow.

One feature which would be nice is the ability run doccmd over a folder. For example, I have a docs/ folder.

$ doccmd --language=python --command="mypy" docs/
Usage: doccmd [OPTIONS] [FILE_PATHS]...
Try 'doccmd --help' for help.

Error: Invalid value for '[FILE_PATHS]...': File 'docs/' is a directory.

In fact, do you know how I could run doccmd over a directory currently? Without specifying all the files individually?

[adamtheturtle]

Thank you @my1e5 for giving doccmd a go and for making an issue! You are the first known user who is not myself, and I don't have this use case, so all help figuring out what to do is welcome.

What I do is equivalent to:

$ doccmd --language=python --command="mypy" docs/**/*.rst

or

$ doccmd --language=python --command="mypy" docs/**/*.md

Does that solve your use case?

If not, or if you still think that it is best to have doccmd take a directory - do you think that doccmd should recursively search for files? With which extensions? (.md, .rst, configurable? if, configurable, how would you like to configure that set of file extensions?).

[my1e5]

Happy to provide feedback ☺️

So my use case is I have a docs/ folder in my project which I would like to run ruff check over. Ultimately I'd like to set up a "check-docs" command which I can then run locally and in my CI.

I guess coming to this as a ruff/mypy user, I'm used to these tools taking a list of files or directories as an argument.

$ ruff check --help
Run Ruff on the given files or directories (default)

Usage: ruff check [OPTIONS] [FILES]...

Arguments:
  [FILES]...  List of files or directories to check [default: .]

$ mypy --help
usage: mypy [-h] [-v] [-V] [more options; see below]
            [-m MODULE] [-p PACKAGE] [-c PROGRAM_TEXT] [files ...]

Mypy is a program that will type check your Python code.

Pass in any files or folders you want to type check. Mypy will
recursively traverse any provided folders to find .py files:

    $ mypy my_program.py my_src_folder

So I think it would feel very natural if doccmd could be used in a similar fashion.

I don't know much about .rst files, my docs folder is exclusively .md files. But I suppose recursively looking for both .rst and .md files would be fine? As mypy does - "Mypy will recursively traverse any provided folders to find .py files:".

I like how mypy and ruff both have an exclude setting you can configure in your pyproject.toml file. (see https://docs.astral.sh/ruff/settings/#exclude)

[tool.mypy]
exclude = ["tests"]

Maybe doccmd could have a table for configuring settings in a doccmd.toml file (I know this is not a Python-exclusive tool), which falls back to pyproject.toml if that file is not present?

[tool.doccmd]
exclude = ["docs/foo"]

Also I tried

$ doccmd --language=python --command="ruff check" docs/**/*.md

And it seems to only run on the first folder within my docs directory and then the first file within that folder.

EDIT -> Oh, it might be because the ruff check commnad errors out and then doesn't run on any more files?

[adamtheturtle]

Thank you @my1e5 for your comment and your edit. Have you figured out the answer to the question in your edit? If so, is your use case problem resolved (even if the interface is not ideal)?

[my1e5]

I found this method which works

$ find docs/  -type f -name "*.md" -exec doccmd --language=python --command="ruff check"  {} \;

[adamtheturtle]

Some open questions:

• What file types do we look for? (.rst, .md, .txt?)
• What is the interface for changing / adding to this set? (on the CLI, and potentially in a configuration file)
• What do we exclude by default? (Nothing? respect .gitignore?)

[adamtheturtle]

A potential plan for this:

• Only use Markdown parsers for Markdown files, and rST parsers for rST files #190
• Add support for folders - recurse for the file extensions specified in Only use Markdown parsers for Markdown files, and rST parsers for rST files #190 options / defaults
• Look at the above open questions
• Test de-dupe across directories / files given (Test that files are not considered twice if they're given alone and in a directory #210)
• De-duplicate extensions (e.g. If .rst is given twice, don't run against those files twice / search twice) (done in De-dupe extensions to improve performance #209 )
• Add flags to not search for Markdown / to not search for rST
• Check that giving . or ~/ as a directory works
• Add an option to not recurse (Add a --max-depth option for recursing into directories. #211)
• See if we can take advantage of Sybil's own recursion logic
• Add a way to exclude files / directories (Add option to exclude files #212 )
• Add option (on by default?) to respect .gitignore (now in Add option to respect .gitignore files #214 )

[adamtheturtle]

@my1e5 doccmd 2024.11.06.1 is now available. This allows you to pass in directories, and has options documented in https://doccmd.readthedocs.io/en/latest/commands.html#doccmd or in doccmd --help such as --myst-extension, --exclude and --max-depth to configure the recursion.

Please give this a go and leave a comment / create issues.