Add hide_success and hide_progress options

[dae]

This is an implementation of the output suppression I mentioned in #68 (comment), and is related to #101. We've been using it in our project for the last year or so (1), and have been very happy with the balance of immediate output without overwhelming amounts of text being shown (2). I present it here not for immediate inclusion, but to rekindle discussion, as functionality like this would be nice to have in a stock n2 in the future.

(1) I've just rebased our version over the latest code

(2) One minor papercut is that certain terminal formatting codes can sometimes cause the progress output to render incorrectly, such as in the wrong color. But that's not an issue introduced by these changes.

[evmar]

Thanks for sending this!

Can you add a section to doc/comparison.md with a name like "ninja extensions" that describes these?

What do you think about naming "hide_last_line" something more like "hide_progress"? I think that better describes the intent of the feature, and might allow more flexibility for changing details on how the feature works in the future, maybe?

For example if you run n2 in a CI pipeline (where there's no terminal, just a stdout logfile) with the hide_last_line flag on, what behavior do you expect, and what behavior do you imagine another user would expect?

[evmar]

I guess the precedent for boolean flags in ninja is the "restat = 1" flag, where it is just a presence/absence check rather than requiring an explicit "1", so maybe we should follow that here?

https://github.com/ninja-build/ninja/blob/2a34463e6ec38dab909af94740070d546354b16c/src/graph.cc#L515

[evmar]

yeah the docs say "if present, ..." for these kinds of flags

[Colecf]

In android, we have a --frontend_file option that makes ninja/n2 emit nothing at all to stdout/stderr, but instead write protobuf-encoded messages about its status to a file. (And in practice we use a FIFO for the file) Then we have a different program wrap ninja and emit the format we want. This may be worth considering as something more generic instead of giving a bunch of discrete flags to control the output.

[evmar]

I love the idea of a separate program for customization progress! But this patch is about a per-build-edge customization of behavior. We had discussed this in the past and I didn’t have any better ideas. I guess you could wrap commands in something else that swallowed their stdout maybe?

[Colecf]

You can do per-build-edge customization based on the information in the EdgeStarted message. In our protobuf the most useful field for that would be the build description, but you could add the ability to add custom fields. One easy way would just be to pass the value of all variables starting with some prefix like "custom_" to the protobuf.

Also in case it wasn't clear, the --frontend_file functionality we have makes ninja redirect all the commands' stdout/stderr into the protobuf as well, so you can play with that too.

[dae]

frontend_file is certainly the most flexible/generic solution, but for users who are otherwise happy with n2's existing pretty output, requiring them to implement their own frontend is not ideal.

I've made the changes requested above. Regarding the non-pretty output case, I don't think hide_progress makes sense there, as that mode doesn't show progress. If you'd like anything else changed, please let me know.

[evmar]

In general I'm a big fan of the frontend pipe idea (I think I might even have a branch around where I tried some of it, though with json instead of proto). But I also think this change looks good for dae's purposes.

[evmar]

FYI I fixed CI after this (it was unrelated to your change, just coincidental timing of some github deprecation apparently)