Terragrunt Log Formatting

[yhakbar]

Release v0.68.14-beta2024111502 introduces new controls for log formatting.

Share your feedback!

[yhakbar]

I'd especially like to invite @grimm26 , @lorengordon , @wyardley and @tide-jamiegwatkin to share your feedback.

We want to make sure we're addressing any concerns that you had early on during our logging redesign, and are soliciting any feedback you may have to improve logging customization further.

[wyardley]

Thanks! I am not in a position to test myself in the next few weeks for various reasons, but really excited to hear about this, and also glad to see it get some testing ahead of time.

[lorengordon]

Is there an env option to set --terragrunt-log-custom-format? I'm not seeing one mentioned in the linked doc...

[levkohimins]

@lorengordon, Sure, there is terragrunt-log-custom-format.

[grimm26]

@yhakbar how can I use this to stop terragrunt from showing all errors twice?

[yhakbar]

Hey @grimm26 ,

This update has more to do with addressing issues like this:
#3431

And to give you full control over the formatting of logs.

We will shortly add another flag named --terragrunt-log-disable-error-summary to prevent emitting the summary of encountered errors after a run-all once the updates referenced here are released.

[yhakbar]

It wasn't that quick, but we do have an in-draft PR that has the code to address this:
#3687

We're currently in the process of redesigning our CLI and have a company-wide break coming up, so we might not be able to get it released soon. Just wanted to give a status update and make sure your concerns weren't being ignored.

[atheiman]

I'd like to run something like terragrunt run-all plan -out tfplan --terragrunt-stdout-file terragrunt.stdout.txt and send stdout for each directory to <directory>/terragrunt.stdout.txt and hide from stdout.

I use run-all against dozens / hundreds of directories and currently I get >300,000 lines of log output in my ci/cd interface. These logs are primarily excessively noisy Terraform refreshing state ... lines, which there are several issues on Terraform for quieting. Mostly the recommendation to quiet Terraform output is to run terraform plan -out=tfplan >/dev/null, however, i dont have a way to run that with terragrunt run-all ... (unless Im missing something).

[yhakbar]

You have two options:

1. You can use terragrunt-out-dir to create a secondary directory to store all the output files that mirrors the filesystem of your stack.
2. You can set -out to ../../tfplan in order to have the plan file stored next to the terragrunt.hcl files. By default, they'll be stored in the .terragrunt-cache directory.

Most users avoid trying to store the actual stdout of a run, and instead store the plan binary, then reproduce stdout as needed.

[atheiman]

Separately from my other comment... is there a value I can specify for --terragrunt-log-custom-format to only show the terraform command from %tf-command-args rather than the command with all the args? For example, %tf-command-args is currently expanding to init -reconfigure -backend=false but I'd rather just see init to know what stage of my init/plan/apply lifecycle the command is executing (I dont need to see -reconfigure -backend=false in the log prefix). Maybe a new replacement named %tf-command would be easier to use for this?

[yhakbar]

This sounds like a feature request!

If you have any golang experience, I don't imagine this would be a difficult task to tackle in a pull request to deliver the feature yourself.

[atheiman]

ok great, I just wanted to confirm there wasnt already a built-in replacement for this I was missing.

[atheiman]

new issue: #3691

[atheiman]

Also Ive noticed that stdout from before_hookwithin a given terragrunt dir is delayed until all run-all terraform command stdout is complete for that terragrunt dir. I have a before_hook that runs terraform show tfplan | tf-summarize to print the plan summary before it is applied (otherwise terraform doesnt tell you much about what it is doing, all you see is <resource.path>: Modifying ...)

Im not seeing the stdout from the before_hook until all the stdout from the module's stdout has been printed. So my before_hook to show plan summary on stdout is printed to the console after the apply output is printed to the console. This is in a ci/cd interface, so the terminal type may be having an effect.

[levkohimins]

Also Ive noticed that stdout from before_hookwithin a given terragrunt dir is delayed until all run-all terraform command stdout is complete for that terragrunt dir. I have a before_hook that runs terraform show tfplan | tf-summarize to print the plan summary before it is applied (otherwise terraform doesnt tell you much about what it is doing, all you see is <resource.path>: Modifying ...)

Im not seeing the stdout from the before_hook until all the stdout from the module's stdout has been printed. So my before_hook to show plan summary on stdout is printed to the console after the apply output is printed to the console. This is in a ci/cd interface, so the terminal type may be having an effect.

This is done to ensure that output from different modules does not overlap each other, for example, if it is json, then mixing output will cause their corruption.

[yhakbar]

Terragrunt does buffer terminal output during run-all to make it easier to avoid mangling output (e.g. if you have one run emitting to stdout, you'd probably prefer that another one not cut it off mid-log).

If you have a simple fixture that can reliably reproduce the undesirable behavior, it might be easier to take a look and see if it's feasible to address.

I'm not sure if we can dedicate resources to it, however, as I'm not sure how many users this is impacting (and how badly). Even if it's evidently undesirable and easy to reproduce, we might need community contribution to address it (including the testing required to avoid accidentally mangling streams between runs of different units).

[atheiman]

Thank you both for each of the replies

@levkohimins Im good with the buffering of a single module output, I like that behavior. The issue Im running into is before_hook stdout is delayed until after the terraform command has run. The before_hook stdout should show before the terraform command stdout.

A simple example of the issue can be seen with terragrunt run-all plan with a subdirectory terragrunt.hcl with a before_hook

before_hook "echo_to_stdout" {
  commands = ["plan"]
  execute = ["bash", "-c", "echo 'this should show before terraform STDOUT, but it is delayed until after terraform STDOUT'"]
}