diff --git a/docs/_docs/04_reference/log-formatting.md b/docs/_docs/04_reference/log-formatting.md index dca141437..afc398e79 100644 --- a/docs/_docs/04_reference/log-formatting.md +++ b/docs/_docs/04_reference/log-formatting.md @@ -5,7 +5,7 @@ category: reference categories_url: reference excerpt: Learn how customize Terragrunt logging. tags: ["log"] -order: 408 +order: 409 nav_title: Documentation nav_title_link: /docs/ redirect_from: @@ -276,24 +276,32 @@ Specific options for placeholders: The examples below replicate the preset formats specified with `--terragrunt-log-format`. They can be useful if you need to change existing formats to suit your needs. +### Pretty + `--terragrunt-log-format pretty` ```shell --terragrunt-log-custom-format "%time(color=light-black) %level(case=upper,width=6,color=preset) %prefix(path=short-relative,color=gradient,suffix=' ')%tf-path(color=cyan,suffix=': ')%msg(path=relative)" ``` +### Bare + `--terragrunt-log-format bare` ```shell --terragrunt-forward-tf-stdout --terragrunt-log-custom-format "%level(case=upper,width=4)[%interval] %msg %prefix(path=short,prefix='prefix=[',suffix=']')" ``` +### Key-value + `--terragrunt-log-format key-value` ```shell --terragrunt-log-custom-format "time=%time(format=rfc3339) level=%level prefix=%prefix(path=short-relative) tf-path=%tf-path(path=filename) msg=%msg(path=relative,color=disable)" ``` +### JSON + `--terragrunt-log-format json` ```shell diff --git a/docs/_docs/04_reference/logging.md b/docs/_docs/04_reference/logging.md new file mode 100644 index 000000000..11e0e640e --- /dev/null +++ b/docs/_docs/04_reference/logging.md @@ -0,0 +1,254 @@ +--- +layout: collection-browser-doc +title: Logging +category: reference +categories_url: reference +excerpt: Learn how Terragrunt decides what to log, and when. +tags: ["log"] +order: 408 +nav_title: Documentation +nav_title_link: /docs/ +--- + +Terragrunt logs messages as it runs to help you understand what it's doing. Given that Terragrunt is an IaC orchestrator, this can result in messages that are surprising if you don't understand what Terragrunt is doing behind the scenes. + +## Log Levels + +To start with, Terragrunt has the following log levels: + +- `STDERR` +- `STDOUT` +- `ERROR` +- `WARN` +- `INFO` +- `DEBUG` +- `TRACE` + +The `STDOUT` and `STDERR` log levels are non-standard, and exist due to Terragrunt's special responsibility as an IaC orchestrator. + +For the most part, whenever you use Terragrunt to run something using another tool (like OpenTofu or Terraform), Terragrunt will capture the stdout and stderr terminal output from that tool, enrich it with additional information, then _log_ it as `STDOUT` or `STDERR` respectively. + +The exception to this is when Terragrunt is running a process in "Headless Mode", where it will instead emit stdout and stderr terminal output to `INFO` and `ERROR` log levels respectively. This happens when + +All other log levels are standard, and are used by Terragrunt to log its own messages. + +For example: + +```bash +$ terragrunt --terragrunt-log-level debug plan +14:20:38.431 DEBUG Terragrunt Version: 0.0.0 +14:20:38.431 DEBUG Did not find any locals block: skipping evaluation. +14:20:38.431 DEBUG Running command: tofu --version +14:20:38.431 DEBUG Engine is not enabled, running command directly in . +14:20:38.451 DEBUG tofu version: 1.8.5 +14:20:38.451 DEBUG Reading Terragrunt config file at ./terragrunt.hcl +14:20:38.451 DEBUG Did not find any locals block: skipping evaluation. +14:20:38.451 DEBUG Did not find any locals block: skipping evaluation. +14:20:38.452 DEBUG Running command: tofu init +14:20:38.452 DEBUG Engine is not enabled, running command directly in . +14:20:38.469 INFO tofu: Initializing the backend... +14:20:38.470 INFO tofu: Initializing provider plugins... +14:20:38.470 INFO tofu: OpenTofu has been successfully initialized! +14:20:38.470 INFO tofu: +14:20:38.470 INFO tofu: You may now begin working with OpenTofu. Try running "tofu plan" to see +14:20:38.470 INFO tofu: any changes that are required for your infrastructure. All OpenTofu commands +14:20:38.470 INFO tofu: should now work. +14:20:38.470 INFO tofu: If you ever set or change modules or backend configuration for OpenTofu, +14:20:38.470 INFO tofu: rerun this command to reinitialize your working directory. If you forget, other +14:20:38.470 INFO tofu: commands will detect it and remind you to do so if necessary. +14:20:38.470 DEBUG Running command: tofu plan +14:20:38.470 DEBUG Engine is not enabled, running command directly in . +14:20:38.490 STDOUT tofu: No changes. Your infrastructure matches the configuration. +14:20:38.490 STDOUT tofu: OpenTofu has compared your real infrastructure against your configuration and +14:20:38.490 STDOUT tofu: found no differences, so no changes are needed. +``` + +Here, we have three types of log messages: + +1. `DEBUG` messages from Terragrunt itself. By default, Terragrunt's log level is `INFO`, but we've set it to `DEBUG` using the `--terragrunt-log-level` flag. +2. `STDOUT` messages from OpenTofu. These are messages that OpenTofu would normally print directly to the terminal, but instead, Terragrunt captures them and logs them as `STDOUT` log messages, along with timestamps and other metadata. +3. `INFO` messages from Terragrunt [auto-init](/docs/features/auto-init). These were initially emitted by OpenTofu, however the user did not specifically ask for them, so Terragrunt logs them as `INFO` messages. + +## Enrichment + +The reason Terragrunt enriches stdout/stderr from the processes is that it is often very useful to have this extra metadata. + +For example: + +```bash +$ terragrunt run-all plan +14:27:45.359 INFO The stack at . will be processed in the following order for command plan: +Group 1 +- Module ./unit-1 +- Module ./unit-2 + + +14:27:45.399 INFO [unit-2] tofu: Initializing the backend... +14:27:45.399 INFO [unit-1] tofu: Initializing the backend... +14:27:45.400 INFO [unit-2] tofu: Initializing provider plugins... +14:27:45.400 INFO [unit-2] tofu: OpenTofu has been successfully initialized! +14:27:45.400 INFO [unit-2] tofu: +14:27:45.400 INFO [unit-2] tofu: You may now begin working with OpenTofu. Try running "tofu plan" to see +14:27:45.400 INFO [unit-2] tofu: any changes that are required for your infrastructure. All OpenTofu commands +14:27:45.400 INFO [unit-2] tofu: should now work. +14:27:45.400 INFO [unit-2] tofu: If you ever set or change modules or backend configuration for OpenTofu, +14:27:45.400 INFO [unit-2] tofu: rerun this command to reinitialize your working directory. If you forget, other +14:27:45.400 INFO [unit-2] tofu: commands will detect it and remind you to do so if necessary. +14:27:45.400 INFO [unit-1] tofu: Initializing provider plugins... +14:27:45.400 INFO [unit-1] tofu: OpenTofu has been successfully initialized! +14:27:45.400 INFO [unit-1] tofu: +14:27:45.400 INFO [unit-1] tofu: You may now begin working with OpenTofu. Try running "tofu plan" to see +14:27:45.400 INFO [unit-1] tofu: any changes that are required for your infrastructure. All OpenTofu commands +14:27:45.400 INFO [unit-1] tofu: should now work. +14:27:45.400 INFO [unit-1] tofu: If you ever set or change modules or backend configuration for OpenTofu, +14:27:45.400 INFO [unit-1] tofu: rerun this command to reinitialize your working directory. If you forget, other +14:27:45.400 INFO [unit-1] tofu: commands will detect it and remind you to do so if necessary. +14:27:45.422 STDOUT [unit-2] tofu: No changes. Your infrastructure matches the configuration. +14:27:45.423 STDOUT [unit-2] tofu: OpenTofu has compared your real infrastructure against your configuration and +14:27:45.423 STDOUT [unit-2] tofu: found no differences, so no changes are needed. +14:27:45.423 STDOUT [unit-1] tofu: No changes. Your infrastructure matches the configuration. +14:27:45.423 STDOUT [unit-1] tofu: OpenTofu has compared your real infrastructure against your configuration and +14:27:45.423 STDOUT [unit-1] tofu: found no differences, so no changes are needed. +``` + +Here you see two different units being run by Terragrunt concurrently, and stdout/stderr for each being emitted in real time. This is really helpful when managing IaC at scale, as it lets you know exactly what each unit in your stack is doing, and how long it is taking. + +It's easier to see the impact of this enrichment if we turn it off, so let's use the [bare](/docs/reference/log-formatting/#bare) preset described in [Log Formatting](docs/reference/log-formatting). + +```bash +$ terragrunt run-all --terragrunt-log-format bare plan +INFO[0000] The stack at /Users/yousif/tmp/testing-stdout-stderr-split will be processed in the following order for command plan: +Group 1 +- Module /Users/yousif/tmp/testing-stdout-stderr-split/unit-1 +- Module /Users/yousif/tmp/testing-stdout-stderr-split/unit-2 + + + +Initializing the backend... + +Initializing provider plugins... + +OpenTofu has been successfully initialized! + +You may now begin working with OpenTofu. Try running "tofu plan" to see +any changes that are required for your infrastructure. All OpenTofu commands +should now work. + +If you ever set or change modules or backend configuration for OpenTofu, +rerun this command to reinitialize your working directory. If you forget, other +commands will detect it and remind you to do so if necessary. + +No changes. Your infrastructure matches the configuration. + +OpenTofu has compared your real infrastructure against your configuration and +found no differences, so no changes are needed. + +Initializing the backend... + +Initializing provider plugins... + +OpenTofu has been successfully initialized! + +You may now begin working with OpenTofu. Try running "tofu plan" to see +any changes that are required for your infrastructure. All OpenTofu commands +should now work. + +If you ever set or change modules or backend configuration for OpenTofu, +rerun this command to reinitialize your working directory. If you forget, other +commands will detect it and remind you to do so if necessary. + +No changes. Your infrastructure matches the configuration. + +OpenTofu has compared your real infrastructure against your configuration and +found no differences, so no changes are needed. +``` + +This tells Terragrunt to log messages from OpenTofu/Terraform without any enrichment. As you can see, it's not as easy to disambiguate the messages from the two units, so Terragrunt automatically buffers the output from each unit and logs them in order so that you can see what each unit is doing. + +## Exceptions to enrichment + +There are exceptions to the general rule that Terragrunt logs stdout/stderr from the processes it runs as `STDOUT` and `STDERR` respectively when not in Headless Mode. + +Because Terragrunt is an IaC orchestrator, it uses its awareness of OpenTofu/Terraform usage to recognize certain circumstances when a user is likely to want stdout/stderr to be emitted exactly as it would when running a process directly. + +An example of this is `terragrunt output`: + +```bash +$ terragrunt output -json +15:20:07.759 INFO tofu: Initializing the backend... +15:20:07.759 INFO tofu: Initializing provider plugins... +15:20:07.759 INFO tofu: OpenTofu has been successfully initialized! +15:20:07.759 INFO tofu: +15:20:07.759 INFO tofu: You may now begin working with OpenTofu. Try running "tofu plan" to see +15:20:07.759 INFO tofu: any changes that are required for your infrastructure. All OpenTofu commands +15:20:07.759 INFO tofu: should now work. +15:20:07.759 INFO tofu: If you ever set or change modules or backend configuration for OpenTofu, +15:20:07.759 INFO tofu: rerun this command to reinitialize your working directory. If you forget, other +15:20:07.759 INFO tofu: commands will detect it and remind you to do so if necessary. +{ + "something": { + "sensitive": false, + "type": "string", + "value": "Hello, World!" + }, + "something_else": { + "sensitive": false, + "type": "string", + "value": "Goodbye, World!" + } +} +``` + +As you can see, the output from OpenTofu here isn't being enriched even though the user explicitly asked Terragrunt to run `output -json`. + +This is because the user is pretty likely to want to programmatically interact with the output of `output`, and so Terragrunt doesn't enrich it. + +If, for example, you wanted to use a tool like `jq` to parse the output of `terragrunt output -json`, you could do that without having to worry about Terragrunt's metadata getting in the way, or disabling anything with an extra flag. + +```bash +$ terragrunt output -json | jq '.something' +15:24:40.310 INFO tofu: Initializing the backend... +15:24:40.311 INFO tofu: Initializing provider plugins... +15:24:40.311 INFO tofu: OpenTofu has been successfully initialized! +15:24:40.311 INFO tofu: +15:24:40.311 INFO tofu: You may now begin working with OpenTofu. Try running "tofu plan" to see +15:24:40.311 INFO tofu: any changes that are required for your infrastructure. All OpenTofu commands +15:24:40.311 INFO tofu: should now work. +15:24:40.311 INFO tofu: If you ever set or change modules or backend configuration for OpenTofu, +15:24:40.311 INFO tofu: rerun this command to reinitialize your working directory. If you forget, other +15:24:40.311 INFO tofu: commands will detect it and remind you to do so if necessary. +{ + "sensitive": false, + "type": "string", + "value": "Hello, World!" +} +``` + +## Disabling Logs + +Finally, you can also disable logs entirely like so: + +```bash +$ terragrunt --terragrunt-log-disable plan + +Initializing the backend... + +Initializing provider plugins... + +OpenTofu has been successfully initialized! + +You may now begin working with OpenTofu. Try running "tofu plan" to see +any changes that are required for your infrastructure. All OpenTofu commands +should now work. + +If you ever set or change modules or backend configuration for OpenTofu, +rerun this command to reinitialize your working directory. If you forget, other +commands will detect it and remind you to do so if necessary. + +No changes. Your infrastructure matches the configuration. + +OpenTofu has compared your real infrastructure against your configuration and +found no differences, so no changes are needed. +``` + +This will give you the closes experience to using OpenTofu/Terraform directly, with Terragrunt doing all of its work in the background.