Terraform not able to find parent directory when we are running terraform plan outside of terrakube it work

[thatsk]

Hello Team,

i am trying to test terraform code with terrakube. It is working fine if i am running outside of terrakube. with terraform cli but when i run via terrakube ui its not able to find one level directory up.

Directory structure:-

Repo:-
.
├── atlantis.yaml
├── cloneterraform.sh
├── data
│   ├── node.yaml
│   ├── node1.yaml 
│   ├── environment.yaml
└── terraform
    ├── backends.tf
    ├── importgenerate.sh
    ├── main.tf
    ├── removekey.sh
    ├── terraform-provider-foreman.log
    ├── terraform.tfvars
    ├── tf-run.sh
    └── variable.tf

terraform is running from terraform directory and trying to get the data from the data directory which has some input in the yaml file.

locals {
  all_nodes = flatten([
    for file in fileset(var.data_path, "*.yaml") :
      file != "environment.yaml" ? yamldecode(file("${var.data_path}/${file}")).nodelist != null ? yamldecode(file("${var.data_path}/${file}")).nodelist : [] : []
  ])
  vars = yamldecode(file("${var.data_path}/environment.yaml"))
}

data_path variable defined to data_path = "../data"
main.tf has to download terraform modules which module has this code which tries to fetch the data.
but seem like the executor is not able to understand the directory.

Planning failed. Terraform encountered an error while generating this plan.

���
��� Error: Invalid function argument
���
��� on .terraform/modules/foreman/main.tf line 21, in locals:
��� 21: vars = yamldecode(file("${var.data_path}/environment.yaml"))
��� ���������������������������������������������������
��� ��� while calling file(path)
��� ��� var.data_path is "../data"
���
��� Invalid value for "path" parameter: no file exists at
��� "../data/environment.yaml"; this function works only with files that are
��� distributed as part of the configuration source code, so if this file will
��� be created by a resource in this configuration you must instead obtain this
��� result from an attribute of that resource.

[thatsk]

@alfespa17 need thoughts on this. This is working fine if you plane clone the repository and run terraform locally.

[thatsk]

path.module, path.root,path.cwd something we need to see might executor is not able to find it or I am missing something ?

[alfespa17]

If I remembered correctly when you define a workspace terrakube clone the entire repository but It will only copy the files of the folder that you have specified in the "path", those files will be available when running the job context.

There is one folder where you can find all the cloned repo, I will check the code later today and update this thread with that info.

[alfespa17]

Here is a little example @thatsk

This is the sample repository

The job context folder will be available in this directory "/home/gitpod/.terraform-spring-boot/executor/{{organizationId}}/{{workspaceId}}" when running in our test environment.

gitpod ~/.terraform-spring-boot/executor/d9b58bd3-f3fc-4056-a026-1163297e80a8/5ed411ca-7ab8-4d2f-b591-02d0d5788afc (main) $ pwd
/home/gitpod/.terraform-spring-boot/executor/d9b58bd3-f3fc-4056-a026-1163297e80a8/5ed411ca-7ab8-4d2f-b591-02d0d5788afc

The original repository is cloned to the following directory:

gitpod ~/.terraform-spring-boot/executor/d9b58bd3-f3fc-4056-a026-1163297e80a8/5ed411ca-7ab8-4d2f-b591-02d0d5788afc/.originRepository (main) $ ls -al
total 28
drwxr-xr-x 5 gitpod gitpod   131 Jul  5 14:40 .
drwxr-xr-x 7 gitpod gitpod  4096 Jul  5 14:40 ..
drwxr-xr-x 2 gitpod gitpod    53 Jul  5 14:40 getting-started
drwxr-xr-x 7 gitpod gitpod   150 Jul  5 14:40 .git
-rw-r--r-- 1 gitpod gitpod    46 Jul  5 14:40 helloWorldTerrakube.sh
-rw-r--r-- 1 gitpod gitpod 11345 Jul  5 14:40 LICENSE
drwxr-xr-x 2 gitpod gitpod    33 Jul  5 14:40 local
-rw-r--r-- 1 gitpod gitpod   390 Jul  5 14:40 main.tf
-rw-r--r-- 1 gitpod gitpod   149 Jul  5 14:40 README.md

Once the repository is cloned, terrakube moves the information from the ".originRepository/{{specificPath}}" to the job context folder, this is one example:

gitpod ~/.terraform-spring-boot/executor/d9b58bd3-f3fc-4056-a026-1163297e80a8/5ed411ca-7ab8-4d2f-b591-02d0d5788afc (main) $ ls -al
total 44
drwxr-xr-x 7 gitpod gitpod  4096 Jul  5 14:40 .
drwxr-xr-x 3 gitpod gitpod    50 Jul  5 14:40 ..
-rw-r--r-- 1 gitpod gitpod    40 Jul  5 14:40 commitHash.info
drwxr-xr-x 2 gitpod gitpod    53 Jul  5 14:40 getting-started
drwxr-xr-x 7 gitpod gitpod   150 Jul  5 14:40 .git
-rw-r--r-- 1 gitpod gitpod    46 Jul  5 14:40 helloWorldTerrakube.sh
-rw-r--r-- 1 gitpod gitpod 11345 Jul  5 14:40 LICENSE
drwxr-xr-x 2 gitpod gitpod    33 Jul  5 14:40 local
-rw-r--r-- 1 gitpod gitpod   390 Jul  5 14:40 main.tf
drwxr-xr-x 5 gitpod gitpod   131 Jul  5 14:40 .originRepository
-rw-r--r-- 1 gitpod gitpod   149 Jul  5 14:40 README.md
drwxr-xr-x 3 gitpod gitpod    48 Jul  5 14:40 .terraform
-rw-r--r-- 1 gitpod gitpod  3117 Jul  5 14:40 terraformLibrary.tfPlan
-rw-r--r-- 1 gitpod gitpod  2197 Jul  5 14:40 .terraform.lock.hcl
-rw-r--r-- 1 gitpod gitpod   210 Jul  5 14:40 terrakube_override.tf

You can find more information of the job flow in this link

In your case your path should be:

data_path = "./.originRepository/data"

I hope this can help you :)

[thatsk]

Does it discouraging to use this https://developer.hashicorp.com/terraform/language/expressions/references#filesystem-and-workspace-info
And use terrakube execution workflow I thought it should be uniform across all terrform tools like terraform cloud, scalr, atlantis. But terrakube is following its own way.

I will try to see your code by updating it and check

[alfespa17]

I think the issue is how we handle the job context files because we only take the files inside one specific folder from the repository and we basically forget about the rest of files, this is something that we could change in the future.

Edit:

I created this issue so we can improve the forlder structure when running jobs