feat(application): deploy applications with custom resources

[gatici]

Description

This PR provides to use custom (OCI image) charm resources with applications. The application resource could be a charm revision or a custom OCI image which needs to be pulled from a repository. The following "juju_application" resource definition becomes valid within this PR. This PR both works with all the Juju versions which are supported by Juju Terraform provider. (Tested with Juju 2.9.x and 3.4.x versions)

resource "juju_application" "ausf" {
  name = "ausf"
  model = var.model_name

  charm {
    name = "sdcore-ausf-k8s"
    channel = var.channel
  }
  resources = {
      ausf-image = "gatici/sdcore-ausf:1.4.0"
      // ausf-image =  "2"
  }
  units = 1
  trust = true
}

Design Spec: https://docs.google.com/document/d/1i236ntmw-qXyqifmEtk_rcszWXcXSUhXkeJ6ByKXIeE/edit

Fixes: #460

Type of change

• Change existing resource
• Change in the tests (one or several tests have been changed)
• Requires a documentation update

Environment

The following steps are performed to prepare the environment.

Install Microk8s:

sudo snap install microk8s --channel=1.29-strict/stable
sudo usermod -a -G snap_microk8s $USER
newgrp snap_microk8s
sudo microk8s enable hostpath-storage

Install Juju:

sudo snap install juju --channel=3.4/stable
juju bootstrap microk8s

Install Terraform:

sudo snap install --classic terraform

Create terraformrc file with following contents. Write the path of terraform.d/plugins directory if it is in a different path.

touch .terraformrc 

provider_installation {
  filesystem_mirror {
    path = "~/.terraform.d/plugins"
    include = ["juju/juju"]
  }
  direct {
    exclude = ["juju/juju"]
  }
}

QA steps

Manual QA steps should be done to test this PR.

Create a folder to store the Terraform files for testing.

mkdir testing
cd testing

Create a main.tf file with following contents:

resource "juju_application" "ausf" {
  name = "ausf"
  model = var.model_name

  charm {
    name = "sdcore-ausf-k8s"
    channel = var.channel
  }
  resources = {
      ausf-image = "gatici/sdcore-ausf:1.4.0"
  }
  units = 1
  trust = true
}

Create a terraform.tf file with following contents:

terraform {
  required_providers {
    juju = {
      source  = "juju/juju"
      version = "0.12.0"
    }
  }
}

Create a terraform.tfvars file with following contents:

model_name ="test40"

Create a variables.tf file with following contents:

variable "model_name" {
  description = "Name of Juju model to deploy application to."
  type        = string
  default     = ""
}

variable "channel" {
  description = "The channel to use when deploying a charm."
  type        = string
  default     = "1.4/edge"
}

variable "db_application_name" {
  description = "The name of the application providing the `database` endpoint."
  type        = string
  default     = "mongodb-k8s"
}

variable "certs_application_name" {
  description = "Name of the application providing the `certificates` integration endpoint."
  type = string
  default = "self-signed-certificates"
}

variable "nrf_application_name" {
  description = "The name of the application providing the `fiveg_nrf` endpoint."
  type        = string
  default     = "nrf"
}

Fetch the PR and run following commads to install it.

make go-install
make install

Create a juju model:

juju add-model test40

Initialise the provider:

terraform init

Run Terraform plan by providing a var-file:

terraform plan -var-file="terraform.tfvars" 

Deploy the application:

terraform apply -auto-approve 

Check the pod to see the attached image.

microk8s.kubectl describe pod ausf-0 -n test40

Additional notes

Test with changing the resources part with different image versions.

Using a revision as string:

resources = {
      ausf-image = "30"
}

[hmlanigan]

Thank you for your contribution. It's a good start.

To proceed, please read and follow the process defined: https://github.com/juju/terraform-provider-juju/wiki/Developing#planning--design

Please also include description of what happens during the following scenarios.

• deploy
• refresh
• the resource element is removed from the plan
• what happens if the file isn't an image location?
• what happens if the value is changed from an int to a file?
• what happens if the value is changed from a file to an int?

[gatici]

Thank you for your contribution. It's a good start.

To proceed, please read and follow the process defined: https://github.com/juju/terraform-provider-juju/wiki/Developing#planning--design

Please also include description of what happens during the following scenarios.

* deploy

* refresh

* the resource element is removed from the plan

* what happens if the file isn't an image location?

* what happens if the value is changed from an int to a file?

* what happens if the value is changed from a file to an int?

Hello @hmlanigan,
The behaviour is summarized below. Please tell me if there is something undesired.

  * If the plan does not specify a resource and resources are added to the plan (as a revision number or a custom resource), specified resources are attached to the application (equivalent to juju attach-resource).
  * If the plan does specify resource revisions and if the charm revision or channel is updated, existing resources are kept.  (Resources are not detached)
  * If the plan does specify resource revisions and resources are removed from the plan:
          - If charm revision/channel is updated, the resources associated with the updated charm revision or channel  is attached.
          - If the charm revision/channel are not updated then the resources associated with the existing charm revision/channel are attached.
  * Charm could be deployed without resource, then resource could be  added later.
  * Resources could be provided in the following formats:
          - A custom repository URL
          - A files ends with .txt, .json  or .yaml
          - A resource revision from CharmHub
    * Charm could be deployed with resources.
    * If the provided resource revision does not exist during initial deployment or update, Client does not start deployment with an error that resource was not found in the store.
    * If the provided custom resource does not exist during initial deployment or update, Client  start deployment and charm could not be deployed properly and charm will be in error state.
    *  If the Provided resource type is not correct then Client fails with incorrect resource error for below scenarios:
          - An image is expected but file does not include image URL
          - A plain text file is expected but a json or yaml file is provided including image URL.
    *  If the provided resource does not exist then Client fails with path is not valid error.
    *  Changing resources from a file to an int or string does not create any problem. Resources are processed smoothly.
        -  If provided resource value is changed from an int to a file  then the image resource from the file is attached.
        -  If provided resource value is changed from a file to an int then image revision is attached.

[hmlanigan]

• If the plan does not specify a resource and resources are added to the plan (as a revision number or a custom resource), specified resources are attached to the application (equivalent to juju attach-resource).
• If the plan does specify resource revisions and if the charm revision or channel is updated, existing resources are kept. (Resources are not detached)

This is counter to current juju behavior. Today juju will upgrade the resources with the charm if a new resource is available, provided that the resource is not an attached local file (but not an ici-image).

This behavior would have to be explicitly implemented if we want it. It'd require a juju change.

Resources can never be detached by juju.

• If the plan does specify resource revisions and resources are removed from the plan:
  - If charm revision/channel is updated, the resources associated with the updated charm revision or channel is attached.
  - If the charm revision/channel are not updated then the resources associated with the existing charm revision/channel are attached.

A charm revision may be in more than one channel. The last I knew resources are associated with a channel only today, not a charm revision.

• Charm could be deployed without resource, then resource could be added later.

Juju will not allow a charm with resources to be deployed without one unless the process errors. If no resource is specified, juju deploy will find the resources in charm hub and use those.

• Resources could be provided in the following formats:
  - A custom repository URL

What is contained in this repository?

      - A files ends with .txt, .json  or .yaml

We will only be allowing revision number and oci-image information be used. We will not support custom files to be uploaded.

      - A resource revision from CharmHub
* Charm could be deployed with resources.
* If the provided resource revision does not exist during initial deployment or update, Client does not start deployment with an error that resource was not found in the store.

What will only work for versions of juju where DeployFromRepostory isn't used in older versions of juju.

* If the provided custom resource does not exist during initial deployment or update, Client  start deployment and charm could not be deployed properly and charm will be in error state.

This is dependent on the charms behavior.

*  If the Provided resource type is not correct then Client fails with incorrect resource error for below scenarios:
      - An image is expected but file does not include image URL
      - A plain text file is expected but a json or yaml file is provided including image URL.
*  If the provided resource does not exist then Client fails with path is not valid error.
*  Changing resources from a file to an int or string does not create any problem. Resources are processed smoothly.

The wording here is tricky. This will only work for oci-image resources. Once you have a file resource using a local upload you can never go back to the revisions numbers. However we will not be supporting local upload of a file resource with terraform.

    -  If provided resource value is changed from an int to a file  then the image resource from the file is attached.
    -  If provided resource value is changed from a file to an int then image revision is attached.

How hard is it to always provide a link to an image rather than handling the data in a file? e.g.

--resource \
     demo-server-image=ghcr.io/canonical/api_demo_server:1.0.1

[gatici]

The review comments are addressed. The resources are updated to latest revision associated with the charm channel upon channel updates. As summarized below, only image resources could be provided as custom resource inputs. Image information could be written directly in a string format or provided in a json or yaml file. This PR both works with deployFromRepository and legacyDeploy methods, so older Juju versions are supported.

Charms could be deployed with specifying the resources (from CharmHub or an OCI image repository). 
If the resources are not specified, the resources which are associated with the Charm in the CharmHub are used. 
Resource inputs are provided in a string format.
-         Resource revision number from CharmHub (string)
-         OCI image information as a URL (string)
-         A path of json or yaml file which includes OCI image repository information (string)

Changing resource input from a revision to a custom OCI resource is processed and updated smoothly according to the provided input.

Resources could be added to the Terraform plan after deployment.
-         If the resources are added to the plan (as a revision number or a custom OCI image resource), specified resources are attached to the application (equivalent to juju attach-resource).

Charm which includes resources could be updated. If the plan does specify resource revisions from CharmHub:
-         if the charm channel is updated, resources get updated to the latest revision associated with the updated channel. If the plan does specify custom OCI image resources:
-         if the charm channel is updated, existing resources are kept. (Resources are not detached)

Resources could be removed from the Terraform plan. If the plan does specify resource revisions from CharmHub or custom OCI images, then resources are removed from the plan:
-         If the charm channel is updated, resources get updated to the latest revision associated with the updated charm channel.
-         If the charm channel is not updated then the resources get updated to the latest revision associated with the existing charm channel.

[hmlanigan]

I'm part way through my review. However I've run out of time to complete it, this is what I have now. I had to spend some time refreshing my knowledge of how juju handles resources, it's a complex bit of code.

I see no need to add the json or yaml file option for oci-images as they can be directly listed in the plan as a URL. It also muddies the water around uploading file resources which we do not intend to support.

[jujubot]

Thanks for opening a pull request! Please follow the instructions here to ensure your pull request is ready for review. Then, a maintainer will review your patch.

@hpidcock @anvial

[jujubot]

Thanks for opening a pull request! Please follow the instructions here to ensure your pull request is ready for review. Then, a maintainer will review your patch.

@hpidcock @anvial

[hmlanigan]

Here are my current comments, though I haven't completed the review yet.

Mostly polish and nitpicks with 2 main questions/concerns. One is about not using the juju api resource client directly and the other is related to the validator logic. More detail with the lines.

Some of the changes are to make the code more consistent with already existing code style.

P.S I'm trying out using https://conventionalcomments.org

[hmlanigan]

QA went well with both k8s and machine charms and their resources.

[gatici]

Here are my current comments, though I haven't completed the review yet.

Mostly polish and nitpicks with 2 main questions/concerns. One is about not using the juju api resource client directly and the other is related to the validator logic. More detail with the lines.

Some of the changes are to make the code more consistent with already existing code style.

P.S I'm trying out using https://conventionalcomments.org

Hello Heather,

I have addressed some of the comments but some of them are still open. I will continue on Monday.
As you suggested in your review, new added ResourceHttpClient is removed. I figured out how to use Juju Resource API client for initial deployment stage. This required some additional info from charm and some helper methods are added under resources.go.

Regarding with the validator, I appreciate for your comments. I tried to improve the logic. I will also add the test cases for each scenario.

[hmlanigan]

LGTM! merging now.

[hmlanigan]

/merge

[hmlanigan]

The failing integration tests are timing out for taking too long. There will shortly be a PR to help with this.