Feature: Baton migration for ArgoCD template changes

[csun-cpointe]

Description

Following #283, to deploy configuration store app before all other apps in Argocd, we have enabled both the Argocd sync wave and the auto sync for the children apps of the app so that after manually synchronizing the main app, the children app can be sync-ed automatically in the order defined by sync waves.

As part of that effort, we will include the migration script to enable the argocd sub-app auto sync feature on the templates so that when upgrade to baseline version 1.9.0, the apps can be deployed in the order defined in the sync wave.

DOD

Acceptance criteria required to realize the requested feature

• Write Baton migration script to update the downstream project argocd values.yaml to inlcude syncPolicy for create namespace, applyoutofsynconly, and automated for the sub app auto-sync
• Write Baton migration script to update the downstream project application templates to enable the syncPolicy configuration

BDD Scenario

Feature: Migrate a downstream project argocd template to include syncPolicy configuration
Scenario: Migrate a downstream project argocd template to include syncPolicy configuration

Test Strategy/Script

How will this feature be verified?

• Create a downstream project from baseline v1.8.0

mvn archetype:generate -U -DarchetypeGroupId=com.boozallen.aissemble \
  -DarchetypeArtifactId=foundation-archetype \
  -DarchetypeVersion=1.8.0 \
  -DgroupId=com.test \
  -DartifactId=test-313 \
  -DprojectGitUrl=test.url \
  -DprojectName=test-313 \
  && cd test-313

• Add the SparkPipeline.json file to the test-313-pipeline-models/src/main/resources/pipelines directory

• Run mvn clean install until all the manual actions are complete

• Once the manual actions are complete, run mvn clean install -Dmaven.build.cache.skipCache=true once to get any remaining manual actions

• Upgrade the project to v1.9.0-SNAPSHOT

  • Update the 1.8.0 to 1.9.0-SNAPSHOT in the Tiltfile and root pom.xml
  • Run mvn clean install -Dmaven.build.cache.skipCache once to trigger the migration script

• Verify the -deploy/src/main/resources/values.yaml file has the sync policy configuration

  syncPolicy:
    syncOptions:
      - CreateNamespace=true
      - ApplyOutOfSyncOnly=true
    automated: {}
  

• Verify the templates under -deploy/src/main/resources/templates/ has helm function to configure the syncPolicy

  {{- with .Values.spec.syncPolicy }}
  syncPolicy:
    {{- toYaml . | nindent 4 }}
  {{- end }}
  

• Add to the fermenter-mda plugin executions in test-313-deploy/pom.xml

<execution>
    <id>configuration-store</id>
    <phase>generate-sources</phase>
    <goals>
        <goal>generate-sources</goal>
    </goals>
    <configuration>
        <basePackage>com.boozallen.aissemble.test</basePackage>
        <profile>configuration-store-deploy-v2</profile>
        <!-- The property variables below are passed to the Generation Context and utilized
             to customize the deployment artifacts. -->
        <propertyVariables>
            <appName>configuration-store</appName>
        </propertyVariables>
    </configuration>
</execution>

• Run mvn clean install until all the manual actions are complete

• Standup the test project within the argocd to ensure the configuration store is deployed before other resources and metastore-service-config configmap is updated with configuration store value

  • In the test-313-deploy/src/main/resources/app/hive-metastore-service/templates/configmap.yaml file:
    add below content to the metadata:

    labels:
      aissemble-configuration-store: enabled
  

  • In the test-313/test-2213-deploy/src/main/resources/apps/hive-metastore-service/values.yaml file

  replace fs.s3a.access.key value `123` with `$getConfigValue(groupName=aws-credentials;propertyName=AWS_ACCESS_KEY_ID)`
  replace fs.s3a.secret.key value `456` with `$getConfigValue(groupName=aws-credentials;propertyName=AWS_SECRET_ACCESS_KEY)`
  

  • Download and unzip the attached 313-helper.zipand copy the src directory to the test-313 root directory
  • Modify the helm templates for Argocd deployment:
    • In the test-313-deploy/src/main/resources/values.yaml file, update the targetRevision and repo to reflect the correct value. e.g.:

    targetRevision: main
    repo: https://github.com/username/test-313
    

    • In the -deploy/src/main/resources/apps/configuration-store/values.yaml file, remove readOnly: true (ref: #309)
    • In the -deploy/src/main/resources/apps/configuration-store/ directory copy the content of the values-dev.yaml to the values-ci.yaml file
      • remove service and service.type from the values-ci.yaml file

        service:
            type: LoadBalancer
      

      • Mac User: In the -deploy/src/main/resources/apps/configuration-store/values-ci.yaml file update the volumePathOnNode to be /<pathToProject>/test-313/src/main/resources/configurations
      • Window User: In the -deploy/src/main/resources/values-ci.yaml file update the volumePathOnNode to be /mnt/c/Users/YOUR_USER/PATH/TO/283-helper/apps/configuration-store/src/main/resources/configurations
  • Create a repo for the project created
    • create a new test-313 repository from the github website
    • at the test-313 root directory run below commands
    • git init
    • git add .
    • git commit -m "init test-283 base
    • git branch -M main
    • git remote add origin https://github.com/<username>/<repo_name>.git
    • git push -u origina main
  • Start a local argocd server
    • helm install aissemble-infrastructure oci://ghcr.io/boozallen/aissemble-infrastructure-chart --version 1.9.0-SNAPSHOT --set jenkins.enabled=false --set ingress-nginx.enabled=false
    • Rancher Deskip -> open main window -> Port Forwarding => forward argocd-server to port 8081
    • argocd admin initial-password -n argocd (keep the password to login to argocd ui and cli console)
    • Open the argocd UI http://localhost:8081/ and login as (admin/previous generated password)
    • In the test-313 root directory run argocd login localhost:8081 (Use username: admin/password: previous generated password to login)
    • Run below command to deploy the test-313 app

        argocd app create test-313 \
         --dest-namespace argocd \
         --dest-server https://kubernetes.default.svc \
         --repo https://github.com/csun-cpointe/test-313 \
         --path test-313-deploy/src/main/resources \
         --revision main \
         --values values.yaml \
         --values values-ci.yaml
    

  • On the ArgoCD UI, verify that the test-313 app is shown
  • Sync test-313 app and verify that only the configuration-store app appears before the configuration-store status becomes healthy.

    argocd app sync test-313 --revision main
    

  • Wait until the hive-metastore-service status is healthy and click into the hive-metastore-service -> metastore-service-config and verify the fs.s3a.access.key and fs.s3a.secret.key values are injected

      <property>
        <name>fs.s3a.access.key</name>
        <value>env-access-key-id</value>
      </property>
      <property>
        <name>fs.s3a.secret.key</name>
        <value>ENC(5vpYAWu/7sWCedm54CI2LZQ5O1TRCSwNdm0YMcAXj2b4sE5rk86iks7ITxzPfksnbDjy7AvI8TreVqykXGy6zA==)</value>
      </property>
      
  

References/Additional Context

A clear and concise description of any alternative solutions or features you've considered.
Add any other context, links, or screenshots about the feature request here.

[csun-cpointe]

DoD completed with @carter-cundiff

[ewilkins-csi]

✅ OTS completed

[carter-cundiff]

Testing passed: