feat: (IAC-1117) Dark Site deployment

viya4-iac-aws-darksite/README.md

@@ -0,0 +1,39 @@
+# Using the `viya4-iac-aws` GitHub project in a Dark Site Deployment Scenario (Experimental)
+
+### Contributors
+
+We thank the following individuals for technical assistance and their contributions for the documentation, scripts and yaml templates that provided the basis for this document.
+- Josh Coburn
+- Matthias Ender
+
+### Background for Dark Site Deployment Scenarios
+
+This documentation contains procedures that can be used to successfully complete a Dark Site deployment using elements of `viya4-iac-aws` GitHub project decribed below.
+
+#### Dark Site Deployment Target Recommendation
+**Note:** _It is recommended for a Dark Site deployment that you use a [BYON](https://github.com/sassoftware/viya4-iac-aws/blob/main/docs/user/BYOnetwork.md) or Bring Your Own Network configuration (scenario 3)  with IAC by creating your own VPC, subnets, AWS private endpoints, and network security groups versus letting `viya4-iac-aws` create these items for you.  If you rely on `viya4-iac-aws` to create the network infrastructure in a Dark Site scenario, you will likely run into failure points.  The procedures listed below assume that you use a BYON configuration scenario with IAC._
+
+### Procedures
+
+1.  **Build a Private VPC for EKS:**
+
+    - **Note:** _Creating AWS VPCs for an EKS cluster can be accomplished in many different ways. Some of those methods include using a CloudFormation template, the AWS console, the AWS CLI, or Terraform scripts._
+
+    - **Note**: If you have an existing VPC environment consisting of a private VPC and a public VPC (optional) that constitutes a Dark Site configuration, skip to the **_Create Custom AMI for Jumpserver/NFSServer_** in step #2 below.
+
+    - Background Information - AWS Reference documentation with details on how to create a VPC for any EKS cluster: https://docs.aws.amazon.com/eks/latest/userguide/creating-a-vpc.html
+
+    - For the cluster VPC ranges, `viya4-iac-aws` defaults to using a CIDR of /16. The `viya4-iac-aws` project creates a /21 VPC with both /22 and /28 sized subnets.  Excluding the `control_plane` subnets, the sizes were chosen by estimating the number of pods (doubling that number to account for viya updates), services, AWS overhead, and then adding a few hundred as a buffer.  
+
+2.  **Create Custom AMI for Jumpserver/NFSServer**
+    - The standard base AMI image used by `viya4-iac-aws` does not include the required NFS related Linux distributution packages.  Normally, `viya4-iac-aws` will attempt to install the NFS packages as part of the VM initialization.  In a Dark Site without access to Internet based resources, installation from an Internet based repository will not be possible.  To mitigate that issue, we'll need to create a custom AMI and then modify the local copy of our `viya4-iac-aws` repository to add references to that custom AMI as well as remove some of the initialization steps in the cloud-init files (for jumpserver and nfs-server). 
+
+    - Instructions to complete the custom AMI creation steps can be found [here](https://github.com/sassoftware/viya4-iac-aws/tree/feat/iac-1117/viya4-iac-aws-darksite/custom-ami/) [here](https://github.com/sassoftware/viya4-iac-aws/tree/main/viya4-aws-darksite/custom-ami/).
+
+3.  **Build tfvars:**
+    - The Terraform scripts create the necessary BYON subnet_ids, `viya4-iac-aws` can be configured to use those subnets for all the subnet_ids map BYON variables in the tfvars file (public, private, control_plane and database).
+    - Do not specify any public access cidrs in your tfvars file.
+    - Set `cluster_api_mode` to private.
+
+4.  **Deploy viya4-iac-aws:**
+    - Ensure that the deployment machine has a route and is allowed ingress to the cluster control plane before attempting a deployment. Executing a `kubectl` command such as `kubectl get nodes` from the deployment VM in the public VPC can be used to confirm connectivty to the EKS control plane.

viya4-iac-aws-darksite/custom-ami/README.md

@@ -0,0 +1,159 @@
+## AWS Darksite Custom AMI
+By default, viya4-iac-aws uses the base Canonical Ubuntu Server image, which does not have the required nfs-kernel-server package.  IAC installs these packages as part of the cloud-init process, however this is not possible in a dark site which has no outbound internet access.  Below are instructions for creating a custom AMI which will ensure that nfs-kernel-server packages are installed for both the jumpserver and NFS server.
+
+Once the AMI is created, you can then apply the mod to viya4-iac-aws by running the [darksite-iac-aws-mods.sh](https://gitlab.sas.com/jocobu/viya4-aws-darksite/-/tree/main/viya4-iac-aws/darksite-iac-aws-mods/darksite-iac-aws-mods.sh) script.
+
+## Notes
+This information was provided by [Frederik Vandenberghe](https://gitlab.sas.com/sbxfrv).  
+
+## Launch EC2-instance based on the current AMI in use
+First thing to do is launch an EC2-instance based on the AMI that's in use as AWS-base image for the nfs-server and the jump-server.  You can see which AMI that is in https://github.com/sassoftware/viya4-iac-aws/blob/main/modules/aws_vm/main.tf
+
+![](img/img1.png)
+
+It's an AMI owned by Canonical AWS account (Canonical is company behind Ubuntu) that provides an Ubuntu AWS AMI images.  So you can find that image in the AMI Catalog by searching with the same search criteria as specified in that main.tf file.
+
+![](img/img2.png) ![](img/img3.png)
+
+
+And exactly like in the main.tf (most_recent = true) select the most recent version and click on Launch Instance with AMI.
+
+![](img/img4.png)
+
+
+
+Give the new instance a name that you will remember.  I choosed m5.xlarge as instance type to stay in sync with the iac scripts.
+
+![](img/img5.png)
+
+For the keypair, you'll want to use a keypair you own and the same keypair you plan on using for viya4-iac-aws and viya4-deployment.  If the public key you'd like to use isn't in EC2 already.. import it now (EC2 > key pairs > import key pair).  In the screenshot below, I've already added it named "aws-key".
+
+<img src="img/img6.png" style="zoom:60%;" />
+
+Click launch instance.
+
+![](img/img6b.png)
+
+Wait for the initialization, check for the assigned IP address and setup SSH session to the running EC2-instance with your private SSH-key.  User to use will be ubuntu. 
+
+![](img/img7.png)
+
+
+
+## Install nfs-kernel-server on running EC2-instance
+
+Run the following commands on the Ubuntu EC2-instance.
+
+```shell
+sudo apt update
+
+sudo apt upgrade
+
+sudo apt install nfs-kernel-server
+```
+
+![](img/img8.png)
+
+
+
+![](img/img9.png)
+
+![](img/img10.png)
+
+
+
+You can now stop the EC2-instance.
+
+![](img/img11.png)
+
+
+
+## Create AMI AWS base image 
+
+Once the EC2-instance where you install nfs-kernel-server is stopped, you can create from that EC2-instance a AWS AMI image.  For that click right on the instance, select Image and templates, Create image.
+
+![](img/img12.png)
+
+
+
+Give the image a menaningful name and description and click Create image button.
+
+![](img/img13.png)
+
+![](img/img14.png)
+
+You can find your image here:
+
+![](img/img15.png)
+
+
+
+Search for private images. Initially it will have status pending.
+
+
+
+![](img/img16.png)
+
+
+
+Wait till it become available.
+
+
+
+![](img/img16b.png)
+
+
+
+## Clone viya4-iac-aws and mod so it's using private owned AMI as AWS base image for NFS and JUMP server
+
+I've provided a [helper script](https://gitlab.sas.com/jocobu/viya4-aws-darksite/-/tree/main/viya4-iac-aws/darksite-iac-aws-mods/darksite-iac-aws-mods.sh) to automatically mod the viya4-iac-aws clone.  However if you'd like to do this manually, here are the proceedures:
+
+
+Clone viya4-iac-aws
+
+https://github.com/sassoftware/viya4-iac-aws.git
+
+```shell
+git clone https://github.com/sassoftware/viya4-iac-aws.git
+```
+
+<img src="img/img17.png" style="zoom:60%;" />
+
+
+
+Go to $WORK_FOLDER/viya4-iac-aws/modules/aws_vm/main.tf
+
+In that file replace field owners AWS account with the AWS account where you created  the AMI.  Also change the name with the name you provided before as input while creating AWS image.  That way IaC will use new created AWS AMI image as base for NFS and JUMP server.
+
+![](img/img19.png)
+
+You can now adapt the cloud-init of NFS and JUMP server as follows.
+
+Go to $WORK_FOLDER/viya4-iac-aws/files/cloud-init/nfs/cloud-config and change as follows:
+
+1 Disable update and upgrade as there's no internet access
+
+2 Disable installation of nfs-kernel-server, again there's no internet access and we did before
+
+![](img/img20.png)
+
+3 Loop only over $public_subnet_cidrs as $private_subnet_cidrs will be the same when deploying with Josh his script on private VPC.
+
+<img src="img/img21.png" style="zoom:60%;" />
+
+Go to $WORK_FOLDER/viya4-iac-aws/files/cloud-init/jump/cloud-config and change as follows:
+
+1 Disable update and upgrade as there's no internet access
+
+2 Disable installation of nfs-common, again there's no internet access
+
+![](img/img22.png)
+
+
+
+Now you're ready to build your viya4-iac-aws docker. I would recommend to give it unique name so you remember why you customized it.
+
+```shell
+docker build -t viya4-iaw-aws-private:5.4.0 .
+```
+