Skip to content

Commit

Permalink
Merge pull request #23 from prajwalakhuj/azure
Browse files Browse the repository at this point in the history
Added Azure AKS & GKE support.
  • Loading branch information
RohitSquareops authored Sep 5, 2023
2 parents 43d13b8 + eff2d01 commit 228c9e1
Show file tree
Hide file tree
Showing 29 changed files with 510 additions and 76 deletions.
28 changes: 27 additions & 1 deletion IAM.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
## IAM Permission

The Policy required to deploy this module:
```hcl
```json
{
"Version": "2012-10-17",
"Statement": [
Expand Down Expand Up @@ -35,3 +35,29 @@ The Policy required to deploy this module:
]
}
```
## Azure Role Permissions

```hcl
permissions {
actions = [
"Microsoft.ManagedIdentity/userAssignedIdentities/delete",
"Microsoft.ManagedIdentity/userAssignedIdentities/read",
"Microsoft.ManagedIdentity/userAssignedIdentities/write",
"Microsoft.Resources/subscriptions/providers/read",
"Microsoft.Resources/subscriptions/resourcegroups/read"]
not_actions = []
}
```

## GCP IAM Permissions

```hcl
permissions = [
"iam.serviceAccounts.create",
"iam.serviceAccounts.delete",
"iam.serviceAccounts.get",
"iam.serviceAccounts.update",
"resourcemanager.projects.getIamPolicy",
"resourcemanager.projects.setIamPolicy"
]
```
35 changes: 21 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,26 +9,32 @@
This ECK module is a Kubernetes operator for Elasticsearch and Kibana that simplifies the deployment, management, and scaling of Elasticsearch and Kibana clusters in Kubernetes environments. The ECK module allows you to easily create and configure Elasticsearch and Kibana clusters, and provides customization options such as persistent volume claim templates and storage classes. Additionally, the ECK module provides security features such as encryption and authentication for Elasticsearch and Kibana clusters. With the ECK module, you can manage Elasticsearch and Kibana clusters in a scalable and efficient manner, while also ensuring the security of your data.

## Important Notes:
This module is compatible with EKS version 1.23,1.24,1.25 and 1.26 which is great news for users deploying the module on an EKS cluster running that version. Review the module's documentation, meet specific configuration requirements, and test thoroughly after deployment to ensure everything works as expected.
This module is compatible with EKS, AKS & GKE which is great news for users deploying the module on an AWS, Azure & GCP cloud. Review the module's documentation, meet specific configuration requirements, and test thoroughly after deployment to ensure everything works as expected.

## Supported Versions Table:

| Resources | Helm Chart Version | K8s supported version |
| Resources | Helm Chart Version | K8s supported version (EKS, AKS & GKE) |
| :-----: | :--- | :--- |
| Elastic-Operator | **2.7.0** | **1.23**,**1.24**,**1.25**,**1.26** |
| ECK | **7.17.3** | **1.23**,**1.24**,**1.25**,**1.26** |
| Elastalert2 | **2.9.0** | **1.23**,**1.24**,**1.25**,**1.26** |
| Elastic-Operator | **2.7.0** | **1.23**,**1.24**,**1.25**,**1.26**,**1.27** |
| ECK | **7.17.3** | **1.23**,**1.24**,**1.25**,**1.26**,**1.27** |
| Elastalert2 | **2.9.0** | **1.23**,**1.24**,**1.25**,**1.26**,**1.27** |


## Usage Example

```hcl
module "aws" {
source = "https://github.com/sq-ia/terraform-kubernetes-elastic.git//modules/resources/aws"
cluster_name = "prod-eks"
}
module "eck" {
source = "https://github.com/sq-ia/terraform-kubernetes-elastic.git"
cluster_name = "dev-cluster"
namespace = "elastic-system"
eck_config = {
hostname = "eck.squareops.in"
eck_values = ""
provider_type = "aws"
hostname = "eck.squareops.in"
eck_values = ""
master_node_sc = "gp2"
data_hot_node_sc = "gp2"
data_warm_node_sc = "gp2"
Expand All @@ -39,8 +45,9 @@ module "eck" {
master_node_count = 1
data_hot_node_count = 2
data_warm_node_count = 2
role_arn = module.aws.role_arn
}
exporter_enabled = true
elastalert_enabled = false
elastalert_config = {
slack_webhook_url = ""
Expand All @@ -50,7 +57,9 @@ module "eck" {
```
Refer [examples](https://github.com/sq-ia/terraform-kubernetes-elastic/tree/main/examples/complete) for more details.
- Refer [AWS examples](https://github.com/sq-ia/terraform-kubernetes-elastic/tree/main/examples/complete/aws) for more details.
- Refer [Azure examples](https://github.com/sq-ia/terraform-kubernetes-elastic/tree/main/examples/complete/azure) for more details.
- Refer [GCP examples](https://github.com/sq-ia/terraform-kubernetes-elastic/tree/main/examples/complete/gcp) for more details.

## IAM Permissions
The required IAM permissions to create resources from this module can be found [here](https://github.com/sq-ia/terraform-kubernetes-elastic/blob/main/IAM.md)
Expand All @@ -72,7 +81,6 @@ No requirements.

| Name | Version |
|------|---------|
| <a name="provider_aws"></a> [aws](#provider\_aws) | n/a |
| <a name="provider_helm"></a> [helm](#provider\_helm) | n/a |
| <a name="provider_kubernetes"></a> [kubernetes](#provider\_kubernetes) | n/a |
| <a name="provider_time"></a> [time](#provider\_time) | n/a |
Expand All @@ -85,15 +93,12 @@ No modules.

| Name | Type |
|------|------|
| [aws_iam_role.eck_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [helm_release.eck_operator](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource |
| [helm_release.elastalert](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource |
| [helm_release.elastic_stack](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource |
| [helm_release.elasticsearch_exporter](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource |
| [kubernetes_namespace.elastic_system](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) | resource |
| [time_sleep.wait_60_sec](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) | resource |
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
| [aws_eks_cluster.kubernetes_cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster) | data source |
| [kubernetes_secret.eck_secret](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/data-sources/secret) | data source |

## Inputs
Expand All @@ -108,6 +113,8 @@ No modules.
| <a name="input_elastalert_enabled"></a> [elastalert\_enabled](#input\_elastalert\_enabled) | Whether the Elastalert tool should be deployed along with the ECK stack or not. | `bool` | `false` | no |
| <a name="input_exporter_enabled"></a> [exporter\_enabled](#input\_exporter\_enabled) | Whether the ECK exporter should be deployed along with the ECK stack or not. | `bool` | `true` | no |
| <a name="input_namespace"></a> [namespace](#input\_namespace) | Name of the Kubernetes namespace where the ECK deployment will be deployed. | `string` | `"elastic-system"` | no |
| <a name="input_provider_type"></a> [provider\_type](#input\_provider\_type) | Choose what type of provider you want (aws, gcp) | `string` | `""` | no |
| <a name="input_role_arn"></a> [role\_arn](#input\_role\_arn) | The s3 bucket role arn for the aws bucket provider | `string` | `""` | no |

## Outputs

Expand Down
2 changes: 2 additions & 0 deletions examples/complete/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ No requirements.
| Name | Version |
|------|---------|
| <a name="provider_aws"></a> [aws](#provider\_aws) | n/a |
| <a name="provider_google"></a> [google](#provider\_google) | n/a |
| <a name="provider_azurerm"></a> [azurerm](#provider\_azurerm) | n/a |

## Modules

Expand Down
44 changes: 44 additions & 0 deletions examples/complete/aws/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
## AWS ECK Example
![squareops_avatar]

[squareops_avatar]: https://squareops.com/wp-content/uploads/2022/12/squareops-logo.png

### [SquareOps Technologies](https://squareops.com/) Your DevOps Partner for Accelerating cloud journey.
<br>
This example will be very useful for users who are new to a module and want to quickly learn how to use it. By reviewing the examples, users can gain a better understanding of how the module works, what features it supports, and how to customize it to their specific needs.

<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
## Requirements

No requirements.

## Providers

| Name | Version |
|------|---------|
| <a name="provider_aws"></a> [aws](#provider\_aws) | n/a |

## Modules

| Name | Source | Version |
|------|--------|---------|
| <a name="module_aws"></a> [aws](#module\_aws) | https://github.com/sq-ia/terraform-kubernetes-elastic.git//modules/resources/aws | n/a |
| <a name="module_eck"></a> [eck](#module\_eck) | https://github.com/sq-ia/terraform-kubernetes-elastic.git | n/a |

## Resources

| Name | Type |
|------|------|
| [aws_eks_cluster.cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster) | data source |
| [aws_eks_cluster_auth.cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) | data source |

## Inputs

No inputs.

## Outputs

| Name | Description |
|------|-------------|
| <a name="output_eck_credentials"></a> [eck\_credentials](#output\_eck\_credentials) | ECK\_Info |
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
File renamed without changes.
File renamed without changes.
42 changes: 42 additions & 0 deletions examples/complete/aws/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
locals {
name = "elastic"
region = "us-east-2"
environment = "prod"
additional_tags = {
Owner = "organization_name"
Expires = "Never"
Department = "Engineering"
}
}

module "aws" {
source = "https://github.com/sq-ia/terraform-kubernetes-elastic.git//modules/resources/aws"
cluster_name = ""
}

module "eck" {
source = "https://github.com/sq-ia/terraform-kubernetes-elastic.git"
namespace = "elastic-system"
eck_config = {
provider_type = "aws"
hostname = "eck.squareops.in"
eck_values = file("./helm/eck.yaml")
master_node_sc = "gp2"
data_hot_node_sc = "gp2"
data_warm_node_sc = "gp2"
master_node_size = "20Gi"
data_hot_node_size = "50Gi"
data_warm_node_size = "50Gi"
kibana_node_count = 1
master_node_count = 1
data_hot_node_count = 2
data_warm_node_count = 2
role_arn = module.aws.role_arn
}
exporter_enabled = true
elastalert_enabled = false
elastalert_config = {
slack_webhook_url = ""
elastalert_values = file("./helm/elastAlert.yaml")
}
}
File renamed without changes.
File renamed without changes.
42 changes: 42 additions & 0 deletions examples/complete/azure/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
## Azure ECK Example
![squareops_avatar]

[squareops_avatar]: https://squareops.com/wp-content/uploads/2022/12/squareops-logo.png

### [SquareOps Technologies](https://squareops.com/) Your DevOps Partner for Accelerating cloud journey.
<br>
This example will be very useful for users who are new to a module and want to quickly learn how to use it. By reviewing the examples, users can gain a better understanding of how the module works, what features it supports, and how to customize it to their specific needs.

<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
## Requirements

No requirements.

## Providers

| Name | Version |
|------|---------|
| <a name="provider_azurerm"></a> [azurerm](#provider\_azurerm) | n/a |

## Modules

| Name | Source | Version |
|------|--------|---------|
| <a name="module_eck"></a> [eck](#module\_eck) | https://github.com/sq-ia/terraform-kubernetes-elastic.git | n/a |

## Resources

| Name | Type |
|------|------|
| [azurerm_kubernetes_cluster.primary](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/data-sources/kubernetes_cluster) | data source |

## Inputs

No inputs.

## Outputs

| Name | Description |
|------|-------------|
| <a name="output_eck_credentials"></a> [eck\_credentials](#output\_eck\_credentials) | ECK\_Info |
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
9 changes: 9 additions & 0 deletions examples/complete/azure/helm/eck.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: "Monitor-Services"
operator: In
values:
- "true"
27 changes: 27 additions & 0 deletions examples/complete/azure/helm/elastAlert.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
rules:
slack_alert: |-
---
name: elast_alert
type: frequency
index: filebeat-*
num_events: 5
timeframe:
minutes: 5
filter:
- query:
query_string:
query: "http.response.status_code : 404"
alert:
- "slack"
slack:
slack_webhook_url: https://hooks.slack.com/services/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
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: "Monitor-Services"
operator: In
values:
- "true"
36 changes: 36 additions & 0 deletions examples/complete/azure/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
locals {
name = "elastic"
region = "eastus"
environment = "prod"
additional_tags = {
Owner = "organization_name"
Expires = "Never"
Department = "Engineering"
}
}

module "eck" {
source = "https://github.com/sq-ia/terraform-kubernetes-elastic.git"
namespace = "elastic-system"
eck_config = {
provider_type = "azure"
hostname = ""
eck_values = file("./helm/eck.yaml")
master_node_sc = "infra-service-sc"
data_hot_node_sc = "infra-service-sc"
data_warm_node_sc = "infra-service-sc"
master_node_size = "20Gi"
data_hot_node_size = "50Gi"
data_warm_node_size = "50Gi"
kibana_node_count = 1
master_node_count = 1
data_hot_node_count = 1
data_warm_node_count = 1
}
exporter_enabled = false
elastalert_enabled = false
elastalert_config = {
slack_webhook_url = ""
elastalert_values = file("./helm/elastAlert.yaml")
}
}
4 changes: 4 additions & 0 deletions examples/complete/azure/output.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
output "eck_credentials" {
description = "ECK_Info"
value = module.eck.eck
}
26 changes: 26 additions & 0 deletions examples/complete/azure/provider.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
provider "azurerm" {
features {}
}

data "azurerm_kubernetes_cluster" "primary" {
name = ""
resource_group_name = ""
}

provider "kubernetes" {
host = data.azurerm_kubernetes_cluster.primary.kube_config.0.host
username = data.azurerm_kubernetes_cluster.primary.kube_config.0.username
password = data.azurerm_kubernetes_cluster.primary.kube_config.0.password
client_certificate = base64decode(data.azurerm_kubernetes_cluster.primary.kube_config.0.client_certificate)
client_key = base64decode(data.azurerm_kubernetes_cluster.primary.kube_config.0.client_key)
cluster_ca_certificate = base64decode(data.azurerm_kubernetes_cluster.primary.kube_config.0.cluster_ca_certificate)
}

provider "helm" {
kubernetes {
host = data.azurerm_kubernetes_cluster.primary.kube_config.0.host
client_key = base64decode(data.azurerm_kubernetes_cluster.primary.kube_config.0.client_key)
client_certificate = base64decode(data.azurerm_kubernetes_cluster.primary.kube_config.0.client_certificate)
cluster_ca_certificate = base64decode(data.azurerm_kubernetes_cluster.primary.kube_config.0.cluster_ca_certificate)
}
}
Loading

0 comments on commit 228c9e1

Please sign in to comment.