Skip to content

Commit

Permalink
feat: generate helm values
Browse files Browse the repository at this point in the history
  • Loading branch information
Telemaco019 committed Aug 5, 2024
1 parent fbfed61 commit 6c9cb2e
Show file tree
Hide file tree
Showing 7 changed files with 303 additions and 33 deletions.
44 changes: 24 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,9 @@ Available on [Terraform Registry](https://registry.terraform.io/modules/nebuly-a

## Outputs

No outputs.
| Name | Description |
|------|-------------|
| <a name="output_helm_values"></a> [helm\_values](#output\_helm\_values) | The values.yaml file for installing Nebuly on the provisioned resources. |


## Inputs
Expand All @@ -53,6 +55,7 @@ No outputs.
| <a name="input_key_vault_soft_delete_retention_days"></a> [key\_vault\_soft\_delete\_retention\_days](#input\_key\_vault\_soft\_delete\_retention\_days) | The number of days that items should be retained for once soft-deleted. This value can be between 7 and 90 (the default) days. | `number` | `7` | no |
| <a name="input_location"></a> [location](#input\_location) | The region where to provision the resources. | `string` | n/a | yes |
| <a name="input_openai_api_key"></a> [openai\_api\_key](#input\_openai\_api\_key) | The API Key used for authenticating with OpenAI. | `string` | n/a | yes |
| <a name="input_platform_domain"></a> [platform\_domain](#input\_platform\_domain) | The domain on which the deployed Nebuly platform will be available. | `string` | n/a | yes |
| <a name="input_postgres_server_admin_username"></a> [postgres\_server\_admin\_username](#input\_postgres\_server\_admin\_username) | The username of the admin user of the PostgreSQL Server. | `string` | `"nebulyadmin"` | no |
| <a name="input_postgres_server_alert_rules"></a> [postgres\_server\_alert\_rules](#input\_postgres\_server\_alert\_rules) | The Azure Monitor alert rules to set on the provisioned PostgreSQL server. | <pre>map(object({<br> description = string<br> frequency = string<br> window_size = string<br> action_group_id = string<br> severity = number<br><br> criteria = optional(<br> object({<br> aggregation = string<br> metric_name = string<br> operator = string<br> threshold = number<br> })<br> , null)<br> dynamic_criteria = optional(<br> object({<br> aggregation = string<br> metric_name = string<br> operator = string<br> alert_sensitivity = string<br> })<br> , null)<br> }))</pre> | `{}` | no |
| <a name="input_postgres_server_high_availability"></a> [postgres\_server\_high\_availability](#input\_postgres\_server\_high\_availability) | High-availability configuration of the DB server. Possible values for mode are: SameZone or ZoneRedundant. | <pre>object({<br> enabled : bool<br> mode : string<br> standby_availability_zone : optional(string, null)<br> })</pre> | <pre>{<br> "enabled": true,<br> "mode": "SameZone"<br>}</pre> | no |
Expand All @@ -71,22 +74,23 @@ No outputs.
## Resources


- resource.azurerm_key_vault.main (/terraform-docs/main.tf#59)
- resource.azurerm_key_vault_secret.postgres_passwords (/terraform-docs/main.tf#272)
- resource.azurerm_key_vault_secret.postgres_users (/terraform-docs/main.tf#261)
- resource.azurerm_kubernetes_cluster_node_pool.linux_pools (/terraform-docs/main.tf#361)
- resource.azurerm_management_lock.postgres_server (/terraform-docs/main.tf#204)
- resource.azurerm_monitor_metric_alert.postgres_server_alerts (/terraform-docs/main.tf#212)
- resource.azurerm_postgresql_flexible_server.main (/terraform-docs/main.tf#130)
- resource.azurerm_postgresql_flexible_server_configuration.mandatory_configurations (/terraform-docs/main.tf#181)
- resource.azurerm_postgresql_flexible_server_configuration.optional_configurations (/terraform-docs/main.tf#174)
- resource.azurerm_postgresql_flexible_server_database.main (/terraform-docs/main.tf#196)
- resource.azurerm_postgresql_flexible_server_firewall_rule.main (/terraform-docs/main.tf#188)
- resource.azurerm_private_endpoint.key_vault (/terraform-docs/main.tf#85)
- resource.azurerm_role_assignment.key_vault_secret_officer__current (/terraform-docs/main.tf#115)
- resource.azurerm_role_assignment.key_vault_secret_user__aks (/terraform-docs/main.tf#110)
- resource.random_password.postgres_server_admin_password (/terraform-docs/main.tf#125)
- resource.tls_private_key.aks (/terraform-docs/main.tf#288)
- data source.azurerm_client_config.current (/terraform-docs/main.tf#47)
- data source.azurerm_resource_group.main (/terraform-docs/main.tf#44)
- data source.azurerm_subnet.aks_nodes (/terraform-docs/main.tf#49)
- resource.azurerm_key_vault.main (/terraform-docs/main.tf#63)
- resource.azurerm_key_vault_secret.openai_api_key (/terraform-docs/main.tf#127)
- resource.azurerm_key_vault_secret.postgres_passwords (/terraform-docs/main.tf#283)
- resource.azurerm_key_vault_secret.postgres_users (/terraform-docs/main.tf#272)
- resource.azurerm_kubernetes_cluster_node_pool.linux_pools (/terraform-docs/main.tf#372)
- resource.azurerm_management_lock.postgres_server (/terraform-docs/main.tf#215)
- resource.azurerm_monitor_metric_alert.postgres_server_alerts (/terraform-docs/main.tf#223)
- resource.azurerm_postgresql_flexible_server.main (/terraform-docs/main.tf#141)
- resource.azurerm_postgresql_flexible_server_configuration.mandatory_configurations (/terraform-docs/main.tf#192)
- resource.azurerm_postgresql_flexible_server_configuration.optional_configurations (/terraform-docs/main.tf#185)
- resource.azurerm_postgresql_flexible_server_database.main (/terraform-docs/main.tf#207)
- resource.azurerm_postgresql_flexible_server_firewall_rule.main (/terraform-docs/main.tf#199)
- resource.azurerm_private_endpoint.key_vault (/terraform-docs/main.tf#89)
- resource.azurerm_role_assignment.key_vault_secret_officer__current (/terraform-docs/main.tf#119)
- resource.azurerm_role_assignment.key_vault_secret_user__aks (/terraform-docs/main.tf#114)
- resource.random_password.postgres_server_admin_password (/terraform-docs/main.tf#136)
- resource.tls_private_key.aks (/terraform-docs/main.tf#299)
- data source.azurerm_client_config.current (/terraform-docs/main.tf#51)
- data source.azurerm_resource_group.main (/terraform-docs/main.tf#48)
- data source.azurerm_subnet.aks_nodes (/terraform-docs/main.tf#53)
12 changes: 12 additions & 0 deletions main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -401,3 +401,15 @@ resource "azurerm_kubernetes_cluster_node_pool" "linux_pools" {
]
}
}


# ------ Post provisioning ------ #
locals {
helm_values = templatefile(
"templates/helm-values.tpl.yaml",
{
platform_domain = var.platform_domain
},
)
}

5 changes: 5 additions & 0 deletions outputs.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
output "helm_values" {
value = local.helm_values
sensitive = true
description = "The values.yaml file for installing Nebuly on the provisioned resources."
}
238 changes: 238 additions & 0 deletions templates/helm-values.tpl.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,238 @@
strimzi:
enabled: true

imagePullSecrets:
- name: nebuly-docker-pull

backend:
image:
repository: "ghcr.io/nebuly-ai/nebuly-backend"
ingress:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: "/$2"
enabled: true
className: "nginx"
hosts:
- host: ${platform_domain}
paths:
- path: /backend(/|$)(.*)
pathType: Prefix

volumeMounts:
- name: secrets-store
mountPath: /mnt/secrets-store
readOnly: true
volumes:
- name: secrets-store
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: nebuly-platform

eventIngestion:
image:
repository: "ghcr.io/nebuly-ai/nebuly-event-ingestion"

rootPath: "/event-ingestion"

ingress:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: "/$2"
enabled: true
className: "nginx"
hosts:
- host: ${platform_domain}
paths:
- path: /event-ingestion(/|$)(.*)
pathType: Prefix

azureml:
enabled: false

aiModels:
registry: aws_s3
aws:
bucketName: "" # TODO

lionLinguist:
image:
repository: "ghcr.io/nebuly-ai/nebuly-lion-linguist"

modelsCache:
enabled: true
storageClassName: gp2-csi
accessModes:
- ReadWriteOnce

resources:
requests:
cpu: "1"
limits:
memory: 8Gi
nvidia.com/gpu: 1

affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: nebuly.com/accelerator
operator: In
values:
- nvidia-tesla-t4

volumeMounts:
- name: secrets-store
mountPath: /mnt/secrets-store
readOnly: true
volumes:
- name: secrets-store
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: nebuly-platform

ingestionWorker:
image:
repository: "ghcr.io/nebuly-ai/nebuly-ingestion-worker"

volumeMounts:
- name: secrets-store
mountPath: /mnt/secrets-store
readOnly: true
volumes:
- name: secrets-store
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: nebuly-platform

kafka:
external: false
zookeeper:
replicas: 1
storage:
class: gp2-csi
replicas: 1
config:
offsets.topic.replication.factor: 1
replica.selector.class: org.apache.kafka.common.replica.RackAwareReplicaSelector
storage:
class: gp2-csi
size: 32Gi
resources:
limits:
memory: 6Gi
requests:
memory: 6Gi

analyticDatabase:
server: "nbllabplatformanalytics.c72cu4g06kwz.us-east-1.rds.amazonaws.com"
name: "analytics"
existingSecret:
name: nebuly-platform-credentials
userKey: analytics-db-username
passwordKey: analytics-db-password

auth:
image:
repository: "ghcr.io/nebuly-ai/nebuly-tenant-registry"

postgresServer: "nbllabplatformauth.c72cu4g06kwz.us-east-1.rds.amazonaws.com"
postgresDatabase: "auth"
existingSecret:
name: nebuly-platform-credentials
postgresUserKey: auth-db-username
postgresPasswordKey: auth-db-password
jwtSigningKey: jwt-key

loginModes: "password"
microsoft:
enabled: false
redirectUri: https://${platform_domain}/backend/auth/oauth/microsoft/callback
tenantId: ""
existingSecret:
name: microsoft-oauth-credentials
clientIdKey: microsoft-oauth-client-id
clientSecretKey: microsoft-oauth-client-secret

volumeMounts:
- name: secrets-store
mountPath: /mnt/secrets-store
readOnly: true
volumes:
- name: secrets-store
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: nebuly-platform
ingress:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: "/auth/$2"
nginx.ingress.kubernetes.io/use-regex: "true"
enabled: true
className: "nginx"
hosts:
- host: ${platform_domain}
paths:
- path: "/backend/auth(/|$)(.*)"
pathType: Prefix

frontend:
image:
repository: "ghcr.io/nebuly-ai/nebuly-frontend"

rootUrl: https://${platform_domain}
backendApiUrl: https://${platform_domain}/backend
authApiUrl: https://${platform_domain}/backend/auth
ingress:
enabled: true
className: "nginx"
hosts:
- host: ${platform_domain}
paths:
- path: /
pathType: Prefix

openAi:
enabled: true
insightsGeneratorDeployment: gpt-4-1106-preview-nebuly
textEmbeddingsDeployment: ada
frustrationDetectionDeployment: gpt-4-1106-preview-nebuly
chatCompletionDeployment: gpt-4-1106-preview-nebuly
endpoint: https://openaixpanseaisandbox-useast2.openai.azure.com/

existingSecret:
name: nebuly-platform-credentials
apiKey: openai-api-key

bootstrap-aws:
enabled: true
ingress-nginx:
controller:
allowSnippetAnnotations: true
config:
http-snippet: |
server {
if ($http_x_forwarded_proto = 'http') {
return 301 https://$host$request_uri;
}
}
service:
targetPorts:
http: http
https: http
annotations:
service.beta.kubernetes.io/aws-load-balancer-security-groups: "sg-0a2684cf0282660e5"
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:us-east-1:533267425677:certificate/6775c5fc-4e7a-4d61-9a1f-35c8455655fa"
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
cluster-autoscaler:
rbac:
serviceAccount:
annotations:
eks.amazonaws.com/role-arn: "arn:aws:iam::533267425677:role/nbllabeks"
21 changes: 15 additions & 6 deletions tests/smoke_test.tftest.hcl
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ provider "azurerm" {
}

run "setup" {
module {
module {
source = "./tests/setup"
}

Expand All @@ -17,20 +17,29 @@ run "smoke_test_plan__default_values" {

variables {
resource_group_name = "rg-platform-inttest"
location = "EastUS"
location = "EastUS"
platform_domain = "intest.nebuly.ai"

# ------ PostgreSQL Database ------ #
postgres_server_networking = { }
postgres_server_networking = {}

# ------ Key Vault ------ #
key_vault_public_network_access_enabled = true

# ------ External Secrets ------ #
openai_api_key = "my-key"

# ------ AKS ------ #
aks_nodes_virtual_network_name = run.setup.azurerm_virtual_network.name
aks_nodes_subnet_name = run.setup.azurerm_subnet.name
aks_nodes_subnet_name = run.setup.azurerm_subnet.name

aks_net_profile_service_cidr = "10.32.0.0/24"
aks_net_profile_service_cidr = "10.32.0.0/24"
aks_net_profile_dns_service_ip = "10.32.0.10"
aks_cluster_admin_object_ids = []
aks_cluster_admin_object_ids = []
}

assert {
condition = strcontains(output.helm_values, var.platform_domain)
error_message = "Helm values incorrect: platform domain is not included."
}
}
7 changes: 0 additions & 7 deletions tests/terraform.auto.tfvars
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,3 @@ tags = {
"project" : "self-deploy"
}


### EKS ###
eks_kubernetes_version = "1.28"
eks_cluster_endpoint_public_access = true

### External secrets ###
openai_api_key = "my-key"
9 changes: 9 additions & 0 deletions variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,15 @@ variable "resource_group_name" {
type = string
description = "The name of the resource group where to provision the resources."
}
variable "platform_domain" {
type = string
description = "The domain on which the deployed Nebuly platform will be available."
validation {
condition = can(regex("(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\\.)+[a-z0-9][a-z0-9-]{0,61}[a-z0-9]", var.platform_domain))
error_message = "The domain name must be a valid domain (e.g., example.com)."
}
}


# ------ PostgreSQL Databases ------ #
variable "postgres_server_sku" {
Expand Down

0 comments on commit 6c9cb2e

Please sign in to comment.