Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release/0.6.0 #37

Merged
merged 3 commits into from
Feb 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
Version 0.6.0 (2024-02-02)
--------------------------
Update LICENSE to SLULA (#35)
Update version to v3.9.0 (#36)

Version 0.5.3 (2024-01-17)
--------------------------
Bump service module to v0.2.1 (#34)
Expand Down
257 changes: 56 additions & 201 deletions LICENSE

Large diffs are not rendered by default.

28 changes: 15 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ module "bad_1_stream" {
module "enrich_kinesis" {
source = "snowplow-devops/enrich-kinesis-ec2/aws"

accept_limited_use_license = true

name = "enrich-server"
vpc_id = var.vpc_id
subnet_ids = var.subnet_ids
Expand Down Expand Up @@ -100,6 +102,8 @@ EOF
module "enrich_kinesis" {
source = "snowplow-devops/enrich-kinesis-ec2/aws"

accept_limited_use_license = true

name = "enrich-server"
vpc_id = var.vpc_id
subnet_ids = var.subnet_ids
Expand Down Expand Up @@ -148,6 +152,8 @@ EOF
module "enrich_kinesis" {
source = "snowplow-devops/enrich-kinesis-ec2/aws"

accept_limited_use_license = true

name = "enrich-server"
vpc_id = var.vpc_id
subnet_ids = var.subnet_ids
Expand Down Expand Up @@ -195,7 +201,7 @@ module "enrich_kinesis" {
| <a name="module_instance_type_metrics"></a> [instance\_type\_metrics](#module\_instance\_type\_metrics) | snowplow-devops/ec2-instance-type-metrics/aws | 0.1.2 |
| <a name="module_kcl_autoscaling"></a> [kcl\_autoscaling](#module\_kcl\_autoscaling) | snowplow-devops/dynamodb-autoscaling/aws | 0.2.0 |
| <a name="module_service"></a> [service](#module\_service) | snowplow-devops/service-ec2/aws | 0.2.1 |
| <a name="module_telemetry"></a> [telemetry](#module\_telemetry) | snowplow-devops/telemetry/snowplow | 0.4.0 |
| <a name="module_telemetry"></a> [telemetry](#module\_telemetry) | snowplow-devops/telemetry/snowplow | 0.5.0 |

## Resources

Expand Down Expand Up @@ -245,7 +251,9 @@ module "enrich_kinesis" {
| <a name="input_ssh_key_name"></a> [ssh\_key\_name](#input\_ssh\_key\_name) | The name of the SSH key-pair to attach to all EC2 nodes deployed | `string` | n/a | yes |
| <a name="input_subnet_ids"></a> [subnet\_ids](#input\_subnet\_ids) | The list of subnets to deploy Enrich across | `list(string)` | n/a | yes |
| <a name="input_vpc_id"></a> [vpc\_id](#input\_vpc\_id) | The VPC to deploy Enrich within (must have DNS hostnames enabled) | `string` | n/a | yes |
| <a name="input_accept_limited_use_license"></a> [accept\_limited\_use\_license](#input\_accept\_limited\_use\_license) | Acceptance of the SLULA terms (https://docs.snowplow.io/limited-use-license-1.0/) | `bool` | `false` | no |
| <a name="input_amazon_linux_2_ami_id"></a> [amazon\_linux\_2\_ami\_id](#input\_amazon\_linux\_2\_ami\_id) | The AMI ID to use which must be based of of Amazon Linux 2; by default the latest community version is used | `string` | `""` | no |
| <a name="input_app_version"></a> [app\_version](#input\_app\_version) | App version to use. This variable facilitates dev flow, the modules may not work with anything other than the default value. | `string` | `"3.9.0"` | no |
| <a name="input_assets_update_period"></a> [assets\_update\_period](#input\_assets\_update\_period) | Period after which enrich assets should be checked for updates (e.g. MaxMind DB) | `string` | `"7 days"` | no |
| <a name="input_associate_public_ip_address"></a> [associate\_public\_ip\_address](#input\_associate\_public\_ip\_address) | Whether to assign a public ip address to this instance | `bool` | `true` | no |
| <a name="input_byte_limit"></a> [byte\_limit](#input\_byte\_limit) | The amount of bytes to buffer events before pushing them to Kinesis | `number` | `1000000` | no |
Expand Down Expand Up @@ -275,7 +283,7 @@ module "enrich_kinesis" {
| <a name="input_iam_permissions_boundary"></a> [iam\_permissions\_boundary](#input\_iam\_permissions\_boundary) | The permissions boundary ARN to set on IAM roles created | `string` | `""` | no |
| <a name="input_initial_position"></a> [initial\_position](#input\_initial\_position) | Where to start processing the input Kinesis Stream from (TRIM\_HORIZON or LATEST) | `string` | `"TRIM_HORIZON"` | no |
| <a name="input_instance_type"></a> [instance\_type](#input\_instance\_type) | The instance type to use | `string` | `"t3a.small"` | no |
| <a name="input_java_opts"></a> [java\_opts](#input\_java\_opts) | Custom JAVA Options | `string` | `"-Dorg.slf4j.simpleLogger.defaultLogLevel=info -XX:MinRAMPercentage=50 -XX:MaxRAMPercentage=75"` | no |
| <a name="input_java_opts"></a> [java\_opts](#input\_java\_opts) | Custom JAVA Options | `string` | `"-XX:InitialRAMPercentage=75 -XX:MaxRAMPercentage=75"` | no |
| <a name="input_kcl_read_max_capacity"></a> [kcl\_read\_max\_capacity](#input\_kcl\_read\_max\_capacity) | The maximum READ capacity for the KCL DynamoDB table | `number` | `10` | no |
| <a name="input_kcl_read_min_capacity"></a> [kcl\_read\_min\_capacity](#input\_kcl\_read\_min\_capacity) | The minimum READ capacity for the KCL DynamoDB table | `number` | `1` | no |
| <a name="input_kcl_write_max_capacity"></a> [kcl\_write\_max\_capacity](#input\_kcl\_write\_max\_capacity) | The maximum WRITE capacity for the KCL DynamoDB table | `number` | `10` | no |
Expand Down Expand Up @@ -305,25 +313,19 @@ module "enrich_kinesis" {

# Copyright and license

The Terraform AWS Enrich Kinesis on EC2 project is Copyright 2021-2023 Snowplow Analytics Ltd.

Licensed under the [Apache License, Version 2.0][license] (the "License");
you may not use this software except in compliance with the License.
Copyright 2021-current Snowplow Analytics Ltd.

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Licensed under the [Snowplow Limited Use License Agreement][license]. _(If you are uncertain how it applies to your use case, check our answers to [frequently asked questions][license-faq].)_

[release]: https://github.com/snowplow-devops/terraform-aws-enrich-kinesis-ec2/releases/latest
[release-image]: https://img.shields.io/github/v/release/snowplow-devops/terraform-aws-enrich-kinesis-ec2

[ci]: https://github.com/snowplow-devops/terraform-aws-enrich-kinesis-ec2/actions?query=workflow%3Aci
[ci-image]: https://github.com/snowplow-devops/terraform-aws-enrich-kinesis-ec2/workflows/ci/badge.svg

[license]: https://www.apache.org/licenses/LICENSE-2.0
[license-image]: https://img.shields.io/badge/license-Apache--2-blue.svg?style=flat
[license]: https://docs.snowplow.io/limited-use-license-1.0/
[license-image]: https://img.shields.io/badge/license-Snowplow--Limited--Use-blue.svg?style=flat
[license-faq]: https://docs.snowplow.io/docs/contributing/limited-use-license-faq/

[registry]: https://registry.terraform.io/modules/snowplow-devops/enrich-kinesis-ec2/aws/latest
[registry-image]: https://img.shields.io/static/v1?label=Terraform&message=Registry&color=7B42BC&logo=terraform
Expand Down
10 changes: 6 additions & 4 deletions main.tf
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
locals {
module_name = "enrich-kinesis-ec2"
module_version = "0.5.3"
module_version = "0.6.0"

app_name = "enrich-kinesis"
app_version = "3.8.0"
app_version = var.app_version

local_tags = {
Name = var.name
Expand All @@ -26,7 +26,7 @@ data "aws_caller_identity" "current" {}

module "telemetry" {
source = "snowplow-devops/telemetry/snowplow"
version = "0.4.0"
version = "0.5.0"

count = var.telemetry_enabled ? 1 : 0

Expand Down Expand Up @@ -373,7 +373,9 @@ locals {
})

user_data = templatefile("${path.module}/templates/user-data.sh.tmpl", {
config = local.hocon
accept_limited_use_license = var.accept_limited_use_license

config_b64 = base64encode(local.hocon)
version = local.app_version
resolver = "dynamodb:${data.aws_region.current.name}/${aws_dynamodb_table.config.name}/snowplow_resolver"
enrichments = "dynamodb:${data.aws_region.current.name}/${aws_dynamodb_table.config.name}/snowplow_enrichment_"
Expand Down
1 change: 1 addition & 0 deletions templates/config.hocon.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@
"autoGeneratedId": "${telemetry_auto_gen_id}"
"moduleName": "${telemetry_module_name}"
"moduleVersion": "${telemetry_module_version}"
"instanceId": $${INSTANCE_ID}
}

"featureFlags" : {
Expand Down
11 changes: 6 additions & 5 deletions templates/user-data.sh.tmpl
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
readonly CONFIG_DIR=/opt/snowplow/config

sudo mkdir -p $${CONFIG_DIR}
sudo cat << EOF > $${CONFIG_DIR}/enrich.hocon
${config}
sudo base64 --decode << EOF > $${CONFIG_DIR}/enrich.hocon
${config_b64}
EOF

sudo docker run \
Expand All @@ -19,9 +19,10 @@ sudo docker run \
--log-opt max-size=10m \
--log-opt max-file=5 \
%{ endif ~}
-v $${CONFIG_DIR}:/snowplow/config \
-e 'JAVA_OPTS=-Dconfig.override_with_env_vars=true ${java_opts}' \
-e "CONFIG_FORCE_telemetry_instanceId=$(get_instance_id)" \
--mount type=bind,source=$${CONFIG_DIR},target=/snowplow/config \
--env JDK_JAVA_OPTIONS='${java_opts}' \
--env ACCEPT_LIMITED_USE_LICENSE=${accept_limited_use_license} \
--env INSTANCE_ID=$(get_instance_id) \
snowplow/snowplow-enrich-kinesis:${version} \
--config /snowplow/config/enrich.hocon \
--iglu-config ${resolver} \
Expand Down
19 changes: 18 additions & 1 deletion variables.tf
Original file line number Diff line number Diff line change
@@ -1,8 +1,25 @@
variable "accept_limited_use_license" {
description = "Acceptance of the SLULA terms (https://docs.snowplow.io/limited-use-license-1.0/)"
type = bool
default = false

validation {
condition = var.accept_limited_use_license
error_message = "Please accept the terms of the Snowplow Limited Use License Agreement to proceed."
}
}

variable "name" {
description = "A name which will be pre-pended to the resources created"
type = string
}

variable "app_version" {
description = "App version to use. This variable facilitates dev flow, the modules may not work with anything other than the default value."
type = string
default = "3.9.0"
}

variable "vpc_id" {
description = "The VPC to deploy Enrich within (must have DNS hostnames enabled)"
type = string
Expand Down Expand Up @@ -104,7 +121,7 @@ variable "cloudwatch_logs_retention_days" {

variable "java_opts" {
description = "Custom JAVA Options"
default = "-Dorg.slf4j.simpleLogger.defaultLogLevel=info -XX:MinRAMPercentage=50 -XX:MaxRAMPercentage=75"
default = "-XX:InitialRAMPercentage=75 -XX:MaxRAMPercentage=75"
type = string
}

Expand Down
Loading