feat(o11y): add o11y terraform module for VM #5341
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -193,6 +193,12 @@ variable "enable_nat_gateway" { | |
| default = true | ||
| } | ||
|
|
||
| variable "enable_observability" { | ||
| description = "Whether to create the observability stack (VictoriaMetrics)." | ||
|
Collaborator
There was a problem hiding this comment. lets keep this description vendor agnostic.
hassaansaleem28 marked this conversation as resolved.
Outdated
|
||
| type = bool | ||
| default = false | ||
| } | ||
|
|
||
| variable "enable_rds_proxy" { | ||
| description = "Whether to create an RDS proxy." | ||
| type = bool | ||
|
|
@@ -285,6 +291,12 @@ variable "frontend_use_fargate_spot" { | |
| default = true | ||
| } | ||
|
|
||
| variable "observability_vm_image" { | ||
| description = "The VictoriaMetrics container image (including digest)." | ||
| type = string | ||
| default = "victoriametrics/victoria-metrics:v1.145.0@sha256:c014fb5a711d38cb24fd0673197592cd1394bb903dbb16aea565620c9c8a3d70" | ||
|
Collaborator
There was a problem hiding this comment. how are we tracking this image with dependabot? Also, we must make sure this image + digest is the same as the local setup, ideally single source of truth for both. See example -- Line 72 in cfdf1ce |
||
| } | ||
|
|
||
| variable "private_subnet_cidrs" { | ||
| description = "A list of CIDR blocks for the private subnets." | ||
| type = list(string) | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| <!-- BEGIN_TF_DOCS --> | ||
| ## Requirements | ||
|
|
||
| | Name | Version | | ||
| | ---- | ------- | | ||
| | <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | ~> 1.15.0 | | ||
|
|
||
| ## Providers | ||
|
|
||
| | Name | Version | | ||
| | ---- | ------- | | ||
| | <a name="provider_aws"></a> [aws](#provider\_aws) | n/a | | ||
|
|
||
| ## Modules | ||
|
|
||
| No modules. | ||
|
|
||
| ## Resources | ||
|
|
||
| | Name | Type | | ||
| | ---- | ---- | | ||
| | [aws_cloudwatch_log_group.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) | resource | | ||
| | [aws_ecs_cluster.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_cluster) | resource | | ||
| | [aws_ecs_cluster_capacity_providers.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_cluster_capacity_providers) | resource | | ||
| | [aws_ecs_service.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) | resource | | ||
| | [aws_ecs_task_definition.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_task_definition) | resource | | ||
| | [aws_efs_file_system.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/efs_file_system) | resource | | ||
| | [aws_efs_mount_target.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/efs_mount_target) | resource | | ||
| | [aws_iam_policy.ecs_task_execution_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource | | ||
| | [aws_iam_role.ecs_task_execution_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | | ||
| | [aws_iam_role_policy_attachment.ecs_task_execution_policy_attachment](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource | | ||
| | [aws_security_group.efs](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource | | ||
| | [aws_security_group.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource | | ||
| | [aws_security_group_rule.efs_from_vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource | | ||
| | [aws_security_group_rule.vm_egress_https](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource | | ||
| | [aws_security_group_rule.vm_ingest_from_apps](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource | | ||
| | [aws_security_group_rule.vm_to_efs](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource | | ||
|
|
||
| ## Inputs | ||
|
|
||
| | Name | Description | Type | Default | Required | | ||
| | ---- | ----------- | ---- | ------- | :------: | | ||
| | <a name="input_app_security_group_ids"></a> [app\_security\_group\_ids](#input\_app\_security\_group\_ids) | Security group IDs of the application tasks allowed to send metrics to VictoriaMetrics. | `list(string)` | n/a | yes | | ||
| | <a name="input_assign_public_ip"></a> [assign\_public\_ip](#input\_assign\_public\_ip) | Whether to assign a public IP to the VictoriaMetrics task. | `bool` | `false` | no | | ||
| | <a name="input_aws_region"></a> [aws\_region](#input\_aws\_region) | The AWS region where the module is deployed. | `string` | n/a | yes | | ||
| | <a name="input_common_tags"></a> [common\_tags](#input\_common\_tags) | A map of common tags to apply to all resources. | `map(string)` | `{}` | no | | ||
| | <a name="input_environment"></a> [environment](#input\_environment) | The environment (e.g., staging, production). | `string` | n/a | yes | | ||
| | <a name="input_kms_key_arn"></a> [kms\_key\_arn](#input\_kms\_key\_arn) | The ARN of the KMS key used to encrypt the EFS file system. | `string` | n/a | yes | | ||
| | <a name="input_log_retention_in_days"></a> [log\_retention\_in\_days](#input\_log\_retention\_in\_days) | The number of days to retain VictoriaMetrics container logs. | `number` | `90` | no | | ||
| | <a name="input_project_name"></a> [project\_name](#input\_project\_name) | The name of the project. | `string` | n/a | yes | | ||
| | <a name="input_subnet_ids"></a> [subnet\_ids](#input\_subnet\_ids) | The private subnet IDs for the EFS mount targets and the VictoriaMetrics task. | `list(string)` | n/a | yes | | ||
|
hassaansaleem28 marked this conversation as resolved.
|
||
| | <a name="input_vm_cpu"></a> [vm\_cpu](#input\_vm\_cpu) | The CPU units for the VictoriaMetrics Fargate task. | `number` | `512` | no | | ||
| | <a name="input_vm_image"></a> [vm\_image](#input\_vm\_image) | The VictoriaMetrics container image (including digest). | `string` | n/a | yes | | ||
| | <a name="input_vm_memory"></a> [vm\_memory](#input\_vm\_memory) | The memory (in MiB) for the VictoriaMetrics Fargate task. | `number` | `1024` | no | | ||
| | <a name="input_vm_port"></a> [vm\_port](#input\_vm\_port) | The port VictoriaMetrics listens on for ingest and queries. | `number` | `8428` | no | | ||
| | <a name="input_vm_retention_period"></a> [vm\_retention\_period](#input\_vm\_retention\_period) | The VictoriaMetrics data retention period (e.g., 12, 5y). | `string` | `"12"` | no | | ||
| | <a name="input_vpc_id"></a> [vpc\_id](#input\_vpc\_id) | The VPC ID where the VictoriaMetrics security group is created. | `string` | n/a | yes | | ||
|
|
||
| ## Outputs | ||
|
|
||
| | Name | Description | | ||
| | ---- | ----------- | | ||
| | <a name="output_efs_file_system_id"></a> [efs\_file\_system\_id](#output\_efs\_file\_system\_id) | The ID of the EFS file system backing VictoriaMetrics storage. | | ||
| | <a name="output_vm_cluster_name"></a> [vm\_cluster\_name](#output\_vm\_cluster\_name) | The name of the ECS cluster running VictoriaMetrics. | | ||
| | <a name="output_vm_security_group_id"></a> [vm\_security\_group\_id](#output\_vm\_security\_group\_id) | The ID of the VictoriaMetrics security group. | | ||
| <!-- END_TF_DOCS --> | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,258 @@ | ||
| terraform { | ||
|
hassaansaleem28 marked this conversation as resolved.
|
||
| required_version = "~> 1.15.0" | ||
|
|
||
| required_providers { | ||
| aws = { | ||
| source = "hashicorp/aws" | ||
| } | ||
| } | ||
| } | ||
|
Comment on lines
+4
to
+10
Collaborator
There was a problem hiding this comment. this needs a |
||
|
|
||
| locals { | ||
| name_prefix = "${var.project_name}-${var.environment}-observability" | ||
|
|
||
| vm_container_definition = { | ||
| command = [ | ||
| "-storageDataPath=/data", | ||
| "-retentionPeriod=${var.vm_retention_period}", | ||
| "-httpListenAddr=:${var.vm_port}", | ||
| ] | ||
| essential = true | ||
| healthCheck = { | ||
| command = ["CMD-SHELL", "wget --spider -q http://localhost:${var.vm_port}/health || exit 1"] | ||
| interval = 30 | ||
| retries = 3 | ||
| startPeriod = 30 | ||
| timeout = 5 | ||
| } | ||
| image = var.vm_image | ||
| logConfiguration = { | ||
| logDriver = "awslogs" | ||
| options = { | ||
| "awslogs-group" = aws_cloudwatch_log_group.vm.name | ||
| "awslogs-region" = var.aws_region | ||
| "awslogs-stream-prefix" = "ecs" | ||
| } | ||
| } | ||
| mountPoints = [ | ||
| { | ||
| containerPath = "/data" | ||
| readOnly = false | ||
| sourceVolume = "vm-data" | ||
| } | ||
| ] | ||
| name = "victoriametrics" | ||
| portMappings = [ | ||
| { | ||
| containerPort = var.vm_port | ||
| hostPort = var.vm_port | ||
| protocol = "tcp" | ||
| } | ||
| ] | ||
| } | ||
| } | ||
|
|
||
| resource "aws_security_group" "vm" { | ||
| description = "Security group for the VictoriaMetrics task" | ||
| name = "${local.name_prefix}-vm-sg" | ||
| tags = merge(var.common_tags, { | ||
| Name = "${local.name_prefix}-vm-sg" | ||
| }) | ||
| vpc_id = var.vpc_id | ||
| } | ||
|
|
||
| resource "aws_security_group_rule" "vm_ingest_from_apps" { | ||
|
hassaansaleem28 marked this conversation as resolved.
|
||
| for_each = toset(var.app_security_group_ids) | ||
|
hassaansaleem28 marked this conversation as resolved.
Outdated
Collaborator
There was a problem hiding this comment. this breaks on an empty state apply. This is localstack logs but same will happen in prod/staging: It is because Example: |
||
|
|
||
| description = "Allow metrics ingest and queries from application tasks" | ||
| from_port = var.vm_port | ||
| protocol = "tcp" | ||
| security_group_id = aws_security_group.vm.id | ||
| source_security_group_id = each.value | ||
| to_port = var.vm_port | ||
| type = "ingress" | ||
| } | ||
|
|
||
| resource "aws_security_group_rule" "vm_egress_https" { | ||
| cidr_blocks = ["0.0.0.0/0"] | ||
| description = "Allow HTTPS egress for container image pulls" | ||
| from_port = 443 | ||
| protocol = "tcp" | ||
| security_group_id = aws_security_group.vm.id | ||
| to_port = 443 | ||
| type = "egress" | ||
| } | ||
|
hassaansaleem28 marked this conversation as resolved.
|
||
|
|
||
| resource "aws_security_group_rule" "vm_to_efs" { | ||
| description = "Allow NFS to the observability EFS" | ||
| from_port = 2049 | ||
| protocol = "tcp" | ||
| security_group_id = aws_security_group.vm.id | ||
| source_security_group_id = aws_security_group.efs.id | ||
| to_port = 2049 | ||
| type = "egress" | ||
| } | ||
|
|
||
| resource "aws_security_group" "efs" { | ||
| description = "Security group for the observability EFS file system" | ||
| name = "${local.name_prefix}-efs-sg" | ||
| tags = merge(var.common_tags, { | ||
| Name = "${local.name_prefix}-efs-sg" | ||
| }) | ||
| vpc_id = var.vpc_id | ||
| } | ||
|
|
||
| resource "aws_security_group_rule" "efs_from_vm" { | ||
|
hassaansaleem28 marked this conversation as resolved.
|
||
| description = "Allow NFS from the VictoriaMetrics task" | ||
| from_port = 2049 | ||
| protocol = "tcp" | ||
| security_group_id = aws_security_group.efs.id | ||
| source_security_group_id = aws_security_group.vm.id | ||
| to_port = 2049 | ||
| type = "ingress" | ||
| } | ||
|
|
||
| resource "aws_efs_file_system" "vm" { | ||
| encrypted = true | ||
| kms_key_id = var.kms_key_arn | ||
|
hassaansaleem28 marked this conversation as resolved.
hassaansaleem28 marked this conversation as resolved.
|
||
| tags = merge(var.common_tags, { | ||
| Name = "${local.name_prefix}-vm" | ||
| }) | ||
|
|
||
| lifecycle { | ||
| prevent_destroy = true | ||
| } | ||
| } | ||
|
|
||
| resource "aws_efs_mount_target" "vm" { | ||
|
cubic-dev-ai[bot] marked this conversation as resolved.
|
||
| for_each = toset(var.subnet_ids) | ||
|
|
||
| file_system_id = aws_efs_file_system.vm.id | ||
| security_groups = [aws_security_group.efs.id] | ||
| subnet_id = each.value | ||
| } | ||
|
|
||
| resource "aws_cloudwatch_log_group" "vm" { | ||
| kms_key_id = var.kms_key_arn | ||
| name = "/aws/ecs/${local.name_prefix}" | ||
| retention_in_days = var.log_retention_in_days | ||
| tags = merge(var.common_tags, { | ||
| Name = "${local.name_prefix}-logs" | ||
| }) | ||
| } | ||
|
|
||
| resource "aws_ecs_cluster" "vm" { | ||
| name = "${local.name_prefix}-cluster" | ||
| tags = merge(var.common_tags, { | ||
| Name = "${local.name_prefix}-cluster" | ||
| }) | ||
|
|
||
| setting { | ||
| name = "containerInsights" | ||
| value = "enabled" | ||
| } | ||
| } | ||
|
|
||
| resource "aws_ecs_cluster_capacity_providers" "vm" { | ||
| capacity_providers = ["FARGATE"] | ||
| cluster_name = aws_ecs_cluster.vm.name | ||
|
|
||
| default_capacity_provider_strategy { | ||
| base = 0 | ||
| capacity_provider = "FARGATE" | ||
| weight = 1 | ||
| } | ||
| } | ||
|
|
||
| resource "aws_iam_role" "ecs_task_execution_role" { | ||
| assume_role_policy = jsonencode({ | ||
| Version = "2012-10-17" | ||
| Statement = [ | ||
| { | ||
| Action = "sts:AssumeRole" | ||
| Effect = "Allow" | ||
| Principal = { | ||
| Service = "ecs-tasks.amazonaws.com" | ||
| } | ||
| } | ||
| ] | ||
| }) | ||
| name = "${local.name_prefix}-execution-role" | ||
| tags = var.common_tags | ||
| } | ||
|
|
||
| resource "aws_iam_policy" "ecs_task_execution_policy" { | ||
| description = "Policy for the VictoriaMetrics ECS task execution - CloudWatch Logs access." | ||
| name = "${local.name_prefix}-execution-policy" | ||
|
|
||
| policy = jsonencode({ | ||
| Version = "2012-10-17" | ||
| Statement = [ | ||
| { | ||
| Action = [ | ||
| "logs:CreateLogStream", | ||
| "logs:PutLogEvents" | ||
| ] | ||
| Effect = "Allow" | ||
| Resource = "${aws_cloudwatch_log_group.vm.arn}:*" | ||
| } | ||
| ] | ||
| }) | ||
| } | ||
|
|
||
| resource "aws_iam_role_policy_attachment" "ecs_task_execution_policy_attachment" { | ||
| policy_arn = aws_iam_policy.ecs_task_execution_policy.arn | ||
| role = aws_iam_role.ecs_task_execution_role.name | ||
| } | ||
|
|
||
| resource "aws_ecs_task_definition" "vm" { | ||
| container_definitions = jsonencode([local.vm_container_definition]) | ||
| cpu = var.vm_cpu | ||
| execution_role_arn = aws_iam_role.ecs_task_execution_role.arn | ||
| family = local.name_prefix | ||
| memory = var.vm_memory | ||
| network_mode = "awsvpc" | ||
| requires_compatibilities = ["FARGATE"] | ||
| runtime_platform { | ||
| cpu_architecture = "ARM64" | ||
| operating_system_family = "LINUX" | ||
| } | ||
| tags = merge(var.common_tags, { | ||
| Name = "${local.name_prefix}-task-def" | ||
| }) | ||
|
|
||
| volume { | ||
| name = "vm-data" | ||
|
|
||
| efs_volume_configuration { | ||
|
hassaansaleem28 marked this conversation as resolved.
|
||
| file_system_id = aws_efs_file_system.vm.id | ||
| transit_encryption = "ENABLED" | ||
| } | ||
| } | ||
|
hassaansaleem28 marked this conversation as resolved.
|
||
| } | ||
|
|
||
| resource "aws_ecs_service" "vm" { | ||
| cluster = aws_ecs_cluster.vm.id | ||
| deployment_maximum_percent = 100 | ||
| deployment_minimum_healthy_percent = 0 | ||
| desired_count = 1 | ||
| name = "${local.name_prefix}-service" | ||
| tags = merge(var.common_tags, { | ||
| Name = "${local.name_prefix}-service" | ||
| }) | ||
| task_definition = aws_ecs_task_definition.vm.arn | ||
|
|
||
| capacity_provider_strategy { | ||
| base = 0 | ||
| capacity_provider = "FARGATE" | ||
| weight = 1 | ||
| } | ||
|
|
||
| network_configuration { | ||
| assign_public_ip = var.assign_public_ip | ||
| security_groups = [aws_security_group.vm.id] | ||
| subnets = var.subnet_ids | ||
| } | ||
|
|
||
| depends_on = [aws_efs_mount_target.vm] | ||
| } | ||
|
hassaansaleem28 marked this conversation as resolved.
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.