feat(o11y): add Grafana and stable VM endpoint #5405
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| FROM grafana/grafana-oss:13.0.2@sha256:5dad0df181cb644a14e13617b913b261a54f7d4fd4510721dba420929f35bea2 | ||
|
|
||
| COPY docker-compose/local/grafana/provisioning /etc/grafana/provisioning | ||
| COPY docker-compose/local/grafana/dashboards /etc/grafana/dashboards | ||
|
|
||
| USER grafana | ||
|
Contributor
There was a problem hiding this comment. P3: The Prompt for AI agents |
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| FROM victoriametrics/victoria-metrics:v1.145.0@sha256:c014fb5a711d38cb24fd0673197592cd1394bb903dbb16aea565620c9c8a3d70 | ||
|
Check warning on line 1 in docker/victoriametrics/Dockerfile
|
||
|
Contributor
There was a problem hiding this comment. 🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/usr/bin/env bash
set -euo pipefail
rg -n -C 3 '^(FROM|USER)' docker/victoriametrics/Dockerfile || true
rg -n -C 4 'o11y-metrics|user:' docker-compose/local/compose.o11y.yaml || true
rg -n -C 4 '65532|user\s*=|efs|access_point' infrastructure/modules/observability --glob '*.tf' || true
rg -n -C 3 'DS-0002|victoriametrics' .trivyignore.yaml || trueRepository: OWASP/Nest Length of output: 10614 🏁 Script executed: #!/usr/bin/env bash
set -euo pipefail
printf '%s\n' '--- Dockerfile and local service ---'
cat -n docker/victoriametrics/Dockerfile
sed -n '30,55p' docker-compose/local/compose.o11y.yaml
printf '%s\n' '--- VictoriaMetrics image config ---'
repo='victoriametrics/victoria-metrics'
digest='sha256:c014fb5a711d38cb24fd0673197592cd1394bb903dbb16aea565620c9c8a3d70'
token="$(curl -fsSL "https://auth.docker.io/token?service=registry.docker.io&scope=repository:${repo}:pull" | jq -r .token)"
manifest="$(curl -fsSL \
-H "Authorization: Bearer ${token}" \
-H 'Accept: application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.v2+json' \
"https://registry-1.docker.io/v2/${repo}/manifests/${digest}")"
printf '%s\n' "$manifest" | jq '{mediaType, config, manifests: (.manifests // []) | map({platform, digest})}'
config_digest="$(printf '%s\n' "$manifest" | jq -r '.config.digest // empty')"
if [ -n "$config_digest" ]; then
curl -fsSL -H "Authorization: Bearer ${token}" \
"https://registry-1.docker.io/v2/${repo}/blobs/${config_digest}" |
jq '{config: {User, WorkingDir, Entrypoint, Cmd}, rootfs}'
else
printf '%s\n' 'The pinned digest is a multi-platform index; inspect each platform manifest config.'
printf '%s\n' "$manifest" | jq -r '.manifests[] | [.platform.os, .platform.architecture, .digest] | `@tsv`' |
while IFS=$'\t' read -r os arch platform_digest; do
platform_manifest="$(curl -fsSL \
-H "Authorization: Bearer ${token}" \
-H 'Accept: application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.v2+json' \
"https://registry-1.docker.io/v2/${repo}/manifests/${platform_digest}")"
platform_config="$(printf '%s\n' "$platform_manifest" | jq -r '.config.digest')"
printf '%s/%s %s\n' "$os" "$arch" "$platform_digest"
curl -fsSL -H "Authorization: Bearer ${token}" \
"https://registry-1.docker.io/v2/${repo}/blobs/${platform_config}" |
jq --arg platform "$os/$arch" '{platform: $platform, config: {User, WorkingDir, Entrypoint, Cmd}, rootfs}'
done
fiRepository: OWASP/Nest Length of output: 7516 Run the local VictoriaMetrics service as non-root. The pinned image has no default 🧰 Tools🪛 Checkov (3.3.9)[low] 1-1: Ensure that HEALTHCHECK instructions have been added to container images (CKV_DOCKER_2) [low] 1-1: Ensure that a user for the container has been created (CKV_DOCKER_3) 🪛 GitHub Check: SonarCloud Code Analysis[warning] 1-1: This image might run with "root" as the default user. Make sure it is safe here. 🤖 Prompt for AI AgentsSource: Linters/SAST tools |
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| <!-- BEGIN_TF_DOCS --> | ||
| ## Requirements | ||
|
|
||
| | Name | Version | | ||
| | ---- | ------- | | ||
| | <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | ~> 1.15.0 | | ||
| | <a name="requirement_aws"></a> [aws](#requirement\_aws) | ~> 6.53.0 | | ||
|
|
||
| ## Providers | ||
|
|
||
| | Name | Version | | ||
| | ---- | ------- | | ||
| | <a name="provider_aws"></a> [aws](#provider\_aws) | 6.53.0 | | ||
|
|
||
| ## Modules | ||
|
|
||
| No modules. | ||
|
|
||
| ## Resources | ||
|
|
||
| | Name | Type | | ||
| | ---- | ---- | | ||
| | [aws_cloudwatch_log_group.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) | resource | | ||
| | [aws_ecs_cluster.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_cluster) | resource | | ||
| | [aws_ecs_cluster_capacity_providers.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_cluster_capacity_providers) | resource | | ||
| | [aws_ecs_service.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) | resource | | ||
| | [aws_ecs_task_definition.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_task_definition) | resource | | ||
| | [aws_efs_access_point.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/efs_access_point) | resource | | ||
| | [aws_efs_file_system.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/efs_file_system) | resource | | ||
| | [aws_efs_mount_target.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/efs_mount_target) | resource | | ||
| | [aws_iam_policy.ecs_task_execution_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource | | ||
| | [aws_iam_role.ecs_task_execution_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | | ||
| | [aws_iam_role_policy_attachment.ecs_task_execution_policy_attachment](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource | | ||
| | [aws_security_group.efs](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource | | ||
| | [aws_security_group.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource | | ||
| | [aws_security_group_rule.efs_from_vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource | | ||
| | [aws_security_group_rule.vm_egress_https](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource | | ||
| | [aws_security_group_rule.vm_ingest_from_apps](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource | | ||
| | [aws_security_group_rule.vm_to_efs](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource | | ||
| | [aws_service_discovery_private_dns_namespace.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/service_discovery_private_dns_namespace) | resource | | ||
| | [aws_service_discovery_service.vm](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/service_discovery_service) | resource | | ||
|
|
||
| ## Inputs | ||
|
|
||
| | Name | Description | Type | Default | Required | | ||
| | ---- | ----------- | ---- | ------- | :------: | | ||
| | <a name="input_app_security_group_ids"></a> [app\_security\_group\_ids](#input\_app\_security\_group\_ids) | Security group IDs of the application tasks allowed to send metrics to VictoriaMetrics. | `list(string)` | n/a | yes | | ||
| | <a name="input_assign_public_ip"></a> [assign\_public\_ip](#input\_assign\_public\_ip) | Whether to assign a public IP to the VictoriaMetrics task. | `bool` | `false` | no | | ||
| | <a name="input_aws_region"></a> [aws\_region](#input\_aws\_region) | The AWS region where the module is deployed. | `string` | n/a | yes | | ||
| | <a name="input_common_tags"></a> [common\_tags](#input\_common\_tags) | A map of common tags to apply to all resources. | `map(string)` | `{}` | no | | ||
| | <a name="input_environment"></a> [environment](#input\_environment) | The environment (e.g., staging, production). | `string` | n/a | yes | | ||
| | <a name="input_kms_key_arn"></a> [kms\_key\_arn](#input\_kms\_key\_arn) | The ARN of the KMS key used to encrypt the EFS file system. | `string` | n/a | yes | | ||
| | <a name="input_log_retention_in_days"></a> [log\_retention\_in\_days](#input\_log\_retention\_in\_days) | The number of days to retain VictoriaMetrics container logs. | `number` | `90` | no | | ||
| | <a name="input_project_name"></a> [project\_name](#input\_project\_name) | The name of the project. | `string` | n/a | yes | | ||
| | <a name="input_subnet_ids"></a> [subnet\_ids](#input\_subnet\_ids) | The private subnet IDs for the EFS mount targets and the VictoriaMetrics task. | `list(string)` | n/a | yes | | ||
| | <a name="input_vm_cpu"></a> [vm\_cpu](#input\_vm\_cpu) | The CPU units for the VictoriaMetrics Fargate task. | `number` | `512` | no | | ||
| | <a name="input_vm_desired_count"></a> [vm\_desired\_count](#input\_vm\_desired\_count) | The number of VictoriaMetrics tasks to run (0 or 1; it is a single-node store). | `number` | `1` | no | | ||
| | <a name="input_vm_image"></a> [vm\_image](#input\_vm\_image) | The VictoriaMetrics container image (including digest). | `string` | n/a | yes | | ||
| | <a name="input_vm_memory"></a> [vm\_memory](#input\_vm\_memory) | The memory (in MiB) for the VictoriaMetrics Fargate task. | `number` | `1024` | no | | ||
| | <a name="input_vm_port"></a> [vm\_port](#input\_vm\_port) | The port VictoriaMetrics listens on for ingest and queries. | `number` | `8428` | no | | ||
| | <a name="input_vm_retention_period"></a> [vm\_retention\_period](#input\_vm\_retention\_period) | The VictoriaMetrics data retention period (e.g., 12, 5y). | `string` | `"12"` | no | | ||
| | <a name="input_vpc_id"></a> [vpc\_id](#input\_vpc\_id) | The VPC ID where the VictoriaMetrics security group is created. | `string` | n/a | yes | | ||
|
|
||
| ## Outputs | ||
|
|
||
| | Name | Description | | ||
| | ---- | ----------- | | ||
| | <a name="output_efs_file_system_id"></a> [efs\_file\_system\_id](#output\_efs\_file\_system\_id) | The ID of the EFS file system backing VictoriaMetrics storage. | | ||
| | <a name="output_vm_cluster_name"></a> [vm\_cluster\_name](#output\_vm\_cluster\_name) | The name of the ECS cluster running VictoriaMetrics. | | ||
| | <a name="output_vm_endpoint"></a> [vm\_endpoint](#output\_vm\_endpoint) | The private host:port endpoint for reaching VictoriaMetrics. | | ||
| | <a name="output_vm_security_group_id"></a> [vm\_security\_group\_id](#output\_vm\_security\_group\_id) | The ID of the VictoriaMetrics security group. | | ||
| <!-- END_TF_DOCS --> |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P2: The o11y-grafana build context is the repository root, but the repo-root
.dockerignoreonly excludese2e/caches and the e2e node_modules/playwright output (.dockerignorein-tree). The Dockerfile only needsdocker-compose/local/grafana/..., so everydocker compose buildfrommake run-o11yships the entire repo — includingfrontend/node_modules,backend/.venv, and.git— to the Docker daemon. In a normal dev checkout this makes the grafana build noticeably slow on every stack start. Consider scoping the context down todocker-compose/local(and adjusting the COPY paths in the Dockerfile tografana/...) or adding a dedicated.dockerignorefor this build so the context only contains the provisioning/dashboards subtree.Prompt for AI agents