-
-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Fix: [New Check]: Detect secrets in AWS Batch job definitions #11836
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Deep070203
wants to merge
7
commits into
prowler-cloud:master
from
Deep070203:fix/auto-patch-1783100399107
Closed
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
d82a4db
feat(sdk): add AWS Amplify app secret scanning check
Deep070203 6a2754d
fix(sdk): resolve lazy payload generator bug in Amplify secrets check
Deep070203 bd3bd9f
chore(sdk): populate Amplify secrets check CLI remediation
Deep070203 25c19b7
Fix: [New Check]: Detect secrets in AWS Batch job definitions
7891c83
Merge branch 'master' into fix/auto-patch-1783100399107
Deep070203 9bd3d4c
docs(sdk): add docstring to batch job definition execute method
28dee53
test(sdk): resolve trufflehog scan alerts in amplify check tests
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
|
|
1 change: 1 addition & 0 deletions
1
prowler/providers/aws/services/amplify/amplify_app_no_secrets_in_environment/__init__.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
|
|
42 changes: 42 additions & 0 deletions
42
...amplify_app_no_secrets_in_environment/amplify_app_no_secrets_in_environment.metadata.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| { | ||
| "Provider": "aws", | ||
| "CheckID": "amplify_app_no_secrets_in_environment", | ||
| "CheckTitle": "Amplify app has no sensitive credentials in environment variables or build settings", | ||
| "CheckType": [ | ||
| "Software and Configuration Checks/AWS Security Best Practices", | ||
| "TTPs/Credential Access", | ||
| "Effects/Data Exposure", | ||
| "Sensitive Data Identifications/Security" | ||
| ], | ||
| "ServiceName": "amplify", | ||
| "SubServiceName": "", | ||
| "ResourceIdTemplate": "arn:partition:amplify:region:account-id:apps/app-id", | ||
| "Severity": "high", | ||
| "ResourceType": "AwsAmplifyApp", | ||
| "ResourceGroup": "security", | ||
| "Description": "AWS Amplify apps and their branches are inspected for hardcoded secrets, such as API keys, tokens, or passwords embedded in environment variables or build settings (buildSpec).", | ||
| "Risk": "Plaintext secrets in Amplify app environment variables or build configurations can be viewed by anyone with read access to the Amplify console, or may leak during the build process, exposing downstream resources and integrations.", | ||
| "RelatedUrl": "", | ||
| "AdditionalURLs": [ | ||
| "https://docs.aws.amazon.com/amplify/latest/userguide/environment-variables.html", | ||
| "https://docs.prowler.com/developer-guide/secret-scanning-checks" | ||
| ], | ||
| "Remediation": { | ||
| "Code": { | ||
| "CLI": "aws amplify update-app --app-id <app-id> --environment-variables <key1=value1,key2=value2>", | ||
| "NativeIaC": "", | ||
| "Other": "1. Access the AWS Amplify console.\n2. Navigate to your app settings, choose Environment variables, and check if any secrets are stored in plaintext.\n3. For actual secrets, migrate them to environment secrets or AWS Secrets Manager / Parameter Store and reference them securely during the build phase.\n4. Clean the variables or buildSpec configuration.", | ||
| "Terraform": "" | ||
| }, | ||
| "Recommendation": { | ||
| "Text": "Avoid storing secrets in plaintext environment variables or build specifications for AWS Amplify apps. Store sensitive settings securely in AWS Systems Manager Parameter Store or AWS Secrets Manager.", | ||
| "Url": "https://hub.prowler.com/check/amplify_app_no_secrets_in_environment" | ||
| } | ||
| }, | ||
| "Categories": [ | ||
| "secrets" | ||
| ], | ||
| "DependsOn": [], | ||
| "RelatedTo": [], | ||
| "Notes": "" | ||
| } |
105 changes: 105 additions & 0 deletions
105
...es/amplify/amplify_app_no_secrets_in_environment/amplify_app_no_secrets_in_environment.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| import json | ||
|
|
||
| from prowler.lib.check.models import Check, Check_Report_AWS | ||
| from prowler.lib.utils.utils import ( | ||
| SecretsScanError, | ||
| annotate_verified_secrets, | ||
| detect_secrets_scan_batch, | ||
| ) | ||
| from prowler.providers.aws.services.amplify.amplify_client import amplify_client | ||
|
|
||
|
|
||
| class amplify_app_no_secrets_in_environment(Check): | ||
| """Check that AWS Amplify apps contain no hardcoded secrets in their environment variables or build settings.""" | ||
|
|
||
| def execute(self) -> list[Check_Report_AWS]: | ||
| findings = [] | ||
| secrets_ignore_patterns = amplify_client.audit_config.get( | ||
| "secrets_ignore_patterns", [] | ||
| ) | ||
| validate = amplify_client.audit_config.get("secrets_validate", False) | ||
| apps = list(amplify_client.apps.values()) | ||
| line_context_by_app = {} | ||
|
|
||
| payloads_list = [] | ||
| for app_index, app in enumerate(apps): | ||
| payload, line_context = _build_app_payload(app) | ||
| line_context_by_app[app_index] = line_context | ||
| if payload: | ||
| payloads_list.append((app_index, payload)) | ||
|
|
||
| scan_error = None | ||
| try: | ||
| batch_results = detect_secrets_scan_batch( | ||
| payloads_list, | ||
| excluded_secrets=secrets_ignore_patterns, | ||
| validate=validate, | ||
| ) | ||
| except SecretsScanError as error: | ||
| batch_results = {} | ||
| scan_error = error | ||
|
|
||
| for app_index, app in enumerate(apps): | ||
| report = Check_Report_AWS(metadata=self.metadata(), resource=app) | ||
| report.resource_tags = app.tags | ||
| report.status = "PASS" | ||
| report.status_extended = f"No secrets found in Amplify app {app.name} environment variables or build settings." | ||
|
|
||
| line_context = line_context_by_app.get(app_index, {}) | ||
| if line_context: | ||
| if scan_error: | ||
| report.status = "MANUAL" | ||
| report.status_extended = ( | ||
| f"Could not scan Amplify app {app.name} environment variables " | ||
| f"for secrets: {scan_error}; manual review is required." | ||
| ) | ||
| findings.append(report) | ||
| continue | ||
|
|
||
| detect_secrets_output = batch_results.get(app_index) | ||
| if detect_secrets_output: | ||
| secrets_string = ", ".join( | ||
| [ | ||
| f"{secret['type']} in {line_context.get(secret['line_number'], 'environment variables/build settings')}" | ||
| for secret in detect_secrets_output | ||
| ] | ||
| ) | ||
| report.status = "FAIL" | ||
| report.status_extended = ( | ||
| f"Potential {'secrets' if len(detect_secrets_output) > 1 else 'secret'} " | ||
| f"found in Amplify app {app.name} environment variables or build settings -> {secrets_string}." | ||
| ) | ||
| annotate_verified_secrets(report, detect_secrets_output) | ||
|
|
||
| findings.append(report) | ||
| return findings | ||
|
|
||
|
|
||
| def _build_app_payload(app) -> tuple[str, dict[int, str]]: | ||
| """Build a line-oriented scan payload and map each line to a field context.""" | ||
| lines = [] | ||
| line_context = {} | ||
|
|
||
| def add_line(context: str, value: str) -> None: | ||
| if value is None: | ||
| return | ||
| lines.append(json.dumps({context: value})) | ||
| line_context[len(lines)] = context | ||
|
|
||
| # App environment variables | ||
| for var_name, var_value in app.environment_variables.items(): | ||
| add_line(f"app environment variable '{var_name}'", var_value) | ||
|
|
||
| # App buildSpec | ||
| if app.build_spec: | ||
| for idx, line in enumerate(app.build_spec.splitlines(), start=1): | ||
| add_line(f"app buildSpec line {idx}", line) | ||
|
|
||
| # Branch environment variables | ||
| for branch in app.branches: | ||
| for var_name, var_value in branch.environment_variables.items(): | ||
| add_line( | ||
| f"branch '{branch.name}' environment variable '{var_name}'", var_value | ||
| ) | ||
|
|
||
| return "\n".join(lines), line_context | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| from prowler.providers.aws.services.amplify.amplify_service import Amplify | ||
| from prowler.providers.common.provider import Provider | ||
|
|
||
| amplify_client = Amplify(Provider.get_global_provider()) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| from botocore.exceptions import ClientError | ||
| from pydantic.v1 import BaseModel, Field | ||
|
|
||
| from prowler.lib.logger import logger | ||
| from prowler.lib.scan_filters.scan_filters import is_resource_filtered | ||
| from prowler.providers.aws.lib.service.service import AWSService | ||
|
|
||
|
|
||
| class Branch(BaseModel): | ||
| """Represents an AWS Amplify App Branch.""" | ||
|
|
||
| name: str | ||
| arn: str | ||
| environment_variables: dict = Field(default_factory=dict) | ||
|
|
||
|
|
||
| class App(BaseModel): | ||
| """Represents an AWS Amplify App.""" | ||
|
|
||
| id: str | ||
| name: str | ||
| arn: str | ||
| region: str | ||
| environment_variables: dict = Field(default_factory=dict) | ||
| build_spec: str = "" | ||
| branches: list[Branch] = Field(default_factory=list) | ||
| tags: list[dict] = Field(default_factory=list) | ||
|
|
||
|
|
||
| class Amplify(AWSService): | ||
| """AWS Amplify service class.""" | ||
|
|
||
| def __init__(self, provider): | ||
| super().__init__(__class__.__name__, provider) | ||
| self.apps = {} | ||
| self.__threading_call__(self._list_apps) | ||
| if self.apps: | ||
| self.__threading_call__(self._list_branches, self.apps.values()) | ||
|
|
||
| def _list_apps(self, regional_client) -> None: | ||
| logger.info("Amplify - Listing apps...") | ||
| try: | ||
| list_apps_paginator = regional_client.get_paginator("list_apps") | ||
| for page in list_apps_paginator.paginate(): | ||
| for app in page.get("apps", []): | ||
| app_id = app.get("appId") | ||
| app_name = app.get("name") | ||
| app_arn = app.get("appArn") | ||
| if not self.audit_resources or is_resource_filtered( | ||
| app_arn, self.audit_resources | ||
| ): | ||
| tags = app.get("tags", {}) | ||
| tags_list = [tags] if tags else [] | ||
| self.apps[app_arn] = App( | ||
| id=app_id, | ||
| name=app_name, | ||
| arn=app_arn, | ||
| region=regional_client.region, | ||
| environment_variables=app.get("environmentVariables", {}), | ||
| build_spec=app.get("buildSpec", ""), | ||
| tags=tags_list, | ||
| ) | ||
| except ClientError as error: | ||
| logger.error( | ||
| f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}" | ||
| ) | ||
| except Exception as error: | ||
| logger.error( | ||
| f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}" | ||
| ) | ||
|
|
||
| def _list_branches(self, app: App) -> None: | ||
| logger.info(f"Amplify - Listing branches for app {app.name}...") | ||
| try: | ||
| regional_client = self.regional_clients[app.region] | ||
| list_branches_paginator = regional_client.get_paginator("list_branches") | ||
| for page in list_branches_paginator.paginate(appId=app.id): | ||
| for branch in page.get("branches", []): | ||
| branch_name = branch.get("branchName") | ||
| branch_arn = branch.get("branchArn") | ||
| app.branches.append( | ||
| Branch( | ||
| name=branch_name, | ||
| arn=branch_arn, | ||
| environment_variables=branch.get("environmentVariables", {}), | ||
| ) | ||
| ) | ||
| except ClientError as error: | ||
| logger.error( | ||
| f"{app.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}" | ||
| ) | ||
| except Exception as error: | ||
| logger.error( | ||
| f"{app.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}" | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| # AWS Batch Service |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| from prowler.providers.aws.services.batch.batch_service import Batch | ||
| from prowler.providers.common.provider import Provider | ||
|
|
||
| batch_client = Batch(Provider.get_global_provider()) |
1 change: 1 addition & 0 deletions
1
prowler/providers/aws/services/batch/batch_job_definition_no_secrets/__init__.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| # batch_job_definition_no_secrets init |
42 changes: 42 additions & 0 deletions
42
...vices/batch/batch_job_definition_no_secrets/batch_job_definition_no_secrets.metadata.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| { | ||
| "Provider": "aws", | ||
| "CheckID": "batch_job_definition_no_secrets", | ||
| "CheckTitle": "AWS Batch job definitions have no secrets in container properties environment variables or commands", | ||
| "CheckType": [ | ||
| "Software and Configuration Checks/AWS Security Best Practices", | ||
| "TTPs/Credential Access", | ||
| "Effects/Data Exposure", | ||
| "Sensitive Data Identifications/Security" | ||
| ], | ||
| "ServiceName": "batch", | ||
| "SubServiceName": "", | ||
| "ResourceIdTemplate": "arn:partition:batch:region:account-id:job-definition/job-definition-name:revision", | ||
| "Severity": "high", | ||
| "ResourceType": "AwsBatchJobDefinition", | ||
| "ResourceGroup": "security", | ||
| "Description": "AWS Batch job definitions are inspected for hardcoded secrets, such as API keys, tokens, or passwords embedded in container environment variables or command parameters.", | ||
| "Risk": "Plaintext secrets in AWS Batch job definitions can be viewed by anyone with read access to the Batch service, or may leak during job execution, exposing downstream resources and integrations.", | ||
| "RelatedUrl": "", | ||
| "AdditionalURLs": [ | ||
| "https://docs.aws.amazon.com/batch/latest/userguide/job_definitions.html", | ||
| "https://docs.prowler.com/developer-guide/secret-scanning-checks" | ||
| ], | ||
| "Remediation": { | ||
| "Code": { | ||
| "CLI": "aws batch register-job-definition --job-definition-name <job-definition-name> --type container --container-properties <json-properties>", | ||
| "NativeIaC": "", | ||
| "Other": "1. Access the AWS Batch console.\n2. Navigate to Job definitions, select your active job definition, and check if any secrets are stored in environment variables or command parameters.\n3. Create a new revision of the job definition without plaintext secrets.\n4. Use secure methods like AWS Systems Manager Parameter Store or AWS Secrets Manager to inject secrets into the container.", | ||
| "Terraform": "" | ||
| }, | ||
| "Recommendation": { | ||
| "Text": "Avoid storing secrets in plaintext environment variables or command parameters for AWS Batch job definitions. Store sensitive settings securely in AWS Systems Manager Parameter Store or AWS Secrets Manager.", | ||
| "Url": "https://hub.prowler.com/check/batch_job_definition_no_secrets" | ||
| } | ||
| }, | ||
| "Categories": [ | ||
| "secrets" | ||
| ], | ||
| "DependsOn": [], | ||
| "RelatedTo": [], | ||
| "Notes": "" | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value
Type hint doesn't match the None-handling contract.
value: strdeclares requiredstr, but the function explicitly early-returns onNone(line 84), meaning callers may legitimately passNone. Should beOptional[str].♻️ Proposed fix
📝 Committable suggestion
🧰 Tools
🪛 ast-grep (0.44.0)
[info] 85-85: use jsonify instead of json.dumps for JSON output
Context: json.dumps({context: value})
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
🤖 Prompt for AI Agents