Skip to content

Add is_valid_currency_code row-level check#1368

Open
SreeramaYeshwanthGowd wants to merge 3 commits into
databrickslabs:mainfrom
SreeramaYeshwanthGowd:feature/1340-is-valid-currency-code
Open

Add is_valid_currency_code row-level check#1368
SreeramaYeshwanthGowd wants to merge 3 commits into
databrickslabs:mainfrom
SreeramaYeshwanthGowd:feature/1340-is-valid-currency-code

Conversation

@SreeramaYeshwanthGowd

@SreeramaYeshwanthGowd SreeramaYeshwanthGowd commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Changes

Adds a new row-level check is_valid_currency_code(column, code_format="alphabetic") that validates values against the ISO 4217 currency codes. ISO 4217 defines a three-letter alphabetic code (e.g. USD) and a three-digit numeric code (e.g. 840); both are supported via the code_format argument ("alphabetic" default, or "numeric"). Validation is a case-sensitive membership test; null values pass with no violation reported.

The code lists are embedded as module-level frozensets generated from the ISO 4217 standard data in the pycountry package (dev-time generation; the regeneration steps are in a comment). The implementation follows the existing pattern-validator checks such as is_valid_email and is_valid_national_id.

Linked issues

Resolves #1340

Tests

  • manually tested

  • added unit tests

  • added integration tests

  • added end-to-end tests

  • added performance tests

  • Unit tests in tests/unit/test_row_checks.py cover the auto-generated column name (default and numeric) and code_format validation (None, non-string, unsupported value).

  • Integration tests in tests/integration/test_row_checks.py cover both the alphabetic and numeric formats (valid, invalid, case-sensitive, empty and null cases) plus column-expression input.

  • is_valid_currency_code is also added to tests/resources/all_row_checks.yaml and the "all checks" integration tests (test_apply_checks_all_checks_as_yaml, test_apply_checks_all_row_checks_as_yaml_with_streaming, and test_apply_checks_all_checks_using_classes).

  • A performance benchmark (test_benchmark_is_valid_currency_code) is added under tests/perf/.

Documentation and Demos

  • added/updated demos
  • added/updated docs
  • added/updated agent skills

docs/dqx/docs/reference/quality_checks.mdx gets a reference-table entry and YAML plus programmatic (DQRowRule) usage examples.

@SreeramaYeshwanthGowd
SreeramaYeshwanthGowd requested a review from a team as a code owner July 23, 2026 21:47
@SreeramaYeshwanthGowd
SreeramaYeshwanthGowd requested review from mwojtyczka and removed request for a team July 23, 2026 21:47
@SreeramaYeshwanthGowd
SreeramaYeshwanthGowd force-pushed the feature/1340-is-valid-currency-code branch from 384bfff to 4ee12bb Compare July 23, 2026 21:58
Comment thread src/databricks/labs/dqx/check_funcs.py Outdated
return _matches_pattern(column, DQPattern.EMAIL_ADDRESS)


# Active ISO 4217 alpha-3 currency codes, derived from the CLDR currency data bundled with Babel:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the contribution.

Active ISO 4217 alpha-3 currency codes,

I'm having hard times finding the notion of alpha-3 in ISO 4217 standard, are you sure it is present there? The standard supports alphanumeric and numeric codes, would it be possible to support both of lists?

derived from the CLDR currency data bundled with Babel:

It might be a better idea to use official resources for standard values list - https://www.iso.org/iso-4217-currency-codes.html.

Comment thread src/databricks/labs/dqx/check_funcs.py Outdated


@register_rule("row")
def is_valid_currency_code(column: str | Column) -> Column:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might not a bad idea to validate ISO 4217 currency alpha code and numeric code. Either by argument or as separate method.

Comment thread src/databricks/labs/dqx/check_funcs.py Outdated
(for example, *USD*, *EUR*, *JPY*).

Validation is a case-sensitive membership test against the set of active ISO 4217
alpha-3 currency codes. Historical codes and non-national special codes (for example

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I doubt the notion of alpha-3 is present in ISO-4217. Can you double check the standard or wiki page, please?

@SreeramaYeshwanthGowd
SreeramaYeshwanthGowd force-pushed the feature/1340-is-valid-currency-code branch from 4ee12bb to b063a52 Compare July 25, 2026 12:32
@SreeramaYeshwanthGowd

SreeramaYeshwanthGowd commented Jul 25, 2026

Copy link
Copy Markdown
Contributor Author

@IvannKurchenko thanks for the review, good points. You're right that ISO 4217 uses alphabetic and numeric codes rather than the alpha-3 term, so I fixed the wording and added a code_format argument to validate either the alphabetic or the numeric code. I also switched the embedded list to pycountry, which packages the official ISO 4217 standard data, and rebased on main.

@SreeramaYeshwanthGowd
SreeramaYeshwanthGowd force-pushed the feature/1340-is-valid-currency-code branch from b063a52 to b0fb282 Compare July 25, 2026 13:03
@SreeramaYeshwanthGowd

Copy link
Copy Markdown
Contributor Author

@IvannKurchenko following your earlier note on using official sources, I moved the currency code lists into resource files loaded with importlib.resources, and referenced the official ISO 4217 source (https://www.iso.org/iso-4217-currency-codes.html) in the docstring and docs. Both the alphabetic and numeric formats have unit and integration tests.

@IvannKurchenko IvannKurchenko left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for all your efforts. I don't have further feedback from my end. Please, wait for approval from other reviewers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE]: New check and generation for currency codes for ISO-4217

2 participants