Add is_valid_currency_code row-level check#1368
Conversation
384bfff to
4ee12bb
Compare
| return _matches_pattern(column, DQPattern.EMAIL_ADDRESS) | ||
|
|
||
|
|
||
| # Active ISO 4217 alpha-3 currency codes, derived from the CLDR currency data bundled with Babel: |
There was a problem hiding this comment.
Thank you for the contribution.
Active ISO 4217 alpha-3 currency codes,
I'm having hard times finding the notion of alpha-3 in ISO 4217 standard, are you sure it is present there? The standard supports alphanumeric and numeric codes, would it be possible to support both of lists?
derived from the CLDR currency data bundled with Babel:
It might be a better idea to use official resources for standard values list - https://www.iso.org/iso-4217-currency-codes.html.
|
|
||
|
|
||
| @register_rule("row") | ||
| def is_valid_currency_code(column: str | Column) -> Column: |
There was a problem hiding this comment.
It might not a bad idea to validate ISO 4217 currency alpha code and numeric code. Either by argument or as separate method.
| (for example, *USD*, *EUR*, *JPY*). | ||
|
|
||
| Validation is a case-sensitive membership test against the set of active ISO 4217 | ||
| alpha-3 currency codes. Historical codes and non-national special codes (for example |
There was a problem hiding this comment.
I doubt the notion of alpha-3 is present in ISO-4217. Can you double check the standard or wiki page, please?
4ee12bb to
b063a52
Compare
|
@IvannKurchenko thanks for the review, good points. You're right that ISO 4217 uses alphabetic and numeric codes rather than the alpha-3 term, so I fixed the wording and added a |
b063a52 to
b0fb282
Compare
|
@IvannKurchenko following your earlier note on using official sources, I moved the currency code lists into resource files loaded with importlib.resources, and referenced the official ISO 4217 source (https://www.iso.org/iso-4217-currency-codes.html) in the docstring and docs. Both the alphabetic and numeric formats have unit and integration tests. |
IvannKurchenko
left a comment
There was a problem hiding this comment.
Thank you for all your efforts. I don't have further feedback from my end. Please, wait for approval from other reviewers.
Changes
Adds a new row-level check
is_valid_currency_code(column, code_format="alphabetic")that validates values against the ISO 4217 currency codes. ISO 4217 defines a three-letter alphabetic code (e.g.USD) and a three-digit numeric code (e.g.840); both are supported via thecode_formatargument ("alphabetic"default, or"numeric"). Validation is a case-sensitive membership test; null values pass with no violation reported.The code lists are embedded as module-level frozensets generated from the ISO 4217 standard data in the
pycountrypackage (dev-time generation; the regeneration steps are in a comment). The implementation follows the existing pattern-validator checks such asis_valid_emailandis_valid_national_id.Linked issues
Resolves #1340
Tests
manually tested
added unit tests
added integration tests
added end-to-end tests
added performance tests
Unit tests in
tests/unit/test_row_checks.pycover the auto-generated column name (default and numeric) andcode_formatvalidation (None, non-string, unsupported value).Integration tests in
tests/integration/test_row_checks.pycover both the alphabetic and numeric formats (valid, invalid, case-sensitive, empty and null cases) plus column-expression input.is_valid_currency_codeis also added totests/resources/all_row_checks.yamland the "all checks" integration tests (test_apply_checks_all_checks_as_yaml,test_apply_checks_all_row_checks_as_yaml_with_streaming, andtest_apply_checks_all_checks_using_classes).A performance benchmark (
test_benchmark_is_valid_currency_code) is added undertests/perf/.Documentation and Demos
docs/dqx/docs/reference/quality_checks.mdxgets a reference-table entry and YAML plus programmatic (DQRowRule) usage examples.