Skip to content

Add valid string case check (#1335)#1347

Open
SyedIshmumAhnaf wants to merge 6 commits into
databrickslabs:mainfrom
SyedIshmumAhnaf:feat/valid-string-case-check
Open

Add valid string case check (#1335)#1347
SyedIshmumAhnaf wants to merge 6 commits into
databrickslabs:mainfrom
SyedIshmumAhnaf:feat/valid-string-case-check

Conversation

@SyedIshmumAhnaf

@SyedIshmumAhnaf SyedIshmumAhnaf commented Jul 19, 2026

Copy link
Copy Markdown

Changes

  • Add a has_valid_string_case row-level check for validating consistent string casing.
  • Support upper, lower, title, and sentence case modes.
  • Cast input values to strings before normalization and comparison so the check can handle non-string columns consistently.
  • Validate the case argument and reject unsupported values.
  • Register the check for declarative rule construction.
  • Add unit and integration test coverage for the new behavior.
  • Document the check and update the DQX check-definition agent skill.

Linked issues

Resolves #1335

Tests

  • manually tested
  • added unit tests
  • added integration tests
  • added end-to-end tests
  • added performance tests

Documentation and Demos

  • added/updated demos
  • added/updated docs
  • added/updated agent skills

@SyedIshmumAhnaf
SyedIshmumAhnaf requested a review from a team as a code owner July 19, 2026 18:41
@SyedIshmumAhnaf
SyedIshmumAhnaf requested review from nehamilak-db and removed request for a team July 19, 2026 18:41
@CLAassistant

CLAassistant commented Jul 19, 2026 •
edited
Loading

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@SyedIshmumAhnaf SyedIshmumAhnaf mentioned this pull request Jul 19, 2026
Open
1 task
@ghanse
ghanse self-requested a review July 20, 2026 12:25
ghanse
ghanse requested changes Jul 20, 2026
View reviewed changes

@ghanse ghanse left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good. Left a few small comments. Can you add some performance tests for this check?

Here's an example:

dqx/tests/perf/test_apply_checks.py

Lines 2144 to 2167 in 9c8fcaf

@pytest.mark.parametrize(
"column",
[
"col1_email_standard",
"col2_email_with_quoted_local_part",
"col3_email_with_ip_domain",
"col4_email_with_multi_part_domain",
],
)
@pytest.mark.benchmark(group="test_benchmark_is_valid_email")
def test_benchmark_is_valid_email(benchmark, ws, generated_email_df, column):
dq_engine = DQEngine(workspace_client=ws, extra_params=EXTRA_PARAMS)
checks = [
DQRowRule(
name=f"{column}_is_valid_email_address",
criticality="warn",
check_func=check_funcs.is_valid_email,
column=column,
),
]
benchmark.group += f" {column}"
checked = dq_engine.apply_checks(generated_email_df, checks)
actual_count = benchmark(lambda: checked.count())
assert actual_count == EXPECTED_ROWS

Comment thread docs/dqx/docs/reference/quality_checks.mdx Outdated
Comment thread skills/dqx-define-checks/SKILL.md Outdated
Comment thread tests/integration/test_row_checks.py Outdated
Comment thread src/databricks/labs/dqx/check_funcs.py Outdated
Comment thread src/databricks/labs/dqx/check_funcs.py Outdated
Comment thread tests/unit/test_build_rules.py Outdated
@ghanse ghanse added under-review This PR is currently being reviewed by one of DQX maintainers. needs-changes Changes required after review labels Jul 20, 2026
@SyedIshmumAhnaf

SyedIshmumAhnaf commented Jul 21, 2026

Copy link
Copy Markdown
Author

Thanks for the review and the helpful comments. I’ll work through the requested changes, add the performance test coverage, and update the PR accordingly.

@SyedIshmumAhnaf

SyedIshmumAhnaf commented Jul 23, 2026

Copy link
Copy Markdown
Author

Hi @ghanse, I’ve incorporated the feedback from the review into PR #1347.

The updates include the requested documentation changes, revised title-case behavior for uppercase abbreviations, updated integration expectations, use of an internal substring-length constant, consolidation of the rule-construction coverage, and performance tests for all four supported case modes.

The PR is ready for another review. Thanks!

@ghanse ghanse mentioned this pull request Jul 24, 2026
Open
ghanse
ghanse approved these changes Jul 25, 2026
View reviewed changes

@ghanse ghanse left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@SyedIshmumAhnaf FYI, I modified the sentence case to require uppercase for the 1st character of the sentence only. The remaining characters can have any case. I think this better matches the intent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ghanse ghanse ghanse approved these changes
@nehamilak-db nehamilak-db Awaiting requested review from nehamilak-db nehamilak-db is a code owner automatically assigned from databrickslabs/dqx-write

Assignees

No one assigned

Labels

needs-changes Changes required after review under-review This PR is currently being reviewed by one of DQX maintainers.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[FEATURE]: New check to validate string case

3 participants

@SyedIshmumAhnaf @CLAassistant @ghanse