Skip to content

Optimize FindQuery.update() with partial writes - #843

Open
lh0156 wants to merge 7 commits into
redis:mainfrom
lh0156:agent/find-query-update-777
Open

Optimize FindQuery.update() with partial writes#843
lh0156 wants to merge 7 commits into
redis:mainfrom
lh0156:agent/find-query-update-777

Conversation

@lh0156

@lh0156 lh0156 commented Jul 26, 2026

Copy link
Copy Markdown

Summary

Implements #777 by updating matching records without materializing full Pydantic models.

What changed

  • Uses FT.SEARCH ... NOCONTENT to collect matching keys page by page.
  • Validates and serializes update values once, including Pydantic constraints and Redis OM storage conversions.
  • Uses partial HSET mappings for HashModel.
  • Uses path-level JSON.SET commands for JsonModel, including nested __ paths.
  • Preserves the use_transaction flag and returns the number of updated records.
  • Keeps query projections out of the key-only search and preserves KNN state when copying queries.
  • Documents the new return value.

Testing

  • pytest -q tests --ignore tests/test_benchmarks.py: 265 passed.
  • pytest -q tests/test_hash_model.py tests/test_json_model.py: 147 passed.
  • Generated sync tests for the new coverage plus Hash/JSON model tests: 157 passed.
  • ruff check and ruff format --check on changed source/tests.
  • python -m compileall -q aredis_om tests.
  • bandit -q -r aredis_om/model/model.py -s B608.

Redis Stack was run locally via Docker Compose for the integration tests; benchmark tests were excluded from the full run.


Note

Medium Risk
Bulk updates now bypass per-instance save() and use direct HSET/JSON.SET, which changes behavior for edge cases (validators, side effects) but is covered by serialization parity and TTL tests; incorrect key-only search or encoding could corrupt many records at once.

Overview
FindQuery.update() no longer loads full models and calls save() per match. It validates and serializes update values once, paginates FT.SEARCH with NOCONTENT to collect keys only (ignoring .only() projections), then applies batched partial writes and returns the number of keys updated.

For HashModel, updates use HSET mappings with the same encoding rules as save() (timestamps, vectors, booleans, etc.), and positive per-field TTLs are re-applied via HEXPIRE after HSET. For JsonModel, updates use path-level JSON.SET, including nested __ paths (e.g. address__city$.address.city). use_transaction still controls pipeline transaction mode.

FindQuery.dict() / copy() now retain knn state. Docs note the int return value. New unit and integration tests cover validation-before-search, pagination, JSON paths, Pydantic v1 nested field resolution, and TTL preservation on bulk hash updates.

Reviewed by Cursor Bugbot for commit 256a3c8. Bugbot is set up for automated code reviews on this repo. Configure here.

@lh0156
lh0156 marked this pull request as ready for review July 30, 2026 09:12
Comment thread aredis_om/model/model.py
Comment thread aredis_om/model/model.py Outdated
@abrookins

Copy link
Copy Markdown
Collaborator

Thanks for opening, @lh0156! I'll take a look ASAP.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

Reviewed by Cursor Bugbot for commit 256a3c8. Configure here.

Comment thread aredis_om/model/model.py
return {
key: ("1" if value else "0") if isinstance(value, bool) else value
for key, value in document.items()
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty HSET mapping when updating nullable fields to None

Medium Severity

When all update values for a HashModel are None (e.g., clearing Optional fields), the None filter in _serialize_update_values produces an empty dict. This empty dict is then passed to pipeline.hset(key, mapping={}), which raises a DataError from redis-py because HSET requires at least one field-value pair. The old code called model.save() which wrote the entire model, so the HSET mapping always contained non-None fields from other columns. The partial-write approach loses that safety net.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 256a3c8. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants