fix(flow-producer): surface aborted transactions instead of silent fail - #4111
Open
mohanrajvenkatesan23-04 wants to merge 2 commits into
Open
Conversation
FlowProducer.add() and addBulk() ignored a `null` return from `multi.exec()`. ioredis returns null when the multi was aborted (e.g. issued against a READONLY replica, a WATCH violation, or a pipeline error), and the producer would happily resolve with a half-formed JobNode whose `job.id` was just the locally-generated UUID — pointing at nothing in Redis. The reporter saw exactly that shape under an Upstash failover window where Redis briefly went read-only. This commit: - For `add()` (single flow, atomic by design): throws on a null result and on the first per-command error. - For `addBulk()` (partial-success semantics, codified by an existing regression test): throws only on a null result. Per-command errors continue to leave that root's `job.id` unassigned so callers can detect partial failures by checking `await queue.getJob(tree.job.id)` — the same contract the existing "should not corrupt id mapping" test exercises. Adds two vitest spies on `client.multi` that override `exec()` to return null and asserts both code paths throw with a clear message. Fixes taskforcesh#3851
Contributor
|
@copilot resolve the merge conflicts in this pull request |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes a FlowProducer edge case where FlowProducer.add() / addBulk() could silently succeed when multi.exec() returns null (aborted transaction), returning a JobNode whose ID doesn’t correspond to anything in Redis.
Changes:
FlowProducer.add(): throws whenmulti.exec()returnsnulland now surfaces per-commandErrors fromexec()results.FlowProducer.addBulk(): throws whenmulti.exec()returnsnullwhile preserving existing partial-success semantics for per-command failures.- Adds regression tests that mock
multi.exec()to returnnulland assert both APIs throw.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
src/classes/flow-producer.ts |
Adds explicit handling for aborted transactions (exec() === null) and improves error surfacing behavior. |
tests/flow.test.ts |
Adds regression coverage for aborted transactions via a mocked multi.exec() returning null. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
|
@copilot resolve the merge conflicts in this pull request |
- Use explicit results === null check in add and addBulk so an empty exec() result is not misclassified as an aborted transaction - Use the public FlowProducer client getter in tests instead of reaching into (flow as any).connection.client
Contributor
Author
|
Thanks for the review! Pushed a follow-up commit addressing all four comments:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Port Impact Checklist
Why
Fixes #3851.
FlowProducer.add()andFlowProducer.addBulk()swallowedmulti.exec()returningnull. ioredis returnsnullfromPipeline.exec()whenever the transaction was aborted — for example issued against a READONLY replica, a WATCH conflict, or a pipeline-level error — and the producer would happily resolve with a half-formedJobNodewhosejob.idwas the locally-generated UUID, pointing at nothing in Redis.The reporter saw exactly that shape under an Upstash failover window where Redis is briefly read-only:
Queue.add()correctly throws in the same scenario; the bug was inflow-producer.ts:225and the parallel block inaddBulkat line 302.How
Two paths needed slightly different treatment because the API contract differs.
add()(single flow — atomic by design): throw whenresults === null, and throw the first per-command error if any. Without this, a transaction abort or a per-command failure leaves the user with a JobNode whose UUID id is unrecoverable.addBulk()(partial-success semantics): an existing regression test (should not corrupt id mapping for successful jobs when some addBulk commands fail) explicitly relies onaddBulkletting some root commands fail while others succeed. So we throw only onresults === null(the entire transaction aborted, every flow is dead). Per-command errors continue to leave that root'sjob.idunassigned, and callers detect partial failure viaawait queue.getJob(tree.job.id) === undefined— the same contract that test asserts.The error message names the cause:
Additional Notes (Optional)
tests/flow.test.tsuse a vitest spy on the nextclient.multi()call to swap in a multi whose.exec()returnsnull. Bothadd()andaddBulk()are asserted to throw/transaction was aborted/. The original Pipeline is otherwise pristine — no impact on neighbouring tests.should not corrupt id mapping...) is preserved unchanged and still passes against the new logic.