Enhancement: Add feature to manually retry failed task by yashpapa6969 · Pull Request #1417 · mher/flower

yashpapa6969 · 2024-12-26T07:57:25Z

Change Log

API Update: Introduced the TaskReapply endpoint to reapply tasks with original arguments.
Error Handling: Ensures JSON-serializable arguments and robust error logging.
Backend Utilities: Added helper functions (parse_args, parse_kwargs, make_json_serializable) for argument parsing and serialization.

iloveitaly · 2025-01-21T22:30:26Z

@mher anything I can do to help get this merged?

…d security

sc68cal

My concerns have been addressed, thank you for this contribution!

yashpapa6969 · 2025-02-02T17:03:05Z

@sc68cal how can i get this merged?

sc68cal · 2025-02-02T19:03:08Z

@sc68cal how can i get this merged?

I am not the maintainer, the maintainer needs to approve and merge

marcus-campos

Awesome!

Ragnarow · 2025-02-05T17:48:39Z

Thanks for the feature, @mher could we get this merged?

matias-martini · 2025-02-14T17:40:03Z

Hi! Thanks for adding this awesome feature @yashpapa6969 ! I tested it locally and noticed a small issue: I think we need to add a route for the new controller in flower/urls.py:

    (r"/api/task/reapply/(.+)", tasks.TaskReapply),

Otherwise the Retry button will fail with 404

yashpapa6969 · 2025-02-14T17:50:27Z

@matias-martini thank you updated

matias-martini · 2025-02-14T17:57:20Z

@yashpapa6969 Thanks for the updates! 🙌

Would you mind adding a test for the new API endpoint? It'll help ensure we cover the most common cases.

Ragnarow · 2025-03-18T22:27:48Z

@yashpapa6969 Thanks for the effort, can we have the tests to merge this?

the-rich-piana · 2025-08-05T16:05:12Z

Any update on this?

alexandercarruthers · 2025-08-18T11:16:24Z

Would be great to see this feature merged !

auvipy

this will also need unit tests to avoid regression

mihai-burduselu-ptt · 2025-09-09T15:14:12Z

@yashpapa6969, thank you so much for the effort! Could you help us with the tests for it so we can review and merge?

bondbenz · 2026-01-07T11:06:11Z

hello, any updates on this? thanks

Copilot

Pull request overview

This PR adds a new feature to manually retry failed tasks through the Flower UI and API. It introduces a /api/task/reapply/{taskid} endpoint that retrieves a failed task's original arguments and reapplies it as a new task.

Key Changes:

Added TaskReapply API endpoint to reapply tasks with their original arguments
Implemented utility functions (parse_args, parse_kwargs, make_json_serializable) to handle argument parsing and JSON serialization
Added a "Retry" button in the UI for tasks in FAILURE state

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
flower/api/tasks.py	Implements the new TaskReapply endpoint handler with error handling and task reapplication logic
flower/utils/tasks.py	Adds helper functions for parsing task arguments from various formats (JSON, Python literals) and ensuring JSON serializability
flower/urls.py	Registers the new /api/task/reapply endpoint route
flower/templates/task.html	Adds a "Retry" button for failed tasks in the task detail page
flower/static/js/flower.js	Implements client-side logic to handle retry button clicks and API communication
tests/unit/api/test_tasks.py	Adds comprehensive test coverage for the TaskReapply endpoint covering success, error cases, and edge cases

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-07T12:16:58Z

+        Get task info and reapply the task with the same arguments.
+
+        :param taskid: ID of the task to reapply.


Missing API documentation: The TaskReapply endpoint lacks comprehensive API documentation that other endpoints in this file have (such as TaskInfo). Consider adding proper docstring documentation including HTTP method, example request/response, parameters description, and status codes.

Suggested change

Get task info and reapply the task with the same arguments.

:param taskid: ID of the task to reapply.

Reapply a previously executed task using the same arguments.

This endpoint retrieves an existing task by its ID, extracts its original

positional and keyword arguments, and submits a new task with the same

name and arguments. A new task ID is returned for the re-applied task.

**Example request**:

.. sourcecode:: http

POST /api/task/reapply/7b3b9f52-1af3-4e0c-8a8a-5a5f9c2f8c64 HTTP/1.1

Host: localhost

Accept: application/json

Cookie: user=...

**Example response**:

.. sourcecode:: http

HTTP/1.1 200 OK

Content-Type: application/json

{

"task-id": "c1a2b3d4-5678-90ab-cdef-1234567890ab",

"state": "PENDING"

}

:param str taskid: ID of the original task to reapply.

:statuscode 200: task successfully reapplied; returns new task ID and state (if backend configured)

:statuscode 400: invalid task arguments or original task has no name

:statuscode 401: unauthorized request

:statuscode 404: unknown task ID or unknown task name

:statuscode 500: internal error while reapplying the task

Fixed in 3bc8db2. Added full API documentation in the same format as TaskInfo/TaskApply: example request/response and all status codes (200/400/401/404/500).

auvipy

please cross check the review comments carefully

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

yashpapa6969 · 2026-01-07T13:09:18Z

okay

Adiorz · 2026-03-03T12:43:00Z

Hi, any update on this one? Thanks

guandi · 2026-03-22T15:29:40Z

Looking for an update as well. This would be a super helpful feature.

- parse_args/parse_kwargs: single literal parser (json.loads then ast.literal_eval only - no code execution), input size cap, and strict result-type validation; truncated reprs (including '...') now raise ValueError instead of silently reapplying wrong arguments - remove unreachable stringified-tuple branch inside the JSON parse path - make_json_serializable: convert tuples/sets to lists and dates to isoformat; raise TypeError for values with no JSON equivalent - TaskReapply: full API docstring matching other endpoints, narrow exception handling for argument parsing (400), apply_async isolated in its own try block (500) - flower.js: guard missing task-id in reapply response; show a generic error alert and log details to the console instead of echoing raw server responses - task.html: fix elif indentation on the Retry button block - tests: cover truncated/non-list args, non-dict kwargs and non-serializable values; add direct unit tests for parse_args, parse_kwargs and make_json_serializable Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Follow-up hardening found by adversarial verification of the review fixes: - reject parsed strings ending in '...': celery's saferepr truncates long strings inside their quotes (e.g. "(7, 'Bearer...')" for a long token), which parses cleanly but would reapply the task with a corrupted argument - catch RecursionError/MemoryError from json.loads too, so pathologically nested input returns 400 instead of an unhandled 500 - make_json_serializable: raise TypeError for sets and non-string dict keys instead of silently changing the type the retried task receives - TaskReapply: use capp.send_task instead of the local task registry, so reapply works when flower runs without the application's task modules importable; preserve the original routing_key/exchange from the event so the retry lands on the same queue Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

yashpapa6969 · 2026-07-08T22:45:06Z

Pushed a second commit (301cf65) after adversarially re-verifying the review fixes end to end against a live broker/worker/flower stack:

Truncated args can no longer be reapplied silently. Celery's saferepr can truncate a long string inside its quotes — e.g. always_fail(7, 'Bearer <2000-char token>') is stored as (7, 'Bearer...'), which parses cleanly and would have retried the task with a corrupted 9-char token. Parsed strings ending in ... are now rejected with 400 (verified live). The trade-off — a legitimate string ending in ... is also rejected — is documented; a 400 is strictly safer than a silent wrong retry.
Pathologically nested input ('['*3000, under the size cap) raised an uncaught RecursionError from json.loads → 500; now a clean 400.
make_json_serializable is stricter: sets and non-string dict keys now raise TypeError (→ 400) instead of silently changing the argument types the retried task receives. (Correction to my earlier reply on the serialization thread: sets are rejected, not converted.)
TaskReapply now uses capp.send_task (same approach as TaskSendTask) instead of the local task registry, so the Retry button works when flower runs without the application's task modules importable — previously it would 404 for every task in that common deployment. The original routing_key/exchange from the event are preserved when present, so the retry lands on the same queue.

Verified: 207 unit tests + pylint 10/10 on a clean clone, plus live end-to-end (failed task → Retry button → new task executes with identical args; truncated-args tasks → 400; unknown id → 404).

Known limitation (inherent to reapplying from event data): custom argsrepr/kwargsrepr sanitization means the stored repr may be a placeholder for the real arguments; flower cannot detect that case.

Enhancement: Addfeature to manually retry failed task

81c2836

iloveitaly mentioned this pull request Jan 21, 2025

[FEATURE] Option to retry a failed task #1348

Open

sc68cal reviewed Jan 22, 2025

View reviewed changes

Comment thread flower/utils/tasks.py Outdated

Replace eval with ast.literal_eval in parse_args function for improve…

cf3b790

…d security

yashpapa6969 requested a review from sc68cal January 25, 2025 16:31

sc68cal reviewed Jan 26, 2025

View reviewed changes

Comment thread flower/utils/tasks.py Outdated

Security: refactored code

012e958

yashpapa6969 requested a review from sc68cal January 26, 2025 14:04

sc68cal approved these changes Jan 28, 2025

View reviewed changes

marcus-campos approved these changes Feb 3, 2025

View reviewed changes

fix adding route for the controller TaskReapply

bc0b8e3

symbiotech approved these changes Feb 21, 2025

View reviewed changes

alexandercarruthers approved these changes Aug 18, 2025

View reviewed changes

auvipy requested changes Aug 18, 2025

View reviewed changes

mdamire approved these changes Sep 22, 2025

View reviewed changes

alex-goswag approved these changes Sep 23, 2025

View reviewed changes

kelvin5290 approved these changes Dec 22, 2025

View reviewed changes

yashpapa6969 and others added 2 commits December 22, 2025 12:39

Merge branch 'mher:master' into master

6966ae8

Add unit tests for task reapplication functionality

323474e

yashpapa6969 requested a review from auvipy December 22, 2025 07:43

auvipy requested a review from Copilot January 7, 2026 12:11

Copilot started reviewing on behalf of auvipy January 7, 2026 12:11 View session

Copilot AI reviewed Jan 7, 2026

View reviewed changes

auvipy requested changes Jan 7, 2026

View reviewed changes

auvipy and others added 2 commits January 7, 2026 18:27

Update flower/utils/tasks.py

a550f13

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update flower/api/tasks.py

598c274

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

yashpapa6969 requested review from alex-goswag, alexandercarruthers, auvipy, kelvin5290, marcus-campos, mdamire, sc68cal and symbiotech July 8, 2026 22:24

		Get task info and reapply the task with the same arguments.

		:param taskid: ID of the task to reapply.

-        Get task info and reapply the task with the same arguments.
-        :param taskid: ID of the task to reapply.
+        Reapply a previously executed task using the same arguments.
+        This endpoint retrieves an existing task by its ID, extracts its original
+        positional and keyword arguments, and submits a new task with the same
+        name and arguments. A new task ID is returned for the re-applied task.
+        **Example request**:
+        .. sourcecode:: http
+           POST /api/task/reapply/7b3b9f52-1af3-4e0c-8a8a-5a5f9c2f8c64 HTTP/1.1
+           Host: localhost
+           Accept: application/json
+           Cookie: user=...
+        **Example response**:
+        .. sourcecode:: http
+           HTTP/1.1 200 OK
+           Content-Type: application/json
+           {
+             "task-id": "c1a2b3d4-5678-90ab-cdef-1234567890ab",
+             "state": "PENDING"
+           }
+        :param str taskid: ID of the original task to reapply.
+        :statuscode 200: task successfully reapplied; returns new task ID and state (if backend configured)
+        :statuscode 400: invalid task arguments or original task has no name
+        :statuscode 401: unauthorized request
+        :statuscode 404: unknown task ID or unknown task name
+        :statuscode 500: internal error while reapplying the task

Conversation

yashpapa6969 commented Dec 26, 2024

Change Log

Uh oh!

iloveitaly commented Jan 21, 2025

Uh oh!

Uh oh!

Uh oh!

sc68cal left a comment

Choose a reason for hiding this comment

Uh oh!

yashpapa6969 commented Feb 2, 2025

Uh oh!

sc68cal commented Feb 2, 2025

Uh oh!

marcus-campos left a comment

Choose a reason for hiding this comment

Uh oh!

Ragnarow commented Feb 5, 2025

Uh oh!

matias-martini commented Feb 14, 2025

Uh oh!

yashpapa6969 commented Feb 14, 2025

Uh oh!

matias-martini commented Feb 14, 2025

Uh oh!

Ragnarow commented Mar 18, 2025

Uh oh!

the-rich-piana commented Aug 5, 2025

Uh oh!

alexandercarruthers commented Aug 18, 2025

Uh oh!

auvipy left a comment

Choose a reason for hiding this comment

Uh oh!

mihai-burduselu-ptt commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bondbenz commented Jan 7, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Jan 7, 2026

Choose a reason for hiding this comment

Uh oh!

yashpapa6969 Jul 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

auvipy left a comment

Choose a reason for hiding this comment

Uh oh!

yashpapa6969 commented Jan 7, 2026

Uh oh!

Adiorz commented Mar 3, 2026

Uh oh!

guandi commented Mar 22, 2026

Uh oh!

yashpapa6969 commented Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

19 participants

mihai-burduselu-ptt commented Sep 9, 2025 •

edited

Loading