[ska2704]

What does this PR do?

Fixes #787. This follows up on the batch failure reported in #340.

An exception from one request's decode_request or encode_response currently makes every request in its batch fail. Handle these errors per request so valid siblings still receive their results.

Only successfully decoded inputs reach batch, predict, and unbatch, with their contexts kept in the same order. If every decode fails, prediction is skipped. Errors in these shared operations still fail the surviving batch. Tracking requests that still need a response prevents a later shared failure from sending a second error to an already rejected request.

The API docstrings describe the behavior. Tests cover HTTP and ordinary exceptions, context alignment, all-invalid input, shared-stage failures, and concurrent HTTP requests through a worker. The HTTP tests check the actual prediction batch size so they cannot pass by processing requests individually.

Validation:

• Unchanged main: 52 existing batch/loop tests passed. All 10 final new unit tests fail without the fix; both new HTTP cases also reproduced the failure before the fix.
• PATH="$PWD/.venv/bin:$PATH" PYTHONPATH="$PWD/src" python -m pytest --cov=litserve tests/ -q --color=no --durations=15: 384 passed, 10 skipped on macOS arm64 with Python 3.12.11. The explicit source path fixes local subprocess imports from the editable installation.
• pre-commit run --all-files: passed with pre-commit 4.3.0.
• uv build: source distribution and wheel built successfully.

Before submitting

• Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure to update the docs?
• Did you write any new necessary tests?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in GitHub issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 96.87500% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 86%. Comparing base (68c4943) to head (95fd323).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@         Coverage Diff         @@
##           main   #789   +/-   ##
===================================
  Coverage    86%    86%           
===================================
  Files        40     40           
  Lines      3343   3364   +21     
===================================
+ Hits       2870   2897   +27     
+ Misses      473    467    -6     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.