Skip to content

fix: avoid blocking the event loop during file upload retries - #2939

Open
1fanwang wants to merge 5 commits into
googleapis:mainfrom
1fanwang:1fannnw/async-upload-retry
Open

fix: avoid blocking the event loop during file upload retries#2939
1fanwang wants to merge 5 commits into
googleapis:mainfrom
1fanwang:1fannnw/async-upload-retry

Conversation

@1fanwang

@1fanwang 1fanwang commented Sep 3, 2026

Copy link
Copy Markdown

Summary

Async HTTPX upload retries blocked other coroutines. The retry now awaits its delay, and CI runs the existing upload regressions.

Fixes #2938

Testing Done

  • Local code review completed

Save as repro.py; run PYTHONPATH=. python repro.py on baseline and this PR. The peer is local; the SDK delay is unchanged.

Public HTTPX reproducer
import asyncio, io, time
from concurrent.futures import ThreadPoolExecutor
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from google.genai import Client, types
import httpx

class Peer(BaseHTTPRequestHandler):
  attempts: int = 0

  def do_POST(self) -> None:
    self.rfile.read(int(self.headers['Content-Length']))
    start = self.headers['X-Goog-Upload-Command'] == 'start'
    if not start:
      Peer.attempts += 1
    payload = b'{"file":{"name":"files/local"}}'
    self.send_response(200)
    if start:
      self.send_header('X-Goog-Upload-URL',
                       f'http://{self.headers["Host"]}/upload/v1beta/files')
    elif Peer.attempts == 2:
      self.send_header('X-Goog-Upload-Status', 'final')
    self.send_header('Content-Type', 'application/json')
    self.send_header('Content-Length', str(len(payload)))
    self.end_headers()
    self.wfile.write(payload)

async def check_upload(base_url: str) -> None:
  with Client(
          vertexai=False, api_key='local-only',
          http_options=types.HttpOptions(
              base_url=base_url,
              httpx_async_client=httpx.AsyncClient(trust_env=False),
          ),
      ) as client, io.BytesIO(b'local') as payload:
    async with client.aio:
      upload = asyncio.create_task(client.aio.files.upload(
          file=payload, config={'mime_type': 'text/plain'}))
      gap = 0.0
      while not upload.done():
        last = time.monotonic()
        await asyncio.sleep(0.01)
        gap = max(gap, time.monotonic() - last)
      print((await upload).name, round(gap, 3))
      assert gap < 0.2

with ThreadingHTTPServer(('127.0.0.1', 0), Peer) as server, ThreadPoolExecutor(max_workers=1) as pool:
  pool.submit(server.serve_forever)
  try:
    asyncio.run(check_upload(f'http://127.0.0.1:{server.server_port}'))
  finally:
    server.shutdown()
Before: files/local 1.011
AssertionError
After: files/local 0.013

Signed-off-by: 1fanwang <1fannnw@gmail.com>
@Venkaiahbabuneelam Venkaiahbabuneelam self-assigned this Sep 3, 2026
@Venkaiahbabuneelam Venkaiahbabuneelam added the size:M Code changes between 10-40 lines label Sep 3, 2026
Signed-off-by: 1fanwang <1fannnw@gmail.com>
Signed-off-by: 1fanwang <1fannnw@gmail.com>
@1fanwang 1fanwang changed the title fix: keep async file upload retries cancellable fix: avoid blocking the event loop during file upload retries Sep 12, 2026
Signed-off-by: 1fanwang <1fannnw@gmail.com>
Signed-off-by: 1fanwang <1fannnw@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M Code changes between 10-40 lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Async file upload retry blocks the event loop

2 participants