Support HTTP.jl 2.x alongside 1.x#775
Open
krynju wants to merge 6 commits into
Open
Conversation
HTTP.jl 2.0 is a breaking release. Add a compatibility layer so AWS.jl works on both 1.x and 2.x, selected at load time via a feature flag. Client-side changes (src/utilities/http_compat.jl + call sites): - `StatusError` constructor changed from (status, method, target, response) to (status, response). - `RequestError`, `iserror`, `isbytes`, `resource`, and the `HTTP.Exceptions` submodule were removed; use `HTTP.isrecoverable` and local shims. - Response bodies are buffered as bytes (no `IOBuffer` bodies), header names are canonicalized, and connection timeouts surface as `TimeoutError` rather than `ConnectError`. Drop the GitHub.jl dependency: it pins HTTP < 2 and was used only by the API-generation tooling. Its three REST calls (authenticate/tree/blob) are reimplemented over HTTP + JSON. Tests updated to construct version-appropriate exceptions/bodies. The IMDS mock no longer replicates HTTP.jl's private retry internals (which differ across versions) and instead honors only the retry_check path IMDS relies on. Verified: unit suite passes under both HTTP 1.11 and HTTP 2.4. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bring GitHub.jl back as a dependency and revert the API-generation tooling to use it (the hand-rolled GitHub REST shim from the HTTP 2 work is removed). This keeps all the HTTP.jl 1.x/2.x client compatibility from the parent branch while using GitHub.jl for `parse_aws_metadata`. Note: registered GitHub.jl (v5) still pins HTTP < 2, so with this dependency the resolver currently selects HTTP 1.x. Once GitHub.jl publishes a release with HTTP 1+2 support, HTTP 2 resolves without further changes (widen the `GitHub` compat bound if that support arrives in a new major version). Verified: unit suite passes (HTTP 1.11 + GitHub.jl). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The test helpers gated their HTTP 1.x/2.x branches on an inline `isdefined(HTTP, :Exceptions)`, duplicating the detection that `AWS._HTTP_V2` already centralizes. Reference the package flag instead so version detection has a single source of truth. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
krynju
marked this pull request as ready for review
June 24, 2026 10:20
The IMDSv2 token PUT runs with `retry=false`; a hop-limit rejection manifests as a read timeout. On HTTP.jl 2.x that read-deadline timeout can surface as an `HTTP.TimeoutError` (the same way connection timeouts do — `is_connection_exception` already handles that), not only as a bare `ETIMEDOUT` `IOError`. Without this, `is_ttl_expired_exception` returns false, the IMDSv1 fallback is skipped, and the token fetch hard-fails in hop-limited (container) environments. Match `HTTP.TimeoutError` in the 2.x branch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`_HTTP_V2 = !isdefined(HTTP, :Exceptions)` is no longer reliable: HTTP 2.x re-adds `HTTP.Exceptions` as a deprecating compat shim (JuliaWeb/HTTP.jl#1315), so on a 2.x release carrying that shim the flag would flip to `false` and AWS.jl would silently run its 1.x code paths on HTTP 2. Switch to `pkgversion(HTTP) >= 2` (with an `HTTP.EmptyBody` feature-detection fallback for Julia < 1.9 — a genuine 2.x-only type that is not shimmed back). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`HTTP.VERSION` only exists from HTTP.jl 2.0 on; under 1.x it resolves to the `VERSION` re-exported from `Base` (the Julia version). Guard with `Base.binding_module` before comparing, and bound to the 2.x line (`v"2" <= VERSION < v"3"`) so a future HTTP 3 isn't treated as 2.x. Replaces the prior version-detection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
krynju
force-pushed
the
http2-support-github
branch
from
June 26, 2026 07:14
4aa049a to
db23e1d
Compare
Member
|
I'm not super familiar with http v2 vs v1 but the code looks more or less reasonable to me. However I think something that would de-risk this a lot is if you updated CI to run on BOTH major versions of HTTP. This would ensure we are truly compatible with both. We have done that before with Onda.jl and its dependency on Arrow, see and I think something similar could probably done here |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds support for HTTP.jl 2.x while retaining 1.x, selected at load time via a small compatibility layer.
Project.tomlnow declaresHTTP = "1, 2".HTTP.jl 2.0 is a breaking release; the relevant changes for AWS.jl are bridged in
src/utilities/http_compat.jl(feature-flagged on the presence of theHTTP.Exceptionssubmodule, which exists only on 1.x):StatusErrorconstructor changed from(status, method, target, response)→(status, response).RequestError,iserror,isbytes,resource, and theHTTP.Exceptionssubmodule were removed —HTTP.isrecoverableand local shims replace them.Vector{UInt8}(1.x exposed a stream object when aresponse_streamwas supplied), header names are canonicalized, and connection-attempt timeouts surface asTimeoutErrorrather thanConnectError.Tests construct version-appropriate exceptions/bodies. The IMDS mock no longer reimplements HTTP.jl's private retry internals (which differ across versions) and instead honors only the
retry_checkpath IMDS actually relies on.GitHub.jl / HTTP 2 interaction
GitHub.jl(used only by theparse_aws_metadataAPI-generation tooling) is retained as a dependency. Registered GitHub.jl currently pinsHTTP < 2, so with GitHub.jl present the resolver selects HTTP 1.x today. Once GitHub.jl publishes a release with HTTP 1+2 support, HTTP 2 resolves with no further change (widen theGitHubcompat bound if that support arrives in a new major version).Testing
Unit suite passes under both HTTP 1.11 and HTTP 2.4 (the latter exercised by temporarily dropping GitHub.jl to allow HTTP 2 resolution).
🤖 Generated with Claude Code