test(async): fix sync-streams — $D.run must store 0x89abcdef before its stream.write#679
Closed
samyfodil wants to merge 1 commit into
Closed
test(async): fix sync-streams — $D.run must store 0x89abcdef before its stream.write#679samyfodil wants to merge 1 commit into
samyfodil wants to merge 1 commit into
Conversation
…tream.write The sync-streams test is symmetric: in the first half $C.get stores 0x01234567 into its buffer and stream.writes it, and $D reads+checks 0x01234567; in the second half $D is meant to store 0x89abcdef into its buffer and stream.write it so $C.set reads+checks 0x89abcdef. But $D.run's second stream.write had no preceding i32.store, so it wrote zero-initialized memory: $C.set read 0x00000000 and its (i32.ne (i32.const 0x89abcdef) ...) assertion trapped on unreachable. No implementation (including the spec reference in definitions.py) can pass the test as written. Add the missing store, mirroring $C.get's own store.
samyfodil
added a commit
to samyfodil/wazy
that referenced
this pull request
Jul 18, 2026
…al conformance (#3) * feat(component): Phase 0 async decode + type plumbing Decode the Component Model async canonical ABI surface and plumb stream/future/error-context as value types, with NO async execution yet. Every async component now decodes and either binds or fails loud. Opcodes verified against wasm-tools 1.253 (real fixtures under binary/testdata/async/), not the scoping doc's guesses — which caught real drift (this vintage uses backpressure.inc/dec 0x24/0x25, not .set). - binary: CanonKind constants + async payload fields on Canon (Async, Cancellable, MemIdx, CoreValType, Slot, TaskReturnResult); decodeCanonSection handles kinds 0x05-0x25; task.return uses the functype resultlist grammar (verified); stream/future defvaltypes 0x66/0x65; error-context is a PRIMITIVE valtype 0x64 (verified, rides the PrimitiveDesc path). Fixed core-func-space accounting: every canon except lift produces a core func. - abi: stream/future flatten/load/store as opaque i32 handles (like own/borrow); error-context rides the primitive paths. - wit: parse error-context. - instance: async canon lift fails loud at bind ("not yet supported") — Phase 1 replaces this with real callback-lift routing. Fixtures are real wasm-tools 1.253 output; tests run the compiled binary from the repo root (embedded) per the scratch/BSD CI model. * refactor(component): unify handle table on a tableEntry interface Generalize handleTable from map[uint32]*resourceEntry to map[uint32]tableEntry, mirroring the Canonical ABI's single per-instance index space (definitions.py's Table, which holds resources + waitables + waitable-sets + error-contexts + stream/future ends together). resourceEntry is the sole implementer today; async entry kinds land in later Phase 1 work without reshaping the table again. Zero behavior change: add() delegates to a generic addEntry() (free-list logic moved, not altered); lookup()/DropOwned() type-assert *resourceEntry with a new (currently unreachable) wrong-kind error; all existing resource error text and locking preserved. Gated on the resources + multiple-resources .wast suites and all resource unit tests passing unchanged. * docs,test(component): async runtime design + first-light acceptance fixture - docs/component-model-async-runtime-design.md: the concrete single-threaded scheduler / callback-lift design (sched FIFO run queue, invokeAsyncCallback loop, async-import completion, keep/trap/omit of the reference's green-thread machinery). The Phase 1 runtime spec. - first_light.{wasm,wat}: real wasm-tools output -- an async func(result u32) export whose core func calls task.return(42) then returns EXIT. The Phase 1b MVP acceptance target (resolves without ever waiting). * feat(component): Phase 1 async runtime — callback lift + waitable/table builtins The callback-based canon-lift MVP core plus the waitable/subtask machinery and table builtins. Transliterated from definitions.py per docs/component-model-async-runtime-design.md. Runtime: - sched.go: single-threaded FIFO run queue (enqueue/step/drive/pumpSnapshot); drive's empty-queue-with-unmet-predicate is an exact deadlock trap, not conservative (no goroutines, no external completion in this slice). - task.go: task state machine (initial/started/resolved; cancel states reserved), ctxStorage[2] (context.get/set — load-bearing), enter/exitTask. - async_lift.go: invokeAsyncCallback — canon_lift's callback loop (unpack code|si<<4, EXIT/YIELD, exclusive release/re-acquire sites, EXIT-without-task.return trap). WAIT still fails loud (driver lands next). - waitable.go: waitable/waitableSet/subtask types, eventCode, event tuples. - Unified table gains entryWaitableSet/entrySubtask kinds + getEntry/removeEntry. Builtins wired as hostFuncDefs in graph.go (no reflection/WithFunc): task.return, context.get/set, backpressure.inc/dec, waitable-set.new/drop, waitable.join, subtask.drop. Decode: FuncDesc.Async (async-func type tag 0x43) so real `async func` component types decode; first_light fixture is now a wasm-tools-valid async component that binds and runs (Call returns [u32(42)]). Remaining for the full MVP: waitable-set.wait/poll + the WAIT scheduler drive in invokeAsyncCallback, and async host imports (WithAsyncImport/AsyncCall + the async canon-lower arm). Instance package 90.1% covered; lint (amd64+arm64), cross-builds, and all component tests green. * feat(component): complete Phase 1 async MVP — async host imports + WAIT driver An async export that AWAITS an async host import now runs end to end on wazy. - async_host_import.go: AsyncHostFunc / AsyncCall{Resolve,Defer} / WithAsyncImport / buildAsyncHostWrapper. The async canon-lower arm builds a subtask, lifts args (MAX_FLAT_ASYNC_PARAMS=4; results always via a trailing retptr), calls the host fn. Immediate Resolve (in-call) -> packed RETURNED, no table entry. Deferred Resolve (via a sched thunk) -> parked subtask returns STARTED|subtaski<<4 and installs the reference's subtask_event pending closure (deliverResolve + (SUBTASK, subtaski, state) at delivery). Resolve is one-shot and panics off the scheduler (external completion is a future CallAsync). - async_lift.go: invokeAsyncCallback's WAIT arm is now real — look up the waitable set (kind-trap), numWaiting-bracket, sched.drive(hasPendingEvent) (empty queue + unmet predicate -> exact deadlock trap), deliver the event tuple to the callback. waitable-set.wait/poll builtins share the identical bracket. - graph.go: wire waitable-set.wait/poll; the lower arm routes async (CanonOpt 0x06) lowers to buildAsyncHostWrapper, with a bind-time trap when the registered import kind (WithImport vs WithAsyncImport) disagrees with the lower's async-ness. - resource.go: entryForLend for the async import's caller-managed lend lifecycle. Acceptance: await_import.wasm (deferred, Call -> [u32(99)]) and await_import_immediate.wasm (synchronous resolve -> [u32(7)]) — both wasm-tools -valid async components. Whole-repo go test, lint (amd64+arm64), cross-builds, -race all green; instance pkg 90.1% covered. Deferred to Phase 2/3: cancellation (states reserved), CallAsync/external completion, stream/future events (numbered, unused), async param spill >4 words. * docs(component): Phase 2 streams/futures/error-context runtime design The rendezvous copy state machine (sharedStream/Future heap objects + stream/ future ends as waitable table entries), copyState {idle,copying,cancelling,done}, the guest/host copyBuffer split, value-handle transfer through the four existing handle seams (abi untouched — stream/future are already flat i32s), and the host StreamWriter/Reader API. Load-bearing subtlety pinned: the end's COPYING state flips inside the pending-event thunk at delivery, not at copy completion (three trap edges depend on it). Fully additive to committed Phase 1. * feat(component): Phase 2 async — streams, futures, error-context runtime The rendezvous copy runtime for stream<T>/future<T>/error-context, additive to Phase 1 (async_lift/sched/task/abi/binary untouched). Transliterated from definitions.py per docs/component-model-async-phase2-design.md. - stream.go/errorcontext.go: sharedStream/sharedFuture heap objects (single pending buffer — rendezvous, no elastic host buffer), streamEnd/futureEnd table entries (embed Phase 1's waitable; entryStreamEnd/entryFutureEnd/ entryErrorContext), copyState {idle,copying,cancelling,done}, the copyBuffer split (guestBuffer holds api.Module = memory-growth-safe; hostBuffer holds Go values), element copy via the compiled abi plans with the none_or_number_type memmove fast path, packed=result|progress<<4, BLOCKED sentinel. - LOAD-BEARING: the end's COPYING->IDLE/DONE flip happens inside the pending-event thunk at delivery, not at copy completion — three trap edges (drop-with-undelivered-event, cancel-racing-completion) depend on it; proven by dedicated trap tests. - stream_builtins.go: stream.new/read/write/cancel-*/drop-* (0x0e-0x14), future twins (0x15-0x1b), error-context.new/debug-message/drop (0x1c-0x1e) as hostFuncDefs; value-handle transfer threaded through the handle seams (readable end moves, writable stays; error-context lift is a get not remove); inHostCall bracket for host-end legality. - stream_host.go: StreamReader/Writer + Future twins (completion-callback shape) — host code as one end of a guest stream/future, driving sched. Acceptance: stream_write_host_read (guest writes stream<u8>, host reads) runs; 42 stream/future/error-context tests incl. a named test per trap edge. Whole component suite + lint (amd64/arm64) + cross-builds green; instance pkg ~90%. Follow-up (fidelity, not correctness): the acceptance fixture is wazy-valid but omits the `memory` canon option on stream.write and needs wasm-tools' cm-more-async feature to strict-validate; harden the fixture + confirm wazy requires/honors the memory option on stream copy (parallels the Phase 1 async-functype fidelity fix). * docs(component): Phase 3 design — cancellation, guest-guest, borrow scopes The parkable guestTask state machine (refactors Phase 1's drive loop; sched gains a parked list + becomes a shared *sched per composition tree), task/subtask cancellation (task.onResolve gains cancelled bool; BLOCKED; the sync-blocking arm retires the has_sync_waiter omission), async-lowered calls to guest exports (the FuncInst protocol bridging two tables; entry backpressure parks instead of trapping), and borrow scopes (retire resource.go's borrow-drop ceiling) + resource.drop async. NOT purely additive — refactors committed Phase 1 + touches shipped resource semantics (highest regression risk; gated on the resources / multiple-resources .wast suites per design 4.2). * feat(component): Phase 3 async — cancellation, guest↔guest, borrow scopes The final async phase. Refactors Phase 1's drive loop into a parkable state machine and retires the last shipped-code ceilings, per docs/component-model-async-phase3-design.md. - guest_task.go: the parkable guestTask state machine (parkEntry/parkWait/ parkYield, ready()/resumeReady()); sched gains a parked list and becomes a shared *sched per composition tree so one WAIT drive can resume another instance's parked task. invokeAsyncCallback becomes start + drive(done) with all Phase-1 error strings preserved. - Cancellation: task.cancel (0x05) / subtask.cancel (0x06) with the full state machine (CANCELLED_BEFORE_STARTED/RETURNED, PENDING_CANCEL, on_cancel, TASK_CANCELLED delivery, BLOCKED); task.onResolve gains a cancelled bool; the decoded Canon.Cancellable is threaded into waitable-set.wait/poll; AsyncCall gains OnCancel/ResolveCancelled. cancel_ack fixture + a named test per edge. - Guest↔guest async lower: the FuncInst protocol bridging two instances' tables (lazy arg lift in on_start), entry backpressure PARKS instead of trapping (the old reentrant-async-call trap is gone); mayEnter/exclusiveHeld/activeTask are per-run brackets. - Borrow scopes: resourceEntry.borrowScope + NewBorrowScoped retire resource.go's borrow-drop ceiling; task-exit numBorrows>0 trap; the same-instance-rep case is correctly exempted (host-minted unscoped borrows still fail loud). resource.drop async (0x07) decodes + routes to sync (the pinned reference has no async branch). Shipped resource semantics preserved: the resources / multiple-resources .wast suites pass UNCHANGED (the design's #1 gate). Whole-repo go test, lint (amd64/arm64), plan9/386/windows cross-builds green; instance pkg 89.3% (near the Phase-2 ceiling; the shortfall is pre-existing untested WASI surface). Deferred (as scoped): stackful lift + thread.* (needs engine continuations), public CallAsync / true external completion, WASI 0.3 host (separate scoping). * docs(component): async trace-oracle design The differential trace-oracle: async_scenarios.json (shared battery) → gen_async_oracle.py (interprets ops against definitions.py, emits golden) → async_oracle_test.go (same ops against wazy's builtins, deep-diffs). Scenario ops are 1:1 canon-builtin calls (handles bound by name, indices are output); golden captures packed returns, event tuples, table snapshots, trap sites. Determinism pinned via DETERMINISTIC_PROFILE + shuffle/choice patches. Covers the waitable/subtask/task/stream/future state machines + canon_lower + root cancellation; the callback-lift loop and guest↔guest are the .wast suites' job. * test(component): official async .wast conformance suites + 3 bug fixes Vendor the 31 official WebAssembly/component-model async .wast conformance suites (test/async/, upstream c940736c, split via wasm-tools json-from-wast) and run them through wazy via a new TestAsyncWastConformance. The harness extends the sync one for the linking model (module_definition/module_instance), assert_invalid, assert_uninstantiable, and trap-text matching. Tally: 4 pass / 27 skip / 0 fail. Passing: closed-stream, cross-task-future, validate-no-stream-char, wait-during-callback. Every skip carries an empirically-verified per-suite reason (deferred features — CallAsync, sync-task-block tracking, instance-poisoning-after-trap — plus deeper protocol-trap state machines). Pre-classified so a known-good regression is the only thing that reds CI. Running the official suites surfaced 3 real bugs the hand-written tests missed: - stream.go: an elementless (elemSz==0) copy bounds-checked its pointer even though zero bytes cross the boundary, so the spec's deliberate garbage pointer (0xdeadbeef) spuriously trapped. Skip the check when nb==0. - composition.go/graph.go: a nested-instantiate func arg that aliases a sibling nested-instance export (not a host import) misread the index space; branch on InstanceIdx vs numImported and reuse delegatingHostImport. Unblocked 6 suites' instantiation. - descriptor.go: stream<char> is now rejected at decode (spec forbids it, component-model#607). 7/7 sync suites still pass; whole-repo go test, -race, lint (amd64/arm64) green. * test(component): async trace-oracle — differential vs definitions.py + 1 bug fix The differential conformance oracle for the async state machines, mirroring the sync oracle. Three artifacts (design: docs/component-model-async-oracle-design.md): - async_scenarios.json: 29 scenarios (waitable/subtask/task/stream/future/ error-context/context/backpressure/root-cancel), ops = 1:1 canon-builtin calls, handles bound by name (indices are output). - gen_async_oracle.py: runs each scenario as the body of a real async lift (Store.invoke) against definitions.py under DETERMINISTIC_PROFILE + the shuffle/choice/daemon-thread patches; emits async_oracle_golden.json (packed returns, event tuples, task-resolve, mem, trap sites, final table snapshot). - async_oracle_test.go (TestAsyncOracle): replays the SAME battery via direct hostFuncDef calls on a hand-built Instance/task and deep-diffs the golden byte-for-byte. Golden is reproducible (sha256-checked) + staleness-hash-guarded. All 29 scenarios match the reference exactly. Golden data embedded via abi/async_oracle_data.go (Go embed can't cross package dirs; single source of truth in abi/testdata). Bug found + fixed (async_builtins.go): subtaskCancelHostFuncGraph called st.onCancel() without bracketing sched.pumping, so a host import that cancels SYNCHRONOUSLY (which AsyncCall.OnCancel's contract allows) panicked "called outside the instance scheduler". Bracketed like sched.step does. Documented exclusion: error-context.debug-message string CONTENT is not byte-comparable (DETERMINISTIC_PROFILE forces "" vs wazy loads the real bytes, an intentional deviation); new/drop index allocation stays in the battery. Whole-repo go test + lint (amd64/arm64) green. * test(component): async call benchmarks (perf baseline) BenchmarkCallAsyncFirstLight (callback-lift loop, no wait): 6 allocs/op, 319 B. BenchmarkCallAsyncAwaitImport (full WAIT path — async import defers Resolve, guest WAITs, scheduler drives, SUBTASK event, task.return): 20 allocs/op, 1001 B. Sync BenchmarkCallAdd baseline is 2 allocs/op. Establishes the target for per-async-call allocation reduction. * perf(component): halve per-async-call allocations Cut the async hot-path allocs, each tied to a profiled (-alloc_objects) site, correctness-gated on the trace-oracle + .wast suites + -race (a pooling/reuse bug would corrupt async state across calls; the byte-for-byte oracle passing is the proof it doesn't). BenchmarkCallAsyncFirstLight 6 -> 3 allocs/op (319 -> 258 B) BenchmarkCallAsyncAwaitImport 20 -> 10 allocs/op (1001 -> 835 B) BenchmarkCallAdd (sync floor) 2 -> 2 (unchanged) - sched.go: runq []func()error -> []schedThunk (unboxed 2-word struct); AsyncCall.Defer -> enqueueVoid stores fn directly (kills the double closure); step() pops via in-place shift-copy, preserving backing cap so steady-state single-item queues stop reallocating. - async_host_import.go: applyResolve only resolves cabi_realloc when the result actually uses memory (resultUsesMem, known at bind time). It was building+ discarding a fmt.Errorf/errors.New on EVERY success call for memory-free results. -3 allocs. - task.go/guest_task.go/async_lift.go: task.onResolve/onStart optional; the plain host-entry invokeAsyncCallback skips two trivial forwarding closures. runLoop's cbStack pooled via getUint64Slice. - waitable.go: waitableSet gains an inline elemsBuf[1]; single-member join no longer heap-grows. - async_builtins.go: task.return's scratch coreVals pooled (coreValueSlicePool). Not pooled (documented, correctness): task/guestTask/subtask/waitableSet/ AsyncCall structs have mixed lifetimes (a subtask lives in the handle table until drop; a guest-guest task's requestCancellation is captured as onCancel) so a shared pool can't safely reclaim them; the deferred-completion closures capture call-time state; the result slice escapes to the caller. Residual FirstLight floor = task+guestTask+result (1 above the sync floor). Conformance (oracle + .wast), -race, whole-repo go test, lint (amd64/arm64) green. * fix(component): cross-component stream/future delegation + cancel-copy race Fix two real async defects the official .wast conformance suites surfaced; honestly reclassify two others whose surface bug hid a genuinely deferred gap. - composition.go repToProviderHandle: cross-component stream/future delegation was handed the host-facing *StreamReader/*FutureReader wrapper where the guest-side runtime expects the raw *sharedStream/*sharedFuture, AND it pre-minted a provider-table handle that collided with the provider's own resolveArgHandles minting pass. Unwrap the reader and pass the shared identity through unminted (mirrors the resource rep-transfer contract). - stream_builtins.go/stream.go: stream.cancel-read/write racing a LIVE partial- progress notification silently replayed the stale COMPLETED event instead of superseding it with CANCELLED. Added streamEnd.livePending, keyed on whether the copy left the buffer with unfilled capacity (the exhausting case must still deliver COMPLETED — TestStreamCancel_RacingCompletionDeliversCompleted). Result: cancel-stream flips skip->pass. async wast tally 4 pass / 27 skip / 0 fail -> 5 pass / 26 skip / 0 fail. cancel-subtask and partial-stream-copies get past the delegation error but then hit a no-callback stackful-lift / sync-task-block gap (the CallAsync-family deferred feature), so their skip reason is reclassified bug:->deferred: with the precise post-fix root cause. async-calls-sync livelock remains the one open bug: (still hard-pre-skipped so it can never hang CI). Trace-oracle, resources/multiple-resources .wast, whole-repo go test, -race, lint (amd64/arm64) all green. * docs(component): narrow the async-calls-sync livelock root cause Investigated the one remaining bug:-labeled suite. Disproved the earlier 'no-callback stackful lift' lead: async-calls-sync.wast's lifts all carry async (callback ...), so it's not the stackful gap. A temporary re-entry-depth guard on startAsyncExportTask did NOT fire before the timeout, ruling out unbounded recursion (bounded Go stack) -- confirming a genuine LIVELOCK (sched.drive spins with progressed=true, never converging), not a deadlock the trap should catch. The distinguishing feature is concurrency: run calls sync-func twice CONCURRENTLY, and the two concurrent subtasks + the blocking-call/unblock handshake form a cycle the single-threaded FIFO scheduler doesn't drive to convergence. Fixing it needs the concurrent-task convergence model (deferred CallAsync/parked-concurrency territory), not a sched.drive tweak. Kept hard-pre-skipped so it can never hang go test ./.... Doc-only. * docs(component): stackful async lift design (handoff-goroutine model) The last big deferred async feature. A stackful task (a sync-opts lift of an async-typed func, or an async-no-callback lift) runs CallWithStack on a dedicated goroutine; a blocking builtin (waitable-set.wait, sync stream/future ops) hands a baton back to the driver via unbuffered channels and parks the goroutine on resumeCh. At most one goroutine runs runtime code (happens-before via the unbuffered handoffs => -race-safe with all existing unlocked state). sched.parked becomes []parkedTask (guestTask|stackfulTask); step/drive/pumpSnapshot unchanged; the oracle needs zero adaptation. Sync-task-block trap: activeTask==nil => 'cannot block a synchronous task'; dont-block-start's start-time case gated on sched.instantiating; deadlock text mapping. reapStackful aborts parked goroutines on Close/error (resumeAbort panics a sentinel that unwinds the engine frame) — no leaks. Unblocks 7 suites (cancel-subtask, partial-stream-copies, empty-wait, deadlock, dont-block-start, sync-streams, zero-length) and plausibly the async-calls-sync livelock (its sync-func IS a stackful sync-opts lift). Staged in 7 green steps. * feat(component): stackful async lift (handoff-goroutine model) The last big deferred async feature, per docs/component-model-async-stackful- design.md. A stackful task (a sync-opts lift of an async-typed func, or an async-no-callback lift) runs its core func on a dedicated goroutine; when the guest calls a blocking builtin (waitable-set.wait, sync stream/future ops) the host func hands a baton back to the driver via unbuffered channels and parks the goroutine. At most one goroutine runs runtime code at a time (happens-before via the unbuffered handoffs), so all existing unlocked async state stays race-free (-race clean). sched.parked generalized to []parkedTask (guestTask | stackfulTask); step/drive/pumpSnapshot/deadlock-detection unchanged; the trace-oracle needs no adaptation. reapStackful aborts every parked stackful goroutine on Close/error (a sentinel panic unwinds the engine frame) -- verified no goroutine leak. Sync-task-block trap ("cannot block a synchronous task"), instantiation-time start-block trap, and the deadlock-text mapping added. async wast conformance 5 -> 10 pass / 21 skip / 0 fail (doubled): stackful lift + the sync-task-block trap unblocked empty-wait, dont-block-start, deadlock, sync-streams, zero-length (and the earlier cancel-stream). Suites still needing more (protocol traps / concurrent-task convergence) are honestly reskipped with precise reasons. trace-oracle, resources/multiple-resources .wast, whole-repo go test, -race, lint (amd64/arm64), cross-builds all green. * feat(component): async protocol-trap suites (17/31 pass) + 2 binder fixes Implement the fail-loud protocol traps the official async .wast suites require, flipping 7 more suites skip->pass (10 -> 17 pass / 14 skip / 0 fail). Mostly the trap logic already existed; the panic/error text just lacked the spec's exact substring: - drop-stream (busy remove/drop), drop-subtask (unresolved), drop-waitable-set (with-waiters), futures-must-write (drop write-end without writing). - trap-if-done + trap-if-transfer-in-waitable-set: a real gap too -- liftResult never validated a top-level export RESULT of stream<T>/future<T>. Added validateLiftedStreamFutureResult (+ non-removing peekReadable* ends), so the not-idle / in-waitable-set lift traps fire. - same-component-stream-future (intra-component read+write guard, retexted). Two real binder fixes surfaced: bindInstanceExportGraph now handles Kind==0x01 inline-export instances (func-sort members bound, type-sort skipped); the old TestGraph_BindInstanceExport_InlineKindUnsupported is split into a negative case + a Kind==0x01 positive case. Left skipped with corrected precise reasons: passing-resources (needs per-element resource-handle transfer in copyElements), drop-cross-task-borrow (resource-tag mismatch in a 3-component sibling alias shape), trap-if-sync-and-waitable-set (0x27 thread.* decoder gap), big-interleaving-test (aliased-func type-index resolution against the wrong type space -- separate binder gap). Oracle golden regenerated for the retexted traps (gen_async_oracle.py; only scenarios_sha256 + 2 go_trap_contains literals changed, verified vs definitions.py). Trace-oracle, resources/multiple-resources .wast, whole-repo go test, -race, lint (amd64/arm64) all green. (Note: commit 8095824's message line claiming sync-streams/zero-length were unblocked is stale -- those remain skipped pending callback-task parking; the code/skip-reasons there are correct.) * fix(component): async resource-tag bug + async/sync validate cross-check (19/31) - drop-cross-task-borrow: fixed a resource-type-tag mismatch in the resource canonicalizer (resourceIdentity/originOf) for a 3-component sibling shape where the same resource type is reachable via two alias paths — the tag is now consistent across alias paths. Suite passes. - validate-no-async-abi-for-sync-type: added the decode/bind-time cross-check that a func type's async-ness agrees with its canon lift/lower async CanonOpt. Suite passes. - big-interleaving-test, builtin-trap-poisons-instance, trap-on-reenter: investigated, skip reasons narrowed with precise root causes (aliased-func type-space resolution; sticky instance-poisoning) — left honestly skipped. async wast 17 -> 19 pass / 12 skip / 0 fail. Trace-oracle, resources .wast, whole-repo go test, -race, lint (amd64/arm64) green. * docs(component): final-3 async feature designs (Fable + Codex xhigh) Two independent designs for callback-task parking (per-segment promotion), cross-instance realloc, and per-element resource transfer in streams — the last features to reach 31/31 async .wast. Fable: per-segment goroutine promotion gated on bind-time mayBlockSync (sched/parkedTask core unchanged). Codex xhigh: the cross-check. Land 2 -> 3 -> 1; zero-length is a stale skip (free unskip). * test(component): delete stale zero-length async skip (passes today) startDelegatedFromStackful (stackful-lift sync-lower parking) already covers zero-length's shape: $Parent is a stackful lift, so its sync-lowers of produce/consume park $Parent's goroutine instead of nested-driving. No runtime change needed. 20/31 async suites now pass. * feat(component): cross-core-instance realloc on canon lift resolveReallocFuncGraph no longer hard-errors when a canon lift's realloc option targets a different core instance than the lift's own core func -- the ABI only requires realloc allocate in the memory the lift already reads via memoryBytesOf(be.mod) (which resolves cross-instance for an imported memory), not that it be co-located with the core func. boundExport gains reallocMod (nil in the common same-instance case); finalizeBoundExport resolves reallocFn against it when set. Mirrors the lower-side's existing cross-instance support (canonMemoryAndRealloc). cross-abi-calls' original bind-time rejection is gone, confirming the fix, but the suite still can't pass end to end: it separately needs async-lowered-import param spilling (>4 flat params), an unrelated, pre-existing, undocumented-until-now milestone ceiling (async_host_import.go's maxFlatAsyncParams). Left skipped with a narrower, precise reason instead of forcing that unrelated feature. * feat(component): per-element resource transfer in stream/future copies Splits the stream/future element bind-time ceiling (resolveStreamOrFutureElem): a top-level own<R> element is now allowed (elemContainsBorrowHandle rejects borrow anywhere -- spec-disallowed, its scoped lifetime can't survive an async transfer -- while elemContainsResourceHandle still rejects own below the top level, a real, documented deferral). numeric stays false for an own element (noneOrNumberType), so the memmove fast path is structurally unreachable for it. guestBuffer gains inst/ownElem: read() reduces a loaded own handle to its rep via TakeOwn on the writer's own table (removing it -- ownership transfers); write() mints a fresh own handle via NewOwn on the reader's own table, right at rendezvous time inside copyElements' general lift/lower path -- no changes needed to sharedStream/sharedFuture or copyElements themselves. Element-type identity now compares by resource IDENTITY, not raw local type index: sharedStream/sharedFuture record the creating instance (elemIn), and elemTypesCompatible (composition.go, built on the existing originOf/canonTag) replaces reflect.DeepEqual at the four compare sites (requireStreamEnd/requireFutureEnd, peekReadable{Stream, Future}End) -- needed because composition gives the same resource a different local TypeSpace index on each instance that names it (a producer's own $R vs. a consumer's imported alias of it). Also fixes a real, pre-existing leak this suite's own second assertion exposed: repToProviderHandle's same-instance borrow exemption (provider owns the resource being lowered into its own method) minted a handle that resolveArgHandles only ever read via Rep(), never dropped -- functionally invisible before, but observable the moment a freed table index (e.g. one Feature 3's own TakeOwn just freed) gets reused for it. Fixed with alreadyProviderRep, matching the reference's real same-instance exemption (lower_borrow: cx.inst is rt.impl -- return the rep directly, mint nothing). resource.go's "unknown handle %d" -> "unknown handle index %d" (every existing test matches the "unknown handle" substring, unaffected). passing-resources now passes: 21/31 async suites (up from 20/31). * feat(component): callback-task parking via per-segment promotion The crux feature: a callback-lifted guestTask can now suspend MID-CORE-CALL (a sync stream/future copy, or a sync-lowered call to an async callee) instead of nested-driving the shared scheduler on the current goroutine -- the nested drive could frame-hold the caller's own continuation beneath the suspension, the exact deadlock/livelock async-calls-sync and sync-streams were skipped for. Per-segment promotion (docs/component-model-async-final3-fable.md §1): - Instance.mayBlockSync is set at graph-bind time when an instance's core module(s) can reach a mid-core-call blocking site from a callback task (a sync stream/future copy canon, or a sync-lowered import targeting an async lift). Set at both bind sites: stream_builtins.go's streamFutureCanonHostFunc (sync read/write/cancel canons) and graph.go's lower-func binder (sync lower -> async-lift target). - guestTask gains promoted (set only in startAsyncExportTask's callback arm, i.e. guest-caller-started tasks -- invokeAsyncCallback's host-entry call stays unpromoted, an explicit, documented scope boundary) and seg (the live segment, non-nil while a per-invocation goroutine exists). runSegment executes firstRunBody/runLoopBody inline for an unpromoted task (bit-identical to every currently-green suite) or on a fresh baton goroutine for a promoted one; a new parkBlocked park state + gt.block (transliterated from stackfulTask.block) suspends the segment goroutine mid-call, registering in sched.parked exactly like a stackful park. - task.blocker() generalizes "who can suspend mid-call" (stackfulTask or a promoted guestTask with a live segment) to one taskBlocker interface; async_builtins.go's blockingTask/activeBlocker, stream_builtins.go's three sync-copy waits, host_import.go's sync-lower routing, and graph.go's startDelegatedFromBlocker (renamed from startDelegatedFromStackful) all switch to it -- mechanical, each keeps its existing nested-drive/trap else-arm verbatim. - sched.drainReady runs every already-ready parked task to quiescence after a successful drive, so e.g. a promoted task left ready-but-parked by the invoke that unblocked it still runs to EXIT before that invoke returns. reapStackful generalizes to reapParkedGoroutines, aborting a parked promoted segment the same way it aborts a stackful goroutine; wired into Close and every invoke*/gt.start error path. - task.requestCancellation's gt arm splits out a parkBlocked case that hands off the baton (mirroring the st arm) instead of resuming inline the way a frame-free WAIT/YIELD is. zero-length was already passing (stale skip, deleted separately). Unblocks: - sync-streams: parking mechanics verified fully correct (every event code/ordering assertion passes, both sides reach EXIT/42) via a scratch-patched copy of the fixture; left skipped for a precise, narrow, evidenced residual -- one hardcoded memory-content assertion (`i32.load(bufp)==0x89abcdef`) that wasm-tools disassembly proves is unsatisfiable by ANY implementation (zero i32.store/data-segment targeting that address in the vendored .0.wasm; cross-checked against internal/component/abi/testdata/definitions.py's own SharedStreamImpl write/drop semantics, which are byte-for-byte identical to wazy's and would copy the same zero bytes). - async-calls-sync: passes outright, run2's anticipated YIELD-fairness residue did not materialize. New tests (guest_task_promoted_test.go, no wasm needed): hand-built promoted-guestTask park/resume/goroutine-delta-0, reapParkedGoroutines aborting a parked segment, and a non-cancellable parkBlocked cancel landing as PENDING_CANCEL -- mirroring stackful_task_test.go's existing reap acceptance tests. 23/31 async suites pass (up from 19 at session start); -race clean (package + full whole-repo); no goroutine leaks (reap tests green); trace oracle, sync/async wast conformance, and the whole repo all green. * feat(binary): decode thread.* canon kinds (fail-loud at bind) Recognize the thread.* canonical builtins (thread.yield 0x0c, thread.index 0x26, thread.new-indirect 0x27, thread.suspend 0x29, thread.yield-then-resume 0x2b) that the cancellable/sync-barges-in/trap-if-block-and-sync/trap-if-sync-and-waitable-set async .wast suites use. They now decode cleanly and fail loud at bind ('does not produce a core func') instead of a decode-time rejection; genuine thread.* runtime (stackful suspend-resume-with-value / thread spawning) stays unimplemented. * feat(component): async .wast 24/31 — cross-abi param-spill, trap-wording, sibling/local-lift binder gaps Flip three official async conformance suites skip->pass (22->24; +fused.* sub-cases in the sync suite via the same binder fix): - builtin-trap-poisons-instance: prepend wasmtime's exact `unreachable` trap wording ahead of wazy's message in invokeEntered's CallWithStack error path (wrapUnreachableTrap); poisoning itself already worked. - cross-abi-calls: implement the missing param-spill for async- and sync-lowered imports past their flat-param limit, mirroring the LIFT-direction spill in reverse (liftAsyncHostArgsSpilled / liftHostArgsSpilled via abi.Load over a TupleDesc). - trap-on-reenter binder gaps (both fixed, verified instantiating): canon-lower of a not-yet-instantiated sibling export resolves its WIT signature statically and defers the live *Instance (pendingDelegates); outerFuncArgImport now also binds a (func N) arg naming the outer component's own local canon lift. Two suites stay honestly skipped (precise reasons in asyncWastSuites): trap-on-reenter's deeper cause is mayEnter/entering_set ancestor-exclusion (not poisoning); big-interleaving needs an implicit Task per sync lift. Both are cross-cutting task-model reworks that risk regressing passing composition suites; deferred over guessing. Green: TestAsyncOracle, sync+async .wast (24/7/0), whole-repo, -race, gofmt, lint amd64+arm64. * feat(component): async .wast 25/31 — trap direct parent<->child re-entry (trap-on-reenter) Flip trap-on-reenter skip->pass. All 3 of its assert_traps are DIRECT static parent<->child lineage calls (the .wast's own "for now, trap on parent-to-child / child-to-parent"); 2 of 3 are pure sync (no async task parks), so this is NOT an entering_set/outstanding-task-mayEnter change — that model can't trap these cases (child->parent entering_set is empty = vacuously allowed) and would dangerously over-trap the parking-based composition suites. Instead pin wasmtime's current rule: refuse any direct-lineage delegate at call time. - hostImport.lineage bool, set at exactly the two binder arms that create a direct parent<->child delegate (both added dead-before-now by the earlier trap-on-reenter binder fixes): outerFuncArgImport's own-local-lift instantiate-arg arm (composition.go, child->parent) and computeCanonHostFunc's canon-lower of a locally-nested child export (graph.go, al.InstanceIdx>=numImported, parent->child). - Refuse at call time in buildHostWrapper / buildAsyncHostWrapper with the verbatim "wasm trap: cannot enter component instance". Sibling-delegation arms untouched, so no currently-passing composition suite (async-calls-sync, cancel-subtask, partial-stream-copies, deadlock, ...) changes. Design: docs/component-model-async-taskmodel-design-{brief,fable,codex}.md (Fable §2; Codex's full entering_set+parent-pointer+caller-threading machine rejected as insufficient/unfaithful/dangerous for this suite). Green: TestAsyncWastConformance 25/6/0, TestAsyncOracle (no golden change), sync wast + resources + whole-repo, -race clean. * feat(component): async .wast 26/31 — implicit sync task + driverless-caller drain (big-interleaving-test) Flip big-interleaving-test skip->pass. Two fixes, both needed: 1. Implicit sync task for plain sync canon lifts. Per canon_lift the reference builds a Task for EVERY call regardless of the async option, so current_task() always resolves; wazy only built one for async/stackful lifts, so 23/45 of this suite's sync $Testee/$Mock exports trapped at requireActiveTask when they called a legal-in-sync builtin (waitable-set.poll, backpressure.inc/dec, context.get/set). - task.syncImplicit installed in invokeEntered (fresh ctxStorage/call = spec-correct per Thread.__init__), save/restore for nested guest->guest sync calls, defer cleanup. - Gated behind a bind-time Instance.syncTaskNeeded (set by the 8 current-task builtin constructors) so sync-only components keep a byte-identical, alloc-free, race-free invokeEntered (instance.go documents concurrent sync Call). leaveRun/suspendRun restore activeTask to Instance.syncBase (nil-identical for every existing flow). - Critical non-regression: blockingTask still traps a syncImplicit task (preserves dont-block-start/empty-wait/deadlock's sync-task-block trap); task.return rejects it. 2. Driverless-caller scheduler drain. $Testee.await is a stackful async export reached via a guest<->guest async-lower from $Driver.run (a plain SYNC lift). stackfulTask.block parks unconditionally and the guest<->guest delegate path (startAsyncExportTask) deliberately never drives -- but a sync-lift caller has NO scheduler driver above it, so an already-ready parked callee sat forever and returned STARTED instead of RETURNED. Drain to quiescence in buildAsyncHostWrapper's guest<->guest arm ONLY when the caller is driverless (activeTask nil or syncImplicit); a callback/stackful caller has its own driver above, so async-calls-sync/cancel-subtask/partial-stream-copies/cross-abi-calls are bit-identical. Design: docs/component-model-async-taskmodel-design-{brief,fable,codex}.md (Fable §1 + its §1.6 contingency, resolved with the targeted driverless-only drain). Green: TestAsyncWastConformance 26/5/0, TestAsyncOracle (no golden change), sync wast + resources (byte-identical), whole-repo, -race clean (29.6s), vet + gofmt. * test(component): document sync-streams as a confirmed UPSTREAM fixture bug Traced the lone sync-streams failure to a defect in the vendored (and upstream-main, byte-identical) test/async/sync-streams.wast: $D.run is missing the `(i32.store (local.get $bufp) (i32.const 0x89abcdef))` that mirrors $C.get's own store at wast:37. The two halves are meant to be symmetric ($C.get stores+writes 0x01234567, $D reads+checks it; $D should store+write 0x89abcdef, $C.set reads+checks it) but $D writes its zero-initialized offset 16, so $C.set reads 0 and the assert traps. No i32.store of 0x89abcdef exists in either core module, upstream is identical, and the reference impl fails it too. Our runtime is correct (every event-code/ordering assertion passes; patching only the constant in a scratch .wasm makes the whole suite pass). Left honestly skipped as an upstream defect -- NOT patched, to keep the conformance claim faithful. 26/31 pass, 0 fail unchanged. * docs(component): thread.* fiber-runtime design (Fable + Codex xhigh) Design for the last 4 async .wast suites (cancellable, sync-barges-in, trap-if-block-and-sync, trap-if-sync-and-waitable-set) that need genuine thread.* execution. Key finding: wazy's stackful goroutine+baton IS the reference's Thread/Continuation/block/resume pattern, so this generalizes 1 goroutine/task to N threads/task in a per-Instance thread table. 3 of 4 suites need almost no new fiber machinery (binds + existing taskBlocker.block + trap-text); only trap-if-sync-and-waitable-set needs real spawned threads. 4-stage landing order (27/28/29/30 of 31). * feat(component): async .wast 27/31 — thread.yield (sync-barges-in) Stage A of the thread.* fiber runtime. thread.yield (canon 0x0c) = the reference's Thread.yield_ = wait_until(lambda: True) (definitions.py:411-412, 2723-2727): a cooperative yield with an always-true ready predicate, expressed entirely through the EXISTING taskBlocker.block / sched.pumpSnapshot machinery -- no guestThread or thread table yet (those are Stages C/D). Three arms per the design: - stackful/promoted caller: blk.block(alwaysReady, cancellable) -- a real park resumed on the driver's next step (sync-barges-in's $C.yielder, a stackful lift, hits this). - unpromoted callback task: deliverPendingCancel prologue + one pumpSnapshot round. - sync/task-less: one pumpSnapshot round, never traps (yield is allowed from a sync task, unlike suspend/wait). Bind case in computeCanonHostFunc + syncTaskNeeded/mayBlockSync set at bind time (fires only for components that bind a thread canon -- inert for all 26 prior suites). Design: docs/component-model-async-threads-design-{fable,codex}.md (Fable Stage A). Green: TestAsyncWastConformance 27/4/0, TestAsyncOracle 27 scenarios (no golden change), sync wast + resources, whole-repo, -race clean. * feat(component): async .wast 28/31 — thread.yield-cancellable (cancellable) Stage B of the thread.* fiber runtime. cancellable uses thread.yield-cancellable (canon 0x0c, cancel?=true), which Stage A's threadYieldHostFunc already handles, and Stage A's bind-time mayBlockSync/syncTaskNeeded set already promotes cancellable's $C callback tasks -- so the suite passes with ZERO change to thread.go/graph.go, exactly as the design predicted (Fable §5.4/§8.2/§12 Stage B: zero new-code surface). The four cancel-interplay assertions (delivered-while-parked, pending-cancel prologue-consumed, poll-cancellable) all run on existing, already-tested machinery. Also backfills the thread.yield oracle coverage deferred from Stage A: a "thread.yield" op in gen_async_oracle.py + async_oracle_test.go and two scenarios (thread-yield-basic, thread-yield-cancel-interplay: a non-cancellable yield does NOT consume a pending cancel, a cancellable one does via the prologue). Golden regenerated by the generator (deterministic; regen produces no diff), not hand-edited. Oracle now 31 scenarios. Green: TestAsyncWastConformance 28/3/0, TestAsyncOracle 31/31, sync wast + resources, whole-repo, -race clean. * feat(component): async .wast 29/31 — thread.index/suspend + new-indirect bind (trap-if-block-and-sync) Stage C of the thread.* fiber runtime. Bind the remaining thread canons this suite needs so Instantiate succeeds, and enforce the sync-context trap edges: - threadTable (index-0-reserved free list) on Instance; task.implicitThreadIdx. - thread.index (0x26): real body, lazy per-task implicit-thread slot allocation. - thread.suspend (0x29): blocks via the task blocker; from a sync/task-less context hits the "cannot block a synchronous task before returning" trap (the suite's assertion). - thread.new-indirect (0x27): BIND path only (coreTableTarget threading, table/typeID resolution, ft validated via the consumer's core import signature since the core-type section is raw-decoded -- Fable §11 deviation); call body is a fail-loud stub (Stage D fills execution -- this suite's new-indirect/index call sites are commented out). - thread.yield from a sync task stays allowed (Stage A); "invalid"->"unsupported callback code" trap-text reword (§6.1, no other suite pinned the old text). Running this suite for the first time surfaced 3 latent (pre-existing) bugs, all the same implicit-sync-task class as the big-interleaving work, fixed narrowly: - requireNotSyncImplicit: subtask.cancel + sync stream/future read/write/cancel now trap "cannot block a synchronous task" for a REAL syncImplicit caller (before validating a deliberately-garbage handle), while leaving a genuinely task-less caller on the existing nested-drive fallback (acceptance tests depend on task-less-OK; -race caught the first over-broad pass). - host_import sync-lower-into-async-lift trap now also fires for a real syncImplicit caller post-instantiation, not only in the instantiating window. - be.home now also homes a plain sync re-export when sub.syncTaskNeeded, so the syncImplicit task installs on the instance whose host-func closures actually read activeTask. Green: TestAsyncWastConformance 29/2/0, TestAsyncOracle, sync wast + resources + acceptance, whole-repo, -race clean (count=1), vet + gofmt. * feat(component): async .wast 30/31 — guestThread spawn/switch runtime (trap-if-sync-and-waitable-set) Stage D (final) of the thread.* fiber runtime: real spawned threads. This is the only suite needing genuine thread.* execution; the other 3 landed on binds + existing machinery. - guestThread: the goroutine + unbuffered-channel-baton primitive (shape-identical to stackfulTask/guestSegment), registered in the per-Instance threadTable. Lazy `go th.main()` on first resume; at most one goroutine in the composition tree runs runtime code (single- runnable invariant preserved across N threads). - thread.new-indirect (0x27) execution: LookupFunction(table,typeID,fi) with real call_indirect trap semantics, spawns a SUSPENDED guestThread, returns its index. - thread.yield-then-resume (0x2b): direct switch (other.resumeReady drives other to completion/park) then self-park; switch-then-park proven equivalent to the reference's park-then-switch (Fable §4.5/§11.3). - activeThread/currentBlocker seam: blockingTask/activeBlocker prefer in.activeThread (a spawned thread) over t.blocker(), so a spawned thread's own sync stream/future copy parks itself instead of corrupting the implicit thread's baton. Nil except while a spawned thread holds the baton (Fable's narrow seam over Codex's implicit-thread-as-guestThread rework). - syncWaiter brackets on sync stream/future copies; waitable.join / subtask.cancel trap "waitable cannot be used synchronously while added to a waitable set" (exact spec text). - reapParkedGoroutines gained a *guestThread case (resumeAbort + await final yield); no goroutine leak (leak tests with post-Close goroutine-count assertions, run under -race). Design: docs/component-model-async-threads-design-{fable,codex}.md (Fable throughout; Codex's more invasive implicit-thread rework declined for blast radius). Green: TestAsyncWastConformance 30/1/0 (only skip = upstream sync-streams bug), TestAsyncOracle, sync wast + resources + acceptance, ~45 new thread unit+leak tests, whole-repo, -race clean (37.6s). * test(component): async .wast 31/31 — apply the sync-streams fix locally (pending #679) sync-streams was the last skip: a confirmed bug in the official upstream fixture ($D.run omits the store that would make $C.set's 0x89abcdef assertion satisfiable). The fix is filed upstream as WebAssembly/component-model#679; apply it locally so wazy isn't blocked on the merge. - Add the missing `(i32.store (local.get $bufp) (i32.const 0x89abcdef))` to $D.run in the vendored sync-streams.wast, with a comment referencing #679 and noting it should be removed when we re-vendor after the merge (at which point the fixture is byte-identical to upstream again with zero code change). - Regenerate sync-streams.0.wasm + sync-streams.json via `wasm-tools json-from-wast`. - Un-skip sync-streams. The runtime was already correct here (verified earlier via a scratch-patched .wasm); no engine change. Conformance claim stays honest: official suite + one documented, upstream- filed fixture fix. Green: TestAsyncWastConformance 31 pass / 0 skip / 0 fail, TestAsyncOracle, sync wast + resources + acceptance, whole-repo, -race clean.
Contributor
Author
|
@lukewagner could you explain? |
Member
|
@samyfodil Great find, and thanks for filing! I believe this commit fixes this issue, including your fix, also ensuring that the test waits for |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
test/async/sync-streams.wasttest is symmetric, but the second half is missing a store so it cannot pass in any conforming implementation.First half —
$C.getstores0x01234567into its buffer thenstream.writes it;$D.runreads it back and checks0x01234567:Second half —
$D.runis meant to store0x89abcdefthenstream.writeit, so$C.setreads it back and checks0x89abcdef((i32.ne (i32.const 0x89abcdef) (i32.load (local.get $bufp)))). But$D.runhad noi32.storebefore its secondstream.write:So
$Dwrites its zero-initialized memory at offset 16,$C.setreads0x00000000, and the0x89abcdefassertion traps onunreachable.This affects every implementation, including the spec reference in
design/mvp/canonical-abi/definitions.py(the stream copy transfers the same zero bytes). The fix adds the missing store in$D.run, mirroring$C.get's existing store.Found while running the official async
.wastsuites against an independent Component Model runtime: this was the only one of the async value/stream suites that no correct implementation could pass.