Environment
- Hardware: NVIDIA DGX Spark (GB10, sm_121 / aarch64, 121GB unified memory)
- Driver: 580.173.02
- ComfyUI: v0.37.0 (official tarball)
- Python: 3.12 | PyTorch: 2.13.0+cu130
- comfy-kitchen: 0.2.35
- Model:
minimax_h3_fused_refdelta_r1024_turbo8_mystic07_int8_convrot.safetensors (int8_convrot)
Symptom
With the H3 Fun ControlNet applied (MiniMaxH3FunControlNetApply + control video connected), sampling aborts natively ~2 minutes in: the process dies with a silent SIGABRT (no Python traceback reaches journald; only Fatal Python error: Aborted is observable), systemd restarts it, and the request produces no output.
Key observations that narrow the scope:
Workaround that fully resolves it
Disabling the kitchen CUDA backend so dispatch falls back to triton/eager:
# custom_nodes/zz_kitchen_gb10_fix.py
import comfy_kitchen as ck
ck.disable_backend("cuda")With the CUDA backend disabled, FunCN completes normally on both triton and eager paths. Equivalence check against 0.35.1 (same seed/prompt/chain): per-frame grayscale correlation 0.98–0.99 across the whole clip — the workaround does not change output behavior, only the backend. Cost: FunCN runs ~1.8× slower (537s vs 295s for 124 frames @ 1280×704, 4 steps); pure i2v is unaffected (~300s).
Hypothesis
The CUDA backend's fused int8 path (int8_linear with fused rms_norm/input_act_weight kwargs, introduced alongside the 0.2.34 kwargs interface) hits a native assert or illegal memory access on sm_121 that silently kills the process. Triton and eager implementations of the same operation work correctly on this hardware.
Happy to provide additional traces or test candidate patches (e.g. a guard in the CUDA backend's fused path, or a capability-based fallback for sm_121).
Environment
minimax_h3_fused_refdelta_r1024_turbo8_mystic07_int8_convrot.safetensors(int8_convrot)Symptom
With the H3 Fun ControlNet applied (
MiniMaxH3FunControlNetApply+ control video connected), sampling aborts natively ~2 minutes in: the process dies with a silentSIGABRT(no Python traceback reaches journald; onlyFatal Python error: Abortedis observable), systemd restarts it, and the request produces no output.Key observations that narrow the scope:
--disable-comfy-compilerdoes NOT help on 0.37.0 (it did resolve the older [Bug] MiniMax H3 Fun ControlNet Union aborts at first sampling step; --disable-comfy-compiler completes (Windows, RTX 5080 Laptop) #16230 abort on 0.35.1). So this appears distinct from the malloc-graph/compiler interaction fixed in Fix h3 fun controlnet with comfy compiler. #16240.comfy-kitchen==0.2.33changes the failure mode: instead of the native abort, execution fails cleanly withint8_linear() got an unexpected keyword argument 'input_act_weight'— i.e. 0.37's FunCN path now always passes the four new kwargs (input_act_weight,input_act_eps,residual,residual_scale) that 0.2.33'sint8_lineardoes not accept.Workaround that fully resolves it
Disabling the kitchen CUDA backend so dispatch falls back to triton/eager:
With the CUDA backend disabled, FunCN completes normally on both triton and eager paths. Equivalence check against 0.35.1 (same seed/prompt/chain): per-frame grayscale correlation 0.98–0.99 across the whole clip — the workaround does not change output behavior, only the backend. Cost: FunCN runs ~1.8× slower (537s vs 295s for 124 frames @ 1280×704, 4 steps); pure i2v is unaffected (~300s).
Hypothesis
The CUDA backend's fused int8 path (int8_linear with fused
rms_norm/input_act_weightkwargs, introduced alongside the 0.2.34 kwargs interface) hits a native assert or illegal memory access on sm_121 that silently kills the process. Triton and eager implementations of the same operation work correctly on this hardware.Happy to provide additional traces or test candidate patches (e.g. a guard in the CUDA backend's fused path, or a capability-based fallback for sm_121).