Repository navigation
Dynamic VRAM streaming crashes all generations with HostBuffer.read_file_slice failed → CUDA OOM (regression after Aug 3 2026 update) (CORE-398) #15255
Description
Mod Note:
This is a Cuda error and has been reported to NVIDIA
To workaround, please start comfy with --cuda-device 0 (or a higher number) to restrict comfy to a single GPU or use --disable-pinned-memory for multi GPU functionality
Please start from #15255 (comment) for the latest.
original report
Custom Node Testing
- I have tried disabling custom nodes and the issue persists (see how to disable custom nodes if you need help)
Expected Behavior
Dynamic VRAM streaming should work as it did prior to today's update, or fail gracefully without crashing the whole queue.
Actual Behavior
Environment:
ComfyUI version: 0.30.1
comfy-aimdo version: 0.4.11
comfy-kitchen version: 0.2.26
PyTorch: 2.10.0+cu130
OS: Windows (ComfyUI Desktop)
GPUs: RTX 5060 Ti (16GB, primary) + RTX 3060 (12GB, secondary)
Driver: NVIDIA 610.88, CUDA UMD 13.3
Summary:
After updating ComfyUI Desktop today (Aug 3, 2026), every generation — regardless of model (tested with Krea 2) — fails immediately during sampling with a host buffer read failure that cascades into a CUDA OOM error. This worked correctly before today's update.
Steps to reproduce:
Launch ComfyUI Desktop (dynamic VRAM enabled by default — log shows DynamicVRAM support detected and enabled)
Run any generation (tested: Krea 2 text-to-image)
Sampler starts, model loads and stages into VRAM normally
Crashes ~immediately with:
[ERROR] !!! Exception during processing !!! CUDA error: out of memory
...
File "...\comfy\ops.py", line 228, in cast_modules_with_vbar
handle_pin(s, pin, xfer_source, xfer_dest, subset=subset, size=dest_size)
...
File "...\comfy_aimdo\host_buffer.py", line 109, in read_file_slice
raise RuntimeError("HostBuffer.read_file_slice failed")
RuntimeError: HostBuffer.read_file_slice failed
During handling of the above exception, another exception occurred:
...
File "...\comfy\model_management.py", line 2005, in synchronize
torch.cuda.synchronize()
torch.AcceleratorError: CUDA error: out of memory
Notes:
nvidia-smi confirms this is not a real memory shortage — GPU 0 shows only ~2GB used out of 16.3GB before launch, 14+ GB free.
The crash is not model-size dependent — small and large models both fail identically.
Workaround confirmed: launching with --disable-dynamic-vram fixes it completely; all generations succeed normally with this flag.
This strongly points to a regression in comfy-aimdo's host-buffer/dynamic-VRAM-streaming layer introduced in today's update, since disabling that specific subsystem resolves the issue entirely.
Steps to Reproduce
Image generation in official Comfyui Krea2 turbo / z-image turbo workflows
Debug Logs
File "D:\ComfyUI\standalone-env\Lib\threading.py", line 995, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI\ComfyUI\main.py", line 372, in prompt_worker
e.execute(item[2], prompt_id, extra_data, item[4])
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI\ComfyUI\execution.py", line 728, in execute
asyncio.run(self.execute_async(prompt, prompt_id, extra_data, execute_outputs))
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI\standalone-env\Lib\asyncio\runners.py", line 195, in run
return runner.run(main)
~~~~~~~~~~^^^^^^
File "D:\ComfyUI\standalone-env\Lib\asyncio\runners.py", line 118, in run
return self._loop.run_until_complete(task)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "D:\ComfyUI\standalone-env\Lib\asyncio\base_events.py", line 725, in run_until_complete
return future.result()
~~~~~~~~~~~~~^^
File "D:\ComfyUI\ComfyUI\execution.py", line 789, in execute_async
result, error, ex = await execute(self.server, dynamic_prompt, self.caches, node_id, extra_data, executed, prompt_id, execution_list, pending_subgraph_results, pending_async_nodes, ui_node_outputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI\ComfyUI\execution.py", line 645, in execute
comfy.model_management.unload_all_models()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "D:\ComfyUI\ComfyUI\comfy\model_management.py", line 2027, in unload_all_models
free_memory(1e30, device)
~~~~~~~~~~~^^^^^^^^^^^^^^
File "D:\ComfyUI\ComfyUI\comfy\model_management.py", line 875, in free_memory
memory_to_free = 0 if device is None else memory_required - get_free_memory(device)
~~~~~~~~~~~~~~~^^^^^^^^
File "D:\ComfyUI\ComfyUI\comfy\model_management.py", line 1747, in get_free_memory
mem_free_cuda, _ = torch.cuda.mem_get_info(dev)
~~~~~~~~~~~~~~~~~~~~~~~^^^^^
File "D:\ComfyUI\ComfyUI\.venv\Lib\site-packages\torch\cuda\memory.py", line 897, in mem_get_info
return torch.cuda.cudart().cudaMemGetInfo(device)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
torch.AcceleratorError: CUDA error: out of memory
Search for `cudaErrorMemoryAllocation' in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html for more information.
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
[INFO] FETCH ComfyRegistry Data [DONE]
[INFO] [ComfyUI-Manager] default cache updated: https://api.comfy.org/nodes
FETCH DATA from: D:\ComfyUI\ComfyUI\user\__manager\cache\1514988643_custom-node-list.json [DONE]
[INFO] [ComfyUI-Manager] All startup tasks have been completed.Other
No response
rattus128I have a working theory that this is a bug in Cuda. Im only able to reproduce it in the multi-gpu form, so I have made this test script.
windows_cuda_host_memory_diagnostic.py
Run it from the python venv you run comfy from and paste your results here. @resc863 @WindSingElegy your single GPU data is very interesting.
the script exercises cuda directly in a pattern similar to comfy and attempts to reproduce the irrecoverable error 2. Here is my personal output for reference:
PS C:\Users\rattus> C:\Users\rattus\ComfyUI_windows_portable_nvidia\ComfyUI_windows_portable\python_embeded\python.exe -s .\windows_cuda_host_memory_diagnostic.py
=== One-time comfy-aimdo GPU probe ===
INFO: aimdo: src-…
Activity
coderabbitai commentedon Aug 3, 2026
🔗 Related PRs
#14215 - Avoid ROCm Conv3d crash in Qwen35 vision patch embedding by using equivalent linear projection [open]
#15215 - comfy-aimdo 0.4.11 [merged]
#15299 - Fix OOM_EXCEPTION fallback and guard mem_get_info in get_free_memory (
#15255) [open]#15348 - fix: Graceful fallback for dynamic VRAM init failures (closes
#15255) [open]#15349 - Add MiniMax H3 RTX 3060 Turbo profile [closed]
🧪 Issue enrichment is currently in open beta.
You can configure auto-planning by selecting labels in the issue_enrichment configuration.
To disable automatic issue enrichment, add the following to your
.coderabbit.yaml:💬 Have feedback or questions? Drop into our discord!
SirWidebottom commentedon Aug 3, 2026
i get this too, on any workflow with v30.1,disabling dynamic vram or pinned memory gets it to work now.
Davikar commentedon Aug 3, 2026
Same. Had do downgrade to 0.29.2.
Pombaroom commentedon Aug 3, 2026
The same error. I did ZImage generations ok until this update. Now RAM consumption has become colossal! It goes over 80% and then gives me that error.
I have 24GB RAM and 16GB VRAM.
As SirWidebottom said
It helped. (RAM consumption is 37%, btw)
lzskrasia-spec commentedon Aug 3, 2026
I am experiencing what appears to be the exact same issue, but specifically with Z-Image-Turbo on ComfyUI v0.30.0. The first generation always succeeds without any issues, but the second generation gets stuck indefinitely at the KSampler stage — it never completes, and the process hangs. Real-time VRAM monitoring shows persistently high VRAM usage after the first generation, which never gets released properly. Downgrading ComfyUI from v0.30.0 to v0.29.0 completely resolves the issue; both generations work flawlessly after the downgrade. My hardware is an RTX 4080 with 16GB VRAM and 64GB system RAM. I have reproduced this multiple times with a completely clean v0.30.0 ComfyUI environment, running only Z-Image-Turbo with no other custom nodes or models loaded, and the issue persists every single time on the second generation. nvidia-smi shows plenty of free VRAM (around 14+ GB available), yet the sampler still gets stuck, which strongly suggests that the DynamicVRAM / comfy-aimdo 0.4.11 memory management regression is the root cause — exactly as described in this issue. Thank you for your work on this.
我遇到了几乎完全相同的问题，不过是在 Z-Image-Turbo 模型上，环境是 ComfyUI v0.30.0。第一张图总是能正常生成，但第二张图会无限卡在 KSampler 阶段，进度无法继续，整个流程卡死。实时显存监控显示，第一张图生成后显存占用一直居高不下，没有被正确释放。将 ComfyUI 从 v0.30.0 降级到 v0.29.0 后问题完全消失，两张图都能正常生成。我的硬件是 RTX 4080 16GB 显存和 64GB 系统内存。我在完全干净的 v0.30.0 ComfyUI 环境中多次复现了这个问题，仅加载 Z-Image-Turbo，没有任何其他自定义节点或模型，每次都在第二张图生成时卡住。nvidia-smi 显示显存仍有大量空闲（约 14GB 以上可用），但采样器仍然卡死，这强烈表明 DynamicVRAM / comfy-aimdo 0.4.11 的内存管理回归问题是根本原因，与本 issue 的描述完全一致。感谢开发者的辛勤工作。
vizart-vj commentedon Aug 3, 2026
After updating to 0.30.1 have the same troubles with LTX and Minimax render. Infinite OOM. Z-Image works well.
rattus128 commentedon Aug 4, 2026
Paste of our new RAM usage strategy - this is unlikely related to your crash:
TLDR: RAM management in comfy 0.30.0 has changed and uses more RAM than before to avoid loads from disks. If this is slower for you, start comfy with --fast-disk to change back to a lower RAM, but more disk intensive loading strategy. This is generally better for high speed NVME users.
In comfy 0.23.0 RAM management was changed to facilitate faster loads of models from disks and to unlock async offload to fast NVME disks. However some users, particularly with slower disks noticed performance issues with the new strategy and certain configurations would load models from disk when they previously wouldn't. In version 0.30 a new algorithm for balancing RAM allocation across models and workflows is implement to increase Comfys claim over the system RAM while also minimize RAM churn and disk reload for common usage patterns. To do this comfy now allocates and commits to as much RAM usage as it can get away with based on measured system RAM availability. It will still dynamically avoid swap/pagefile use as much as possible. This algorithm can be slightly slower for users with very fast disks due to extra RAM traffic or can put more pressure on system ram in general. --fast-disk can be added to the comfy startup args to prefer some extra disk loading over a lot of RAM allocation and traffic.
The performance improvement for users with slow disks is significantly larger than the performance loss for high speed disk usage which is why we are making this the new default mode.
Some users also have a fast C: and slow model disk. In this case the use of the pagefile/swap to buffer models can be an advantage over optimized original disk load. So if this is you, start comfy with --high-ram to fully avoid comfys anti-pagefile logic.
We are investigating options to auto-detect these variables and choose the best fit automatically in a future comfy release.
rattus128 commentedon Aug 4, 2026
Does --fask-disk solve it for people?
SirWidebottom commentedon Aug 4, 2026
@rattus128 for me, --fast disk worked, so did disabling dynamic vram or pinned memory. High vram got an OOM error.
rattus128 commentedon Aug 4, 2026
Ok @SirWidebottom and everyone else, are you able to help me out and put it back in the bad state and install this comfy-aimdo version? https://github.com/Comfy-Org/comfy-aimdo/actions/runs/30872550472. Download the wheel for your platform and pip install it.
It adds extra information that helps us distinguish possible root causes. We are having trouble reproducing this issue so the increased information will bring us closer to the variable that matters and a fix.
I will merge that to comfy-aimdo 0.4.12 anyway, but its immediately testable for those with this problem. Please paste the whole crash log from the first error. If in doubt, paste everything from "got prompt".
Please include your system information. If in doubt, paste entire comfy startup log.
SirWidebottom commentedon Aug 4, 2026
@rattus128
here is the error after i installed the comfy-aimdo wheel
ComfyError.txt
akashzeno commentedon Aug 4, 2026
@rattus128 Ran the
0.4.12.dev3wheel. New ERROR line plus a controlled A/B that isolates thetrigger to the second GPU being visible. Your question on #14250 — "do you have a second GPU
installed by any chance?" — looks right.
Diagnostic line
Device copy,
result=2(cudaErrorMemoryAllocation), first block of the first sampler step.On a smaller workflow the same failure occurs on a 19 MiB transfer:
A 19 MiB host→device copy failing on a 16 GB card while torch holds 1,248 MiB reserved.
The A/B
Tightest pair — 10 minutes apart, same workflow (euler, latent
[(1,24,37,22,38),(1,32,2,207)]), same machine state, nothing else touched:cuda:05080 +cuda:13060)cuda:0only, via--cuda-device <5080 UUID>)result=2, 19 MiBSupporting pair on a larger workflow (
res_multistep, latent[(1,24,37,30,54),…]), with hostcommit measured before launch:
That crash was after a clean reboot with a fixed 32 GB pagefile; pagefile peak across the whole
run was 42 MB. No host-memory pressure of any kind.
Honest note on that first pair: the crash ran aimdo
0.4.12.dev3and the success0.4.11,because my debug launcher and normal launcher differ. That is covered separately — a run on
0.4.11with the 3060 visible also crashed, so within 0.4.11 alone it is visible → crash,hidden → success (twice). The failure reproduces on both aimdo versions.
Torch's allocator never sees pressure in any crash:
CUDA OOMs: 0, cudaMalloc retries: 0, peak1,104–2,169 MiB reserved on a 16 GB card.
Ruled out
my earlier "every crash at 95%+ commit" readings were taken after each crash, when the
dead-but-resident ComfyUI process still held ~31 GB. Enlarging the pagefile from 4 GB to a fixed
32 GB changed nothing.
eulerandres_multistep.(controlled pair, same workflow). Expected: the crash is in
prefetch_queue_popfor block 0,before any attention executes.
v0.30.0-1-g14b05228andv0.30.0-4-g9a9fdb10(latter includese377e263and16e3f303).via reflog; last pull was that morning).
Possible lead on the device index
device=0comes fromdestination2.device.indexincomfy/memory_management.py:74— a torchindex. On this box torch and NVML disagree about which GPU that is:
device=0is the 5080 to torch and the 3060 to NVML. If anything on the aimdo side resolves thatinteger through NVML or a DXGI/WDDM adapter table rather than the CUDA runtime ordering, it lands on
the wrong GPU. I can't tell which from outside.
Possibly relevant asymmetry: my displays are on the 3060; the 5080 is headless.
aimdo enumerates in torch order (5080 first), the opposite of NVML's:
Happy to run any further instrumented build — logging which adapter or context the copy actually
targets would settle it.
One thing I can't explain
This exact workflow (euler, 0.2 MP) ran fine 24 times on Aug 3 with both GPUs visible — always
launched the same way, no device restriction — then began failing that evening and has failed every
time since with the 3060 visible (6/6), while succeeding every time with it hidden (17/17). No code
change in that window. So there may be a latent state or ordering dependency rather than a hard
deterministic fault — worth knowing if you try to reproduce and it works first time.
Note: this thread may hold two different problems
Several reporters here are single-GPU and describe RAM climbing to 80%+ before the error, which
matches the 0.30.0 RAM strategy you described rather than this. Mine is not that — 75.7 GB of free
commit and a 42 MB pagefile peak. Might be worth splitting so the multi-GPU signal isn't lost.
Separately:
e377e263skips WindowsThe swap-aware pinning clamp is on the non-Windows branch only; Windows keeps a flat
ram * 0.40.A small or system-managed pagefile is the exact Windows analogue of a swapless Linux box — commit
limit = RAM + pagefile, and pinned memory is non-pageable and charges commit.
GlobalMemoryStatusEx'sullTotalPageFilegives the commit limit directly, so the same clamp isexpressible there. Not the cause of my crash, but it looks like the same class of bug and may be
what some of the single-GPU reporters above are hitting.
System
Process note: after this crash the
prompt_workerthread dies but the process stays alive holding~31 GB. Users who retry without restarting will see every subsequent prompt fail, which is probably
why this reads as "all generations broken" once it starts.
Full log
Complete run log (DEBUG level, ANSI stripped) from the 64 MiB crash:
https://gist.github.com/akashzeno/b1bdbf2650273e84ef1ecd0aa4d55fcc
The
aimdo:lines are written to stderr by the native layer and don't go through Python logging, sothey aren't in that file — the ones that matter are quoted at the top. Happy to capture stderr to a
file if that's more useful.
Pombaroom commentedon Aug 4, 2026
My log
ComfyUIError.txt
If it helps, I have Windows 10 pro 64bit, Ryzen 5 5600, RTX 5060 Ti 16GB VRAM, 24GB RAM (8GB+16GB)
Btw, your --fast-disk also works (as did --disable-pinned-memory)
55 remaining items
blome5569 commentedon Aug 27, 2026
So there is no solution for this really? I apologize for what most likely is a really dumb question for you guys that "speak this language".
Do I understand it correctly that there is no way to run video/image models splitted on multiple GPU?
(at least consumer GPUs)
Not really related to this, but I got 2x 3090 so that's why I'm curious on this topic.
thanksoo commentedon Aug 28, 2026
DELUXA replied to me that they could try "use-ck-attention", but I don't have time to give it a try yet. You can give it a shot first and see how it works. I'm looking forward to your response.
0xDELUXA commentedon Aug 28, 2026
The flag to use is
--use-ck-attentionon AMD too, but obviously I wasn't saying it solves this Nvidia-only issue.ThomasUK888 commentedon Aug 29, 2026
Single-GPU Linux / Blackwell data point — same signature, different root cause (SageAttention 1.0.6)
Posting this because my symptoms matched this issue exactly, both recommended workarounds appeared to help, and the actual cause turned out to be unrelated to the streaming subsystem. Others here may want to rule it out.
Environment: RTX 5090 (sm_120, single GPU) · driver 595.84 · Ubuntu 26.04, kernel 7.0.0-30, glibc 2.43 · ComfyUI 0.34.0 (
0a33ed6c) · torch 2.13.0+cu130 · Python 3.14.4 · comfy-aimdo 0.4.15 · comfy-kitchen 0.2.31 · MiniMax H3int8_convrot(20 GB) + TE, DynamicVRAM streaming.Symptom: identical to this thread. Long video renders (~3 min+) died reliably; short ones always passed.
RuntimeError: HostBuffer.read_file_slice failed→NVRM: Xid 31 ... MMU Fault ... FAULT_PDE ACCESS_TYPE_VIRT_READ→terminate called after throwing 'c10::AcceleratorError'→Fatal Python error: Aborted, thrown fromfree_implinCUDAMallocAsyncAllocator.cpp:206.Why the workarounds were misleading. Each flag changed where the fault was reported, not whether it happened:
hostbuf.read_file_slice(memory_management.py:70)--disable-pinned-memoryread_file_to_device(memory_management.py:59)--disable-dynamic-vramcomfy_kitchen/tensor/base.py:66 to_deviceWith
--disable-pinned-memoryone long render completed before the next one aborted, which read as a partial fix. It wasn't. Because CUDA reports these errors asynchronously, the traceback lands at whatever synchronises next — so the moving stack was an artifact, not a narrowing signal.Actual cause:
--use-sage-attentionwas set the whole time and was the one variable I never removed. PyPI's latestsageattentionis 1.0.6, which is pure Triton JIT with zero compiled kernels, targeting sm_80/86/89. On sm_120 those kernels get JIT-ed onto an architecture that postdates them, and go out of bounds once the sequence is long enough — hence short renders passing and long ones failing.Fix: built SageAttention 2.2.0 from source with
TORCH_CUDA_ARCH_LIST=12.0(it has explicitHAS_SM120support).cuobjdump --list-elfconfirms sm_120 cubins; a numeric check againstscaled_dot_product_attentionat 32k sequence length passes with cos-sim 0.9992. The long render that previously aborted at ~167 s now completes.Worth checking if you're on Blackwell:
pip show sageattention— if it says 1.0.6 and you pass--use-sage-attention, try dropping the flag before reaching for--disable-pinned-memoryor--disable-dynamic-vram. Note thatpip install sageattentionsilently installs the pre-Blackwell 1.0.6, since 2.x is source-only.To be clear about scope: this was the cause on my machine. Given the reports here span multi-GPU Windows configs and models other than H3, I'd expect more than one bug under this issue — but the async-traceback effect makes attention worth eliminating early.
hedongmin070-ops commentedon Aug 31, 2026
too bad that all i can do is switch to the version 0.29.2 or older..
volnodumcev commentedon Sep 2, 2026
RTX 5070Ti the same problem
0JARVIS0 commentedon Sep 7, 2026
Another reproduction case, Linux + Blackwell single-GPU.
Environment
Error: same cuMemSetAccess → Fault failed: 2 at model_vbar.py:94.
Reproduction: 100% under high concurrency. First gen always succeeds, subsequent queue gens fail. Single manual runs always succeed (same as #15269).
--disable-dynamic-vram fixes the crash but causes OOM in comfy_kitchen int8 linear (torch.cat(scaled_parts)) — model is 19.52 GiB, not enough room for int8 intermediates with 124-frame latents.
Not multi-GPU — single GPU per process. Common factor across all reports seems to be cuMemSetAccess failing after repeated fault/unmap cycles. Can run diagnostic script on Linux if available.
myprivacygithub commentedon Sep 29, 2026
This issue can't be fixed?
jishuku commentedon Sep 30, 2026
The issue started with version 0.37 for me. I tried downgrading to 0.36 and I've now been able to run multiple generations without any problems. I have a single 5070 Ti 16GB, 32GB RAM, Windows 11, ComfyUI Desktop, CUDA 13.0 (cu130), PyTorch 2.12.1, comfy-aimdo 0.5.3 (on the 0.36 instance). When running 0.37, adding --disable-pinned-memory fixed it but made everything slower. I'm using Minimax H3 with the Seedhunter 2.1 workflow.
vilma-agent commentedon Oct 1, 2026
Single-GPU Windows Blackwell data point:
--disable-pinned-memorydoes NOT prevent this abort on ComfyUI 0.37.0 + comfy-aimdo 0.5.5Different first-failure line than the
hostbuf_read_file_slicereports: here the abort originates inside the aimdo malloc graph (malloc_graph_pop→ access violation 0x0), then re-raises asaimdo memory compile error.Environment
--disable-pinned-memory(already active)--listen 127.0.0.1 --port 8188Raw log
Relevant to @jishuku above: on 0.37 with comfy-aimdo 0.5.3 a single 5070 Ti 16GB was fixed by
--disable-pinned-memory. On 0.5.5 here the flag is already set and it still aborts, single GPU, cu130. Is the 0.5.3 → 0.5.5 malloc-graph change expected to reintroduce this path? Happy to run a capture with more logging if that helps.drvngenai-cloud commentedon Oct 7, 2026
When exactly is this going to be fixed? Things have been a mess since the update in late September. I’ve kept installing the latest updates just in case, but the issue persists.
LaelaZorana commentedon Oct 8, 2026
I read the whole thread, 74 comments. The multi-GPU issue is indeed a driver bug that was discovered by rattus128 in this thread, and can be avoided by using the default of GPU 0. The issue here is a bit different.
On 0.37.0 the stacktrace posted by vilma-agent seems to come from another place. There is indeed an error happening in pop, however, cleanup_prefetch_queues is then called on the same graph from the finally block of execution.py. There the abort call results in the aimdo memory compile error that covers up the original failure.
The actual issue with pop is in comfy-aimdo but I have a patch for comfy/model_prefetch.py which will reset active flag in case of a pop failure, and log a warning in case of failed abort. That helps revealing the original error. It resets the cast buffers correctly, as well. I did a quick and dirty run with a CPU mock and stubbed-out comfy-aimdo module, with the modification it crashes on cleanup 0 of 20 times while on master 46493d8 it crashes 20 of 20 times.
I opened it as #16883.
vilma-agent commentedon Oct 9, 2026
Confirming from the report side: that matches exactly what I hit. On 0.37.0 + comfy-aimdo 0.5.5, single GPU (RTX 5060 Ti 16GB, sm_120, driver 610.88, cu130), the first failure was the pop itself; the "aimdo memory compile error" only appeared later during cleanup and masked it. Good to have it separated.
If #16883 lands in a build I can pull onto the same box, I'll re-run the Wan 2.2 fp8 VAE-decode repro and post whatever original error it uncovers. That's the part I actually want to see.
maxiedaniels commentedon Oct 16, 2026
Is this fixed with the latest update for anyone
vilma-agent commentedon Oct 16, 2026
Quick factual answer, since I just checked the repo:
--disable-pinned-memoryhelps some configs but does not stop the 0.37.0 + comfy-aimdo 0.5.5 pop abort (my 2026-10-01 datapoint).So: not fixed for me yet, and no merged fix to point at. I have not retested on the newest build; if I can pull it onto the same Blackwell box I reported from, I will post the result here.