[bugfix]: drop redundant h3_sequential_load from Spark FastH3 configs - #36
Closed
aryan5v wants to merge 21 commits into
Closed
[bugfix]: drop redundant h3_sequential_load from Spark FastH3 configs#36aryan5v wants to merge 21 commits into
aryan5v wants to merge 21 commits into
Conversation
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
…nse GEMM (hao-ai-lab#1788) Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
…ab#1793) Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Shahrad Zomorrodi <264690209+shahradzomorrodi@users.noreply.github.com>
…ao-ai-lab#1803) Co-authored-by: Kyle <shh075@ucsd.edu> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Satyam Srivastava <srivastavasatyam53@gmail.com> Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
hao-ai-lab#1805) Co-authored-by: Vaish, Ishan <isvaish@UCSD.EDU> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Fold the two-Spark recipe into FastH3 Preview so users pick NVIDIA DGX Spark, then 1 Spark or 2 Sparks, instead of a separate catalog card.
lazy_module_load owns component deferral on GB10; h3_sequential_load is suppressed when lazy is on (hao-ai-lab#1803 review). Remove the dead flag from the three Spark YAML presets and add a regression test. Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The three Spark FastH3 YAML presets shipped with both
lazy_module_load: trueandh3_sequential_load: true. On GB10/unified memory,lazy_module_loadowns component deferral and explicitly suppresses sequential load (see_defer_denoise_modulesinminimax_h3_pipeline.py, fixed in the hao-ai-lab#1803 review). Theh3_sequential_loadflag in those configs was therefore dead and misleading.Solution
h3_sequential_load: truefrom:examples/inference/basic/basic_fasth3_spark.yamlexamples/inference/basic/basic_fasth3_spark_pair.yamlexamples/serving/openai_fasth3_spark.yamllazy_module_loadis the Spark deferral path;h3_sequential_loadremains the H3-only fallback when lazy is off (--no-lazy-module-load).tests/local_tests/test_cookbook_serving.py.Context
h3_sequential_load(load Qwen3-VL → run conditioning → release → load DiT/VAE).lazy_module_load(general per-component load-on-use / release-after-stage).torch.compilecan attach to the lazy proxy.Sequential load is not dead code — it is still used on discrete GPUs without lazy load, or when
--no-lazy-module-loadis passed on Spark.Testing
test_spark_configs_use_lazy_load_not_sequential(requires dev env with torch/pytest to run locally).