Skip to content

[kernel] VSA kernel: one sm_100a / sm_103a image per listed arch; un-gate backward on sm_103a - #1833

Open
lpc0220 wants to merge 2 commits into
hao-ai-lab:mainfrom
lpc0220:pengchengl/vsa-cmake-per-arch
Open

[kernel] VSA kernel: one sm_100a / sm_103a image per listed arch; un-gate backward on sm_103a#1833
lpc0220 wants to merge 2 commits into
hao-ai-lab:mainfrom
lpc0220:pengchengl/vsa-cmake-per-arch

Conversation

@lpc0220

@lpc0220 lpc0220 commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Per-architecture VSA images. ENABLE_VSA_SM100_FAMILY ([feat] Add native SM103a VSA support #1812) compiled both the sm_100a and the sm_103a image whenever either 10.0a or 10.3a appeared in TORCH_CUDA_ARCH_LIST, so a GB200 build (or the auto-detected default) also carried an sm_103a image and the arch list no longer meant what it said. Every listed architecture paid for two images: a larger .so, and double the nvcc time for the VSA sources (the forward alone is several minutes per image). It is replaced by ENABLE_VSA_SM100A / ENABLE_VSA_SM103A with one -gencode per listed architecture, the way the 9.0a / 12.0a entries are already handled. The publish workflow lists 10.3a explicitly for the two Blackwell wheels, which keeps shipping both images.
  • Backward on sm_103a. The VSA backward ([kernel] sm_100a CUDA backward for VSA block-sparse attention (blk64) #1819) now builds the same images as the forward: its kernels' multi-arch guards admit the sm_103a pass (they compiled to empty stubs before), block_sparse_attn_bwd_sm100a.is_supported accepts compute capability (10, 3), and the tests' skip conditions follow.

Ungating GB300 for the forward and backward only gives B300/GB300 users an alternative to the Triton-only path: these are the GB200 kernels compiled for sm_103a, not kernels tuned for it. GB300-specific VSA kernels (using the sm_103a-only features) are left as future work.

No kernel code changes; the sm_100a images are byte-identical to before.

Test plan

  • GB200 (driver 580.178.04): TORCH_CUDA_ARCH_LIST=10.0a wheel; tests/test_block_sparse_sm100a.py 37, tests/test_block_sparse_bwd_sm100a.py 16, tests/test_block_sparse_sm100a_dispatch.py 11 passed; cmake reports building block_sparse_sm100a (sm_100a; ...).
  • TORCH_CUDA_ARCH_LIST=10.3a wheel (before the backward enable): only sm_103a images in the .so, block_sparse_sm100a_fwd present, no backward symbol, cmake reports (sm_103a; ...).
  • GB300: TORCH_CUDA_ARCH_LIST=10.3a wheel; the same three suites 37 / 16 / 11 passed; compute-sanitizer --tool memcheck on the backward clean; perf below.

GB300 perf (sm_103a image = the GB200 kernels recompiled, nothing tuned for GB300)

B=1 H=8 D=128 bf16, 25% density, median of 3 alternating passes, TFLOPS. Forward in the selected-pairs convention, backward 2.5x that. Triton forward = the vendored reference copy; Triton backward = fastvideo_kernel kernels, k2q precomputed.

S fwd ours fwd Triton ratio bwd ours bwd Triton ratio
4096 635 166 3.83x 382 189 2.03x
8192 955 318 3.01x 575 303 1.90x
16384 1199 461 2.60x 705 380 1.85x
32768 1295 524 2.47x 780 421 1.85x
65536 1303 564 2.31x 811 438 1.85x
131072 1230 570 2.16x 796 443 1.80x

root and others added 2 commits September 8, 2026 11:44
…h; backward on sm_103a

ENABLE_VSA_SM100_FAMILY built both data-center Blackwell images whenever either 10.0a or 10.3a
appeared in TORCH_CUDA_ARCH_LIST, so a GB200 build (or the auto-detected default) also carried an
sm_103a image and the arch list no longer meant what it said. It is replaced by ENABLE_VSA_SM100A
and ENABLE_VSA_SM103A, one -gencode per listed architecture, the way the 9.0a / 12.0a entries are
handled. The publish workflow lists 10.3a explicitly for the two Blackwell wheels.

The VSA backward now builds the same images as the forward: its kernels' multi-arch guards admit
the sm_103a pass (they compiled to empty stubs before), the Python is_supported accepts compute
capability (10, 3), and the tests' skip conditions follow. The sm_103a backward is compile-only
so far; B300/GB300 validation is the next step.

Verified on GB200 (driver 580.178.04): TORCH_CUDA_ARCH_LIST=10.0a wheel -> 64 tests passed;
TORCH_CUDA_ARCH_LIST=10.3a wheel (before the backward enable) carried only sm_103a images and no
backward symbol, as intended.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@mergify mergify Bot added scope: kernel CUDA kernels, fastvideo-kernel scope: infra CI, tests, Docker, build labels Sep 9, 2026
@mergify

mergify Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🔴 1 of 1 protections blocking · waiting on 👀 reviews and 🤖 CI

Protection Waiting on
🔴 PR merge requirements 👀 reviews and 🤖 CI

🔴 PR merge requirements

Waiting for

  • #approved-reviews-by>=1
  • check-success=fastcheck-passed
  • check-success=full-suite-passed
This rule is failing.
  • #approved-reviews-by>=1
  • check-success=fastcheck-passed
  • check-success=full-suite-passed
  • check-success~=pre-commit
  • title~=(?i)^\[(feat|feature|bugfix|fix|refactor|perf|ci|doc|docs|misc|chore|kernel|new.?model|skill|skills|infra)\]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

scope: infra CI, tests, Docker, build scope: kernel CUDA kernels, fastvideo-kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant