[Draft] Introduce internode communication support via NCCL GIN - #67

Draft

Rachmanino wants to merge 30 commits into

mainfrom

wt/nccl-gin-internode

Commits on Aug 3, 2026

[Feat] Detect GIN-capable NCCL and plumb TL_ENABLE_NCCL_GIN
Rachmanino
committed
6d30a29
[Refactor] Give the distributed metadata table a named layout
Rachmanino
committed
20ec569
[Feat] Add GIN put/signal/wait device templates and the T.nccl_gin DSL
Rachmanino
committed
e182d91
[Feat] Register the allocator arena as one NCCL window and create a GIN devcomm
Rachmanino
committed
504bf27
[Example] Add inter-node allgather, allreduce and reduce-scatter over GIN
Rachmanino
committed
58e557e
[Test] Guard the metadata layout and cover the GIN host and device paths
Rachmanino
committed
9a46006
[Example] Add inter-node AllGather-GEMM and GEMM-ReduceScatter over GIN
Rachmanino
committed
dec9038

Commits on Aug 4, 2026

Commits on Aug 5, 2026