Skip to content

feat(megatron): Add support for Gated Delta Net (GDN) & Kimi Delta Attention (KDA) - #676

Open
clairesonglee wants to merge 44 commits into
mainfrom
clairlee/kda-optimized-training-patch
Open

feat(megatron): Add support for Gated Delta Net (GDN) & Kimi Delta Attention (KDA)#676
clairesonglee wants to merge 44 commits into
mainfrom
clairlee/kda-optimized-training-patch

Make hybrid model tooling user-agnostic

83ad26f
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar