Skip to content

Add Automicrobatching for Non-Powers-of-2 + Fixes to FSDP deadlocks using Adaptive Sync Hooks - #3503

Draft
JackZ-db wants to merge 23 commits into
mosaicml:mainfrom
JackZ-db:jz/auto_non_powers_of_2
Draft

Add Automicrobatching for Non-Powers-of-2 + Fixes to FSDP deadlocks using Adaptive Sync Hooks#3503
JackZ-db wants to merge 23 commits into
mosaicml:mainfrom
JackZ-db:jz/auto_non_powers_of_2

only drop hooks after 3 consecutive successes with this microbatch size

0b6a30d
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs