Record every cabled pair in a jaccl-ring hostfile - #4269
Open
erwinzhang7 wants to merge 1 commit into
Open
Conversation
configure_jaccl_ring wrote None for every pair that was not a ring neighbour, including pairs that are physically cabled. configure_jaccl emits the full matrix, so the connectivity survived in a mesh hostfile and was discarded in a ring one, before any C++ saw it. The information was already there. IPConfigurator keys on every connected Thunderbolt port and assigns addresses for each, and its setup() brings up an interface for every pair it holds, so the machines are cabled and configured for the whole topology while the hostfile describes a subset. Nothing about how the group runs changes. MLX_JACCL_RING selects the ring, not the shape of the matrix, and launch_jaccl only requires the matrix to be square with a null diagonal, both of which still hold. The ring neighbours keep the width the ring was built with. What it buys is that a link is no longer lost. A subgroup can only contain members that are directly connected, so the pairs dropped here are exactly the ones a future split would need, and the hostfile is everything the runtime is ever given.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
configure_jaccl_ringwritesNonefor every pair that is not a ring neighbour, includingpairs that are physically cabled:
configure_jaccla few lines above emits the full N x N matrix, so connectivity survives in amesh hostfile and is discarded in a ring one, before anything in C++ sees it.
The information is already available.
IPConfiguratorkeys on every connected Thunderbolt portand assigns addresses per cable, and its
setup()brings up an interface for each pair itholds, so the machines are cabled and configured for the whole topology while the hostfile
describes a subset of it.
Nothing changes how the group runs.
MLX_JACCL_RINGselects the ring, not the shape ofthe matrix, and
launch_jacclonly requires the matrix to be square with a null diagonal. Bothstill hold, ring neighbours keep the width the ring was built with, and a hostfile for a
topology with no extra cables is byte-identical to before.
What it buys is that a link is no longer lost. A subgroup can only contain members that are
directly connected, so the pairs dropped here are the ones
Group.split()would need, and thehostfile is everything the runtime is given. Related to #3205.
The matrix construction moves into
jaccl_ring_devices(ring, count, ips), a pure function overa device map, so it is testable without a cluster. Five tests cover a plain ring, a ring with a
chord, the square-and-null-diagonal contract
launch_jacclenforces, ring ordering, andmultiple cables between neighbours.