Sitelet https://github.com/astra-sim/astra-sim/pull/343
Skip to content

Symbolic Representation of Collective Size - #343

Draft
jinsun-yoo wants to merge 1 commit into
astra-sim:masterfrom
jinsun-yoo:comm_size
Draft

jinsun-yoo wants to merge 1 commit into
astra-sim:masterfrom
jinsun-yoo:comm_size

Conversation

@jinsun-yoo

Copy link
Copy Markdown
Collaborator

Ref: collectiveapi PR 1

Summary

Previously custom collectives had the message size of each send/recv message hardcoded into the custom collective et, which overrode the message size in the workload chakra ET (undesirable!)

This PR resolves this by replacing the hardcoded message size with a symbolic representation of the message size in the custom collective et. The custom collective et now holds SEND 1 chunk out of 8 chunks insead of SEND 1MB of data. The system layer then resolves the message size.

DRAFT PR

This is a draft PR. The feature is functional, but some stuff needs to be done:

  • Add more examples.
  • Add reproducing scripts.
  • Comments/documentation
  • Validate how the collective size works for all gather (local buffer v. global buffer)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant