JIT: Rematerialize spilled FP/SIMD/mask constants instead of stack spill/reload#130313
Open
tannergooding wants to merge 8 commits into
Open
JIT: Rematerialize spilled FP/SIMD/mask constants instead of stack spill/reload#130313tannergooding wants to merge 8 commits into
tannergooding wants to merge 8 commits into
Conversation
…ill/reload Floating-point, SIMD, and mask constants (CNS_DBL, CNS_VEC, CNS_MSK) have no GC liveness and can be rematerialized cheaply on xarch without a scratch register (zero via xorps, all-bits-set via pcmpeqd/vpternlogd, or a RIP-relative load from the constant's memory home). Rather than spilling such a value to the stack and reloading it, skip creating the spill temp and store in genProduceReg, and rematerialize the constant at the reload point in genUnspillRegIfNeeded. A shared predicate isRematerializableConstant gates both sites and is currently scoped to TARGET_XARCH; other targets can be enabled later for the subset of values that likewise need no scratch register. Values consumed directly from the spill temp as a contained memory operand (GTF_NOREG_AT_USE) continue to be spilled normally. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Extend the constant rematerialization added for xarch to Arm64, for the subset of CNS_DBL/CNS_VEC/CNS_MSK values that genSetRegToConst encodes directly into an instruction without a scratch register: 0.0/fmov-immediate doubles, zero/all-bits- set/movi-broadcast vectors, and all masks (pfalse/ptrue). Values that Arm64 would otherwise load from the constant pool via adrp/ldr need a scratch address register that is not reserved at the reload point, so those remain spilled. isRematerializableConstant gains an Arm64 branch mirroring the no-temp fast paths in genSetRegToConst, and genUnspillRegIfNeeded rematerializes into the reload target via the GenTree* overload of genSetRegToConst (which honors the explicit target register on Arm64). The genProduceReg skip-store site is already gated by the predicate. SPMI asmdiffs (coreclr_tests, windows-arm64): 554 methods improved, 0 regressions, -5432 bytes, with spill store/reload pairs replaced by movi/fmov/mvni/pfalse/ptrue. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
github-actions
Bot
added
the
area-CodeGen-coreclr
Contributor
|
Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR changes CoreCLR JIT codegen to avoid stack spill/reload for certain spilled FP/SIMD/mask constants by rematerializing them at the reload point instead. It introduces a shared predicate to ensure the optimization is only applied when the constant can be materialized without requiring scratch registers (and without GC bookkeeping).
Changes:
- Update
CodeGen::genUnspillRegIfNeededto rematerialize eligible spilledGT_CNS_DBL/GT_CNS_VEC/GT_CNS_MSKinto the reload target register (xarch/arm64) instead of loading from a spill temp. - Add
CodeGen::isRematerializableConstantto centralize the eligibility logic and keep the spill-skip and reload-remat sites consistent. - Update
CodeGen::genProduceRegto skip creating a spill temp / emitting the spill store for eligible constants (while preserving the existing behavior forGTF_NOREG_AT_USEcases that must read from the spill temp as a contained memory operand).
Show a summary per file
| File | Description |
|---|---|
| src/coreclr/jit/codegenlinear.cpp | Implements the rematerialization path on unspill, adds the shared predicate, and skips spill-temp/store creation for eligible constants. |
| src/coreclr/jit/codegen.h | Declares the new isRematerializableConstant(GenTree*) helper on CodeGen. |
Copilot's findings
- Files reviewed: 2/2 changed files
- Comments generated: 0
When a rematerializable FP/SIMD/mask constant is spilled immediately after its definition (GTF_SPILL with spill-after semantics), the value is rematerialized at each reload point and its stack store is skipped. The original definition therefore materializes a value into a register that is never read. Add a shared isRematerializedConstantSpill predicate and use it to also skip emitting the definition (genSetRegToConst) at the genCodeForTreeNode sites on xarch and arm64, in addition to the existing store-skip in genProduceReg. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The data section packed entries sequentially with no per-entry alignment padding, so emitDataGenFind's ((curOffs % alignment) == 0) guard could reject a valid match when the existing constant landed at a misaligned offset, emitting a duplicate. Give each dataSection a dsOffset computed via AlignUp in the begin-functions and use it as the single source of truth for lookup, output, and display. This honors the requested alignment (a bug if it wasn't) while SMALL_CODE still gets tight packing via its smaller requested alignment. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
tannergooding
requested review from
a team,
AaronRobinsonMSFT and
janvorli
as code owners
Member
Author
Sorry for the noise. The |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Floating-point, SIMD, and mask constants (
CNS_DBL,CNS_VEC,CNS_MSK) have no GC liveness and, for a well-defined subset of values, can be rematerialized cheaply without needing a scratch register. Rather than spilling such a value to the stack and reloading it, this PR skips creating the spill temp/store ingenProduceRegand instead rematerializes the constant at the reload point ingenUnspillRegIfNeeded.Details
A shared predicate
isRematerializableConstantgates both the skip-store site and the rematerialization site. It only returnstruefor values thatgenSetRegToConstcan encode directly, i.e. that need no scratch register:xorps), all-bits-set (pcmpeqd/vpternlogd), or a RIP-relative load from the constant''s memory home.0.0/fmov-immediate doubles, zero/all-bits-set/movi-broadcast vectors, and all masks (pfalse/ptrue). Values that Arm64 would otherwise load from the constant pool viaadrp/ldrneed a scratch address register that is not reserved at the reload point, so those remain spilled.Rematerialization goes through the
GenTree*overload ofgenSetRegToConst, which honors the explicit target register. Values consumed directly from the spill temp as a contained memory operand (GTF_NOREG_AT_USE) continue to be spilled normally.Diffs
SPMI asmdiffs (coreclr_tests, windows-arm64): 554 methods improved, 0 regressions, -5432 bytes, with spill store/reload pairs replaced by
movi/fmov/mvni/pfalse/ptrue.Note
This PR description was generated with the assistance of GitHub Copilot.