JIT: less loop inversion for bottom tested loops with no evident IV by AndyAyersMS · Pull Request #130368 · dotnet/runtime

AndyAyersMS · 2026-07-08T17:53:41Z

Make loop inversion a bit less aggressive.

If a loop is bottom-tested and has no evident IV, only invert if there is a hint that inversion might lead to a beneficial CSE (call or possibly invariant load). Analysis is a approximate and piggy backs on existing IR walks we already do.

Fixes #130045.

Note

This change and PR description were produced with GitHub Copilot CLI.

Under JitLoopInversionRequireBenefitForBottomTested (off by default), skip inverting an already bottom-tested loop with no recognized IV unless the duplicated condition holds a call or a loop-invariant, hoistable load. Targets the arm64 regressions in dotnet#130045 while keeping the Dictionary Enumerator.MoveNext inversion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

dotnet-policy-service · 2026-07-08T17:54:50Z

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

AndyAyersMS · 2026-07-08T17:56:32Z

@EgorBot -arm64 --envvars DOTNET_JitLoopInversionRequireBenefitForBottomTested:1

using System;
using System.Numerics;
using BenchmarkDotNet.Attributes;

public class Bench
{
    private readonly int[] _a = new int[512];
    private readonly int[] _b = new int[512];
    private BigInteger _x, _y;

    [GlobalSetup]
    public void Setup()
    {
        for (int i = 0; i < _a.Length; i++) { _a[i] = i; _b[i] = i; }
        byte[] bytes = new byte[259];
        new Random(42).NextBytes(bytes);
        bytes[^1] &= 0x7f;
        _x = new BigInteger(bytes);
        _y = new BigInteger(bytes);
    }

    [Benchmark]
    public bool Span_SequenceEqual() => _a.AsSpan().SequenceEqual(_b);

    [Benchmark]
    public int Span_IndexOf_NotFound() => _a.AsSpan().IndexOf(-1);

    [Benchmark]
    public int BigInteger_CompareTo() => _x.CompareTo(_y);

    [Benchmark]
    public bool BigInteger_Equals() => _x.Equals(_y);
}

Base is main (the env var names a config it doesn't have, so it runs the current #129868 codegen); the PR side has the gate enabled — so the diff isolates the fix on these #130045 cases.

Note

Comment generated with GitHub Copilot CLI.

Copilot

Pull request overview

This PR adds an (opt-in) heuristic gate to optTryInvertWhileLoop so that when a loop is already bottom-tested and AnalyzeIteration recognizes no IV, loop inversion is only performed if duplicating the condition block appears to have a “benefit” (call or potentially hoistable memory load). It also introduces a new JIT config knob to enable this gate.

Changes:

Track “exiting cond latch seen” separately from “IV-test latch” and only apply the new gate in the bottom-tested + no-IV case.
Classify condBlock for calls/indirections and use the existing loop size walk to conservatively detect stores to locals used in indirection address expressions.
Add JitLoopInversionRequireBenefitForBottomTested config (default off).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
`src/coreclr/jit/optimizer.cpp`	Adds the benefit-based gate and condition classification for bottom-tested no-IV loops.
`src/coreclr/jit/jitconfigvalues.h`	Introduces the new release config switch controlling the gate.

AndyAyersMS · 2026-07-08T20:33:11Z

Re-running on server arm64 (Cobalt 100) — the earlier -arm64 landed on an Apple M4, which doesn't share the #130045 microarchitecture.

@EgorBot -linux_arm64 --envvars DOTNET_JitLoopInversionRequireBenefitForBottomTested:1

using System;
using System.Numerics;
using BenchmarkDotNet.Attributes;

public class Bench
{
    private readonly int[] _a = new int[512];
    private readonly int[] _b = new int[512];
    private BigInteger _x, _y;

    [GlobalSetup]
    public void Setup()
    {
        for (int i = 0; i < _a.Length; i++) { _a[i] = i; _b[i] = i; }
        byte[] bytes = new byte[259];
        new Random(42).NextBytes(bytes);
        bytes[^1] &= 0x7f;
        _x = new BigInteger(bytes);
        _y = new BigInteger(bytes);
    }

    [Benchmark]
    public bool Span_SequenceEqual() => _a.AsSpan().SequenceEqual(_b);

    [Benchmark]
    public int Span_IndexOf_NotFound() => _a.AsSpan().IndexOf(-1);

    [Benchmark]
    public int BigInteger_CompareTo() => _x.CompareTo(_y);

    [Benchmark]
    public bool BigInteger_Equals() => _x.Equals(_y);
}

Base is main (env var names a config it lacks, so it runs the current #129868 codegen); PR side has the gate enabled.

Note

Comment generated with GitHub Copilot CLI.

AndyAyersMS · 2026-07-08T21:11:20Z

Adding the two biggest #130045 regressions (LastIndexOfAnyExcept("ßäöüÄÖÜ"), ContainsKeyTrue<int,int>.IDictionary) plus two anchors, across server arm64 (Cobalt) and x64. Note: the fleet has no Windows Zen4, so Windows here is Turin (Zen5); Linux x64 is Genoa (Zen4).

@EgorBot -linux_arm64 -ubuntu24_azure_genoa -windows_x64 --envvars DOTNET_JitLoopInversionRequireBenefitForBottomTested:1

using System;
using System.Buffers;
using System.Collections.Generic;
using System.Linq;
using System.Numerics;
using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes;

public class Bench
{
    private SearchValues<char> _searchValues;
    private char[] _textExcept;

    private int[] _found;
    private Dictionary<int, int> _dictionary;

    private readonly int[] _a = new int[512];
    private readonly int[] _b = new int[512];
    private BigInteger _x, _y;

    [GlobalSetup]
    public void Setup()
    {
        _searchValues = SearchValues.Create("ßäöüÄÖÜ");
        _textExcept = new string('ß', 256).ToCharArray();
        _textExcept[128] = '\n';

        _found = Enumerable.Range(0, 512).ToArray();
        _dictionary = _found.ToDictionary(k => k, k => k);

        for (int i = 0; i < _a.Length; i++) { _a[i] = i; _b[i] = i; }
        byte[] bytes = new byte[259];
        new Random(42).NextBytes(bytes);
        bytes[^1] &= 0x7f;
        _x = new BigInteger(bytes);
        _y = new BigInteger(bytes);
    }

    [Benchmark]
    public int LastIndexOfAnyExcept() => _textExcept.AsSpan().LastIndexOfAnyExcept(_searchValues);

    [Benchmark]
    public bool ContainsKeyTrue_IDictionary() => ContainsKey(_dictionary);

    [MethodImpl(MethodImplOptions.NoInlining)]
    private bool ContainsKey(IDictionary<int, int> collection)
    {
        bool result = false;
        var found = _found;
        for (int i = 0; i < found.Length; i++)
            result ^= collection.ContainsKey(found[i]);
        return result;
    }

    [Benchmark]
    public bool Span_SequenceEqual() => _a.AsSpan().SequenceEqual(_b);

    [Benchmark]
    public int BigInteger_CompareTo() => _x.CompareTo(_y);
}

Base is main (env var names a config it lacks, running the current #129868 codegen); PR side has the gate enabled.

Note

Comment generated with GitHub Copilot CLI.

AndyAyersMS · 2026-07-08T21:12:53Z

Also on Ampere (Neoverse-N1) — the core class behind the arm64-ubuntu lab queue, distinct from Cobalt's N2.

@EgorBot -ubuntu24_azure_ampere --envvars DOTNET_JitLoopInversionRequireBenefitForBottomTested:1

using System;
using System.Buffers;
using System.Collections.Generic;
using System.Linq;
using System.Numerics;
using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes;

public class Bench
{
    private SearchValues<char> _searchValues;
    private char[] _textExcept;

    private int[] _found;
    private Dictionary<int, int> _dictionary;

    private readonly int[] _a = new int[512];
    private readonly int[] _b = new int[512];
    private BigInteger _x, _y;

    [GlobalSetup]
    public void Setup()
    {
        _searchValues = SearchValues.Create("ßäöüÄÖÜ");
        _textExcept = new string('ß', 256).ToCharArray();
        _textExcept[128] = '\n';

        _found = Enumerable.Range(0, 512).ToArray();
        _dictionary = _found.ToDictionary(k => k, k => k);

        for (int i = 0; i < _a.Length; i++) { _a[i] = i; _b[i] = i; }
        byte[] bytes = new byte[259];
        new Random(42).NextBytes(bytes);
        bytes[^1] &= 0x7f;
        _x = new BigInteger(bytes);
        _y = new BigInteger(bytes);
    }

    [Benchmark]
    public int LastIndexOfAnyExcept() => _textExcept.AsSpan().LastIndexOfAnyExcept(_searchValues);

    [Benchmark]
    public bool ContainsKeyTrue_IDictionary() => ContainsKey(_dictionary);

    [MethodImpl(MethodImplOptions.NoInlining)]
    private bool ContainsKey(IDictionary<int, int> collection)
    {
        bool result = false;
        var found = _found;
        for (int i = 0; i < found.Length; i++)
            result ^= collection.ContainsKey(found[i]);
        return result;
    }

    [Benchmark]
    public bool Span_SequenceEqual() => _a.AsSpan().SequenceEqual(_b);

    [Benchmark]
    public int BigInteger_CompareTo() => _x.CompareTo(_y);
}

Base is main (env var names a config it lacks, running the current #129868 codegen); PR side has the gate enabled.

Note

Comment generated with GitHub Copilot CLI.

AndyAyersMS · 2026-07-08T23:48:38Z

@EgorBot -ubuntu24_azure_ampere --envvars DOTNET_JitLoopInversionRequireBenefitForBottomTested:1 --filter "ContainsKeyTrueInt32IDictionary"

Real perf-repo benchmark this time (random ValuesGenerator keys). Base is main (env var names a config it lacks → current #129868 codegen); PR side has the gate enabled.

Note

Comment generated with GitHub Copilot CLI.

AndyAyersMS · 2026-07-09T00:46:03Z

Retry of the ContainsKeyTrue<Int32,Int32>.IDictionary filter run (prior attempt returned NA on the PR side with a bare BDN RuntimeError — no exception, and it reproduces neither in a local osx-arm64 run of the same benchmark under the config nor in SPMI replay, so it looks like a transient harness hiccup).

@EgorBot -ubuntu24_azure_ampere --envvars DOTNET_JitLoopInversionRequireBenefitForBottomTested:1 --filter "ContainsKeyTrueInt32IDictionary"

Base is main (env var names a config it lacks → current #129868 codegen); PR side has the gate enabled.

Note

Comment generated with GitHub Copilot CLI.

AndyAyersMS · 2026-07-09T01:31:34Z

EgorBot validation summary

Ran the #130045 cases across the hardware classes behind the perf lab, comparing main (current #129868 codegen) vs this PR with the gate enabled (--envvars DOTNET_JitLoopInversionRequireBenefitForBottomTested:1; main ignores the unknown config, so it serves as the regressed baseline).

Hardware → lab mapping: Ampere N1 = arm64-ubuntu, Cobalt N2 = arm64-azlinux, Genoa Zen4 = Linux/x64, Apple M4 = not a lab target.

The two biggest regressions recover to their lab baselines

Benchmark (hardware)	lab baseline → regressed	EgorBot: main → PR
`ContainsKeyTrue<int,int>.IDictionary` (Ampere N1)	3.36 µs → 4.85 µs	5.09 µs → 3.42 µs (−33%)
`SearchValuesCharTests.LastIndexOfAnyExcept` (Genoa Zen4)	87.8 ns → 131.9 ns	140.1 ns → 99.8 ns (−29%)

The PR numbers land right on the pre-#129868 lab baselines.

Full matrix (fix effect on the benchmark's Mean; negative = faster)

Benchmark	Apple M4	Ampere N1	Cobalt N2	Genoa Zen4
`BigInteger.CompareTo`	−44%	−49%	−36%	−21%
`Span<int>.SequenceEqual`	~0%	−15%	−6.6%	+4.4%
`LastIndexOfAnyExcept`	—	~0%	~0%	−29%
`ContainsKeyTrue.IDictionary`	—	−33%	~0%	~0%
`Span<int>.IndexOf` (not found)	+6%	—	+2%	—

Reading it

Every benchmark recovers on the hardware where it actually regressed: ContainsKeyTrue and SequenceEqual on Ampere N1, LastIndexOfAnyExcept on Zen4, BigInteger.CompareTo everywhere.
The give-back is small and localized to Span scans on cores where that inversion is genuinely beneficial (SequenceEqual +4% on Zen4, IndexOf +2–6% on M4/N2).
ContainsKeyTrue/LastIndexOfAnyExcept show ~0% on non-matching cores because they never regressed there (LastIndexOfAnyExcept was an x64-only regression; it improved on x86).

Notes: ContainsKeyTrue and LastIndexOfAnyExcept use --filter against the real perf-repo benchmarks; the rest are inline repros. One --filter run returned a transient NA on the PR side (no exception; not reproducible locally or in SPMI) and passed on retry.

Note

Comment generated with GitHub Copilot CLI.

Defer the benefit-gate bitvec allocation to when the config is on, correct the "indirection address locals" wording, and soften the config comment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

…bottom-tested-benefit

Remove the JitLoopInversionRequireBenefitForBottomTested config and always apply the gate: for an already bottom-tested loop with no recognized IV, only invert when the duplicated condition has a call or an indirection whose address has no loop-varying local. Also address review feedback: track GT_LCL_ADDR in indirection addresses via OperIsAnyLocal, and describe the check as an indirection rather than a load. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

The message claimed a "hoistable" benefit, but the check only requires a call or an indirection whose address has no loop-varying local (which does not by itself prove the load is hoistable). Describe it accurately. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

AndyAyersMS · 2026-07-15T00:54:38Z

@jakobbotsch PTAL
fyi @dotnet/jit-contrib

azure-pipelines · 2026-07-15T00:54:51Z

Azure Pipelines: Successfully started running 5 pipeline(s). 10 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI review requested due to automatic review settings July 8, 2026 17:53

github-actions Bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jul 8, 2026

Copilot started reviewing on behalf of AndyAyersMS July 8, 2026 17:54 View session

dotnet-policy-service Bot assigned AndyAyersMS Jul 8, 2026

EgorBot mentioned this pull request Jul 8, 2026

Benchmarks for dotnet/runtime#130368 (for @AndyAyersMS) EgorBot/Benchmarks#294

Open

Copilot AI reviewed Jul 8, 2026

View reviewed changes

Comment thread src/coreclr/jit/optimizer.cpp Outdated

Comment thread src/coreclr/jit/optimizer.cpp Outdated

Comment thread src/coreclr/jit/jitconfigvalues.h Outdated

EgorBot mentioned this pull request Jul 8, 2026

Benchmarks for dotnet/runtime#130368 (for @AndyAyersMS) EgorBot/Benchmarks#295

Open

EgorBot mentioned this pull request Jul 8, 2026

Benchmarks for dotnet/runtime#130368 (for @AndyAyersMS) EgorBot/Benchmarks#296

Open

EgorBot mentioned this pull request Jul 8, 2026

Benchmarks for dotnet/runtime#130368 (for @AndyAyersMS) EgorBot/Benchmarks#297

Open

EgorBot mentioned this pull request Jul 8, 2026

Benchmarks for dotnet/runtime#130368 (for @AndyAyersMS) EgorBot/Benchmarks#298

Open

EgorBot mentioned this pull request Jul 9, 2026

Benchmarks for dotnet/runtime#130368 (for @AndyAyersMS) EgorBot/Benchmarks#299

Open

build-analysis Bot mentioned this pull request Jul 9, 2026

The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008

Open

3 tasks

JIT: address review feedback; clang-format

b016750

Defer the benefit-gate bitvec allocation to when the config is on, correct the "indirection address locals" wording, and soften the config comment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings July 9, 2026 01:39

Copilot started reviewing on behalf of AndyAyersMS July 9, 2026 01:40 View session

Copilot AI reviewed Jul 9, 2026

View reviewed changes

Comment thread src/coreclr/jit/optimizer.cpp

Comment thread src/coreclr/jit/jitconfigvalues.h Outdated

AndyAyersMS and others added 2 commits July 14, 2026 11:11

Merge remote-tracking branch 'upstream/main' into jit-loop-inversion-…

cccc4bd

…bottom-tested-benefit

Copilot AI review requested due to automatic review settings July 14, 2026 22:16

Copilot started reviewing on behalf of AndyAyersMS July 14, 2026 22:17 View session

Copilot AI reviewed Jul 14, 2026

View reviewed changes

Comment thread src/coreclr/jit/optimizer.cpp

Comment thread src/coreclr/jit/optimizer.cpp

Copilot AI review requested due to automatic review settings July 14, 2026 22:23

Copilot started reviewing on behalf of AndyAyersMS July 14, 2026 22:24 View session

AndyAyersMS changed the title ~~JIT: gate inversion of bottom-tested no-IV loops on a hoisting benefit~~ JIT: gate inversion of bottom-tested no-IV loops on a call or non-loop-varying indirection Jul 14, 2026

Copilot AI reviewed Jul 14, 2026

View reviewed changes

Comment thread src/coreclr/jit/optimizer.cpp

AndyAyersMS changed the title ~~JIT: gate inversion of bottom-tested no-IV loops on a call or non-loop-varying indirection~~ JIT: less loop inversion for bottom tested loops with no evident IV Jul 15, 2026

AndyAyersMS marked this pull request as ready for review July 15, 2026 00:54

AndyAyersMS requested a review from jakobbotsch July 15, 2026 00:55

Uh oh!

Conversation

AndyAyersMS commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dotnet-policy-service Bot commented Jul 8, 2026

Uh oh!

AndyAyersMS commented Jul 8, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AndyAyersMS commented Jul 8, 2026

Uh oh!

AndyAyersMS commented Jul 8, 2026

Uh oh!

AndyAyersMS commented Jul 8, 2026

Uh oh!

AndyAyersMS commented Jul 8, 2026

Uh oh!

AndyAyersMS commented Jul 9, 2026

Uh oh!

AndyAyersMS commented Jul 9, 2026

EgorBot validation summary

The two biggest regressions recover to their lab baselines

Full matrix (fix effect on the benchmark's Mean; negative = faster)

Reading it

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

AndyAyersMS commented Jul 15, 2026

Uh oh!

azure-pipelines Bot commented Jul 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

AndyAyersMS commented Jul 8, 2026 •

edited

Loading