load_aware_locality: implement lb policy by jukie · Pull Request #45725 · envoyproxy/envoy

jukie · 2026-06-21T02:56:05Z

Commit Message: load_aware_locality: implement ORCA utilization-based locality picking

Additional Description:
Implements the envoy.load_balancing_policies.load_aware_locality extension added in #44341 which is a locality-picking LB policy that weights localities by backend utilization aggregated from ORCA load reports.

Some notes:

Currently in-band reports only. Out of band reporting will be supported but depends on orca: decouple OOB report delivery from CSWRR weight manager #45953.
Please see caveats in policy rst documentation, mostly applicable for hash-based or async (dynamic module LB) endpoint picking policies.

Risk Level: low/medium - opt-in extension that is still work-in-progress
Testing: Added unit and integration tests, also verified in a real environment.
Docs Changes: Un-orphaned docs/root/intro/arch_overview/upstream/load_balancing/load_aware_locality.rst which will now be rendered in site docs
Release Notes: Added
Platform Specific Features: N/A
Fixes #43665

AI disclosure: portions of this PR and it’s tests were developed with LLM but I have reviewed and understand the code

repokitteh-read-only · 2026-06-21T02:56:10Z

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #45725 was opened by jukie.

see: more, trace.

jukie · 2026-06-21T04:15:44Z

/coverage

repokitteh-read-only · 2026-06-21T04:15:49Z

Coverage for this Pull Request will be rendered here:

https://storage.googleapis.com/envoy-cncf-pr/45725/coverage/index.html

For comparison, current coverage on main branch is here:

https://storage.googleapis.com/envoy-cncf-postsubmit/main/coverage/index.html

The coverage results are (re-)rendered each time the CI Envoy/Checks (coverage) job completes.

🐱

Caused by: a #45725 (comment) was created by @jukie.

see: more, trace.

Resolve the endpoint-picking child policy, validate policy-level knobs (update period, smoothing constant, variance and probe fractions), reject unsupported enable_oob_load_report, and register the LoadAwareLocality factory. Promote the extension from wip to alpha. Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

Add LocalityLbHostData, the per-host LB policy data that receives ORCA load reports via the multi-slot HostLbPolicyData mechanism and stores a lock-free utilization value behind a release/acquire freshness timestamp for the main-thread weight computation. Reports without a positive utilization signal are dropped (matching gRFC A58 / CSWRR) so signal-less hosts age out rather than pinning their locality as fresh-and-idle. Also lands two foundational types used by the config factory: the LoadAwareLocalityStats counter struct and the LoadAwareLocalityLbConfig holder. Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

Add the immutable RoutingWeightsSnapshot (per-priority, per-source locality weights keyed by Locality identity) and the WorkerLocalLbFactory that owns the shared child ThreadAwareLoadBalancer, publishes snapshots to workers through a thread-local slot, and instantiates the worker LB and its per-locality child LBs. Keying weights by Locality identity keeps the main-thread-to-worker mapping stable when localities are added or removed between snapshot ticks. Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

Add WorkerLocalLb, the per-worker data-path load balancer. It derives LoadBalancerBase for live priority, health, and panic selection, chooses a locality within the selected priority/source by capacity-weighted random over the routing snapshot (looked up by Locality identity), and delegates endpoint selection to a per-locality child LB. Per-source child LBs (healthy/degraded/all-hosts) are built per locality and re-synced on membership changes, with topology rebuilds gated on an actual membership delta. Endpoint retry stays with the child LB; a stack-scoped context wrapper suppresses re-running priority and hash policies, and async child selection is rejected to protect that wrapper. Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

Add LoadAwareLocalityLoadBalancer, the main-thread ThreadAwareLoadBalancer. On a periodic timer it reads each host's ORCA utilization and computes per-locality routing weights for the healthy, degraded, and all-host source slices: EWMA-smoothed headroom (host_count * (1 - smoothed_util)), local preference when cross-locality variance is below threshold, and a small remote probe fraction to keep remote signal fresh. The result is published to workers as an immutable snapshot. Attaches LocalityLbHostData to hosts on membership changes and emits the all_overloaded, local_preferred, probe_active, and stale_locality counters. Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

Bring the load_aware_locality architecture doc in line with the implementation, un-orphan it into the load balancing toctree, and add the new-feature changelog entry. The doc covers the five-stage locality weight computation, the child endpoint-picking policy, the emitted stats, and migration notes from zone-aware routing. Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

Add config, unit, and integration tests. config_test covers policy-knob validation and child-policy resolution; the unit test covers the main-thread weight computation (headroom, EWMA stale-carry, local preference, remote probe, all-overloaded host-count fallback) and the worker-local LB (live priority/panic selection, capacity-weighted locality selection, child delegation, and membership churn); the integration test exercises end-to-end ORCA-driven locality routing. Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

…gle-pass locality pick Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

jukie · 2026-07-06T02:14:15Z

/coverage

repokitteh-read-only · 2026-07-06T02:14:20Z

Coverage for this Pull Request will be rendered here:

https://storage.googleapis.com/envoy-cncf-pr/45725/coverage/index.html

For comparison, current coverage on main branch is here:

https://storage.googleapis.com/envoy-cncf-postsubmit/main/coverage/index.html

The coverage results are (re-)rendered each time the CI Envoy/Checks (coverage) job completes.

🐱

Caused by: a #45725 (comment) was created by @jukie.

see: more, trace.

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

…l_active stat, review cleanups Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

jukie · 2026-07-06T21:37:07Z

+  if (lb_config.has_enable_oob_load_report() && lb_config.enable_oob_load_report().value()) {
+    return absl::InvalidArgumentError(
+        "load_aware_locality: enable_oob_load_report is not yet supported");
+  }
+  if (lb_config.has_oob_reporting_period()) {
+    return absl::InvalidArgumentError("load_aware_locality: oob_reporting_period requires "
+                                      "enable_oob_load_report, which is not yet supported");
+  }


Can remove rejection if preferred but OOB is not currently wired up and depends on #45953

jukie · 2026-07-06T21:38:33Z

 Utilization is derived from each host's ORCA report using the same
-extraction as CSWRR (precedence may be flipped by the
+extraction as CSWRR. The runtime flag
 ``envoy.reloadable_features.orca_weight_manager_use_named_metrics_first``
-runtime feature). By default:
+defaults to ``true``, so the default precedence is:

-1. ``application_utilization`` -- value in [0, 1], used when reported and
-   greater than 0.
-2. Named metrics via ``metric_names_for_computing_utilization`` -- max of
-   present values, used when ``application_utilization`` is not reported.
+1. Named metrics via ``metric_names_for_computing_utilization`` -- max of
+   present values, used when the result is greater than 0.
+2. ``application_utilization`` -- used when greater than 0.
 3. ``cpu_utilization`` -- final fallback.


Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

jukie · 2026-07-06T22:15:54Z

/gemini review

gemini-code-assist

Code Review

This pull request implements the envoy.load_balancing_policies.load_aware_locality load balancer policy, which weights localities based on ORCA-derived utilization headroom and supports in-band ORCA reporting across all priority levels. The changes include configuration parsing, main-thread and worker-local load balancing logic, and comprehensive unit, configuration, and integration tests. Feedback on the pull request points out that the integration test target in the BUILD file should list the envoy.load_balancing_policies.round_robin extension in its extension_names attribute, as it is instantiated during the integration test.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

gemini-code-assist · 2026-07-06T22:18:38Z

+    name = "integration_test",
+    size = "large",
+    srcs = ["integration_test.cc"],
+    extension_names = ["envoy.load_balancing_policies.load_aware_locality"],


According to the general rules, for integration tests using envoy_extension_cc_test, all production extensions instantiated by the integration configuration must be listed in the extension_names attribute. Since the integration test instantiates the envoy.load_balancing_policies.round_robin extension, it should be added to extension_names.

extension_names = [ "envoy.load_balancing_policies.load_aware_locality", "envoy.load_balancing_policies.round_robin", ],

References

For integration tests using envoy_extension_cc_test, list all production extensions that the integration configuration instantiates in the extension_names attribute. Do not add these extra extension names to config or unit tests, only to the integration test target.

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

jukie force-pushed the load-aware-lb-impl branch 2 times, most recently from ac7a860 to 2f0a702 Compare June 25, 2026 06:22

jukie force-pushed the load-aware-lb-impl branch from 8d276fd to 6221cab Compare July 2, 2026 04:26

jukie added 9 commits July 3, 2026 13:19

move child LB ownership to main thread, enforce 100ms tick floor, sin…

3fb4a75

…gle-pass locality pick Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

absl --> std

8acb258

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

jukie force-pushed the load-aware-lb-impl branch from 6221cab to 8acb258 Compare July 5, 2026 23:44

jukie added 2 commits July 5, 2026 18:30

Fix std typos

2523af0

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

trim tests, comments, and use peekAnotherHost/chooseHost helper

71ace56

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

jukie added 2 commits July 5, 2026 21:18

spellcheck and coverage

358f46a

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

priority-growth sync fixes, weight-cache refresh on raw compare, spil…

c857553

…l_active stat, review cleanups Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

jukie changed the title ~~lb-policies: Implement LoadAwareLocality Lb~~ load_aware_locality: implement lb policy Jul 6, 2026

jukie commented Jul 6, 2026

View reviewed changes

Flip back to WIP

2650f89

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

gemini-code-assist Bot reviewed Jul 6, 2026

View reviewed changes

jukie added 2 commits July 6, 2026 16:48

Include round_robin in test extension_names

21fbca1

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

more review fixes

fb02adb

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

load_aware_locality: implement lb policy#45725

load_aware_locality: implement lb policy#45725
jukie wants to merge 16 commits into
envoyproxy:mainfrom
jukie:load-aware-lb-impl

jukie commented Jun 21, 2026 •

edited

Loading

Uh oh!

repokitteh-read-only Bot commented Jun 21, 2026

Uh oh!

jukie commented Jun 21, 2026

Uh oh!

repokitteh-read-only Bot commented Jun 21, 2026

Uh oh!

jukie commented Jul 6, 2026

Uh oh!

repokitteh-read-only Bot commented Jul 6, 2026

Uh oh!

jukie Jul 6, 2026

Uh oh!

jukie Jul 6, 2026

Uh oh!

jukie commented Jul 6, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot Jul 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

jukie commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

repokitteh-read-only Bot commented Jun 21, 2026

Uh oh!

jukie commented Jun 21, 2026

Uh oh!

repokitteh-read-only Bot commented Jun 21, 2026

Uh oh!

jukie commented Jul 6, 2026

Uh oh!

repokitteh-read-only Bot commented Jul 6, 2026

Uh oh!

jukie Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

jukie Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

jukie commented Jul 6, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jukie commented Jun 21, 2026 •

edited

Loading