internal/db2/history: rewrite /assets pagination as bounded CTE merge query

[hexdigest]

PR Checklist

PR Structure

• This PR has reasonably narrow scope (if not, break it down into smaller PRs).
• This PR avoids mixing refactoring changes with feature changes (split into two PRs otherwise).
• This PR's title starts with name of package that is most changed in the PR, ex. services/friendbot, or all or doc if the changes are broad or impact many packages.

Thoroughness

• This PR adds tests for the most critical parts of the new functionality or fixes.
• I've updated any docs (developer docs (https://developers.stellar.org/api/), .md files, etc... affected by this change). Take a look in the docs folder for a given service, like this one (https://github.com/stellar/stellar-horizon/tree/main/internal/docs).

Release planning

• I've reviewed the changes in this PR and if I consider them worthwhile for being mentioned on release notes then I have updated the relevant CHANGELOG.md within the component folder structure. For example, if I changed horizon, then I updated (CHANGELOG.md (CHANGELOG.md). I add a new line item describing the change and reference to this PR. If I don't update a CHANGELOG, I acknowledge this PR's change may not be mentioned in future release notes.

• I've decided if this PR requires a new major/minor version according to semver (https://semver.org/), or if it's mainly a patch change. The PR is targeted at the next release branch if it's not a patch change.

What

Rewrite internal/db2/history /assets pagination to use a bounded CTE merge query instead of paginating across a FULL OUTER JOIN with COALESCE(...).

The new query:

• reads exp_asset_stats rows in one indexed branch
• reads contract-only asset_contracts rows in a second indexed branch
• merges them in SQL with CTEs and UNION ALL
• applies the final ORDER BY and LIMIT over the bounded merged result set

This preserves the existing response semantics while avoiding full-table scan and sort behavior.

Why

The previous /assets query paginated over a FULL OUTER JOIN and used COALESCE(...) in both filtering and ordering. That made the query non-sargable, so Postgres could not use the existing indexes effectively.

For the request:

/assets?asset_code=USDC&asset_issuer=GA5ZSEJYB37JRC5AVCIA5MOP4RHTM335X2KGX3IHOJAPP5RE34K4KZVN&limit=1
the EXPLAIN ANALYZE results were:

Before

  Limit  (cost=103.26..3168.04 rows=1 width=596) (actual time=130.211..130.214 rows=1 loops=1)
    Buffers: shared hit=19625
    ->  Nested Loop Left Join  (cost=103.26..30751.01 rows=10 width=596) (actual time=130.210..130.212 rows=1 loops=1)
          Buffers: shared hit=19625
          ->  Hash Full Join  (cost=102.99..30747.92 rows=10 width=501) (actual time=130.184..130.186 rows=1 loops=1)
                Hash Cond: ((exp_asset_stats.asset_type = asset_contracts.asset_type) AND ((exp_asset_stats.asset_code)::text = (asset_contracts.asset_code)::text) AND
                ((exp_asset_stats.asset_issuer)::text = (asset_contracts.asset_issuer)::text))
                Filter: (((COALESCE(exp_asset_stats.asset_code, asset_contracts.asset_code))::text = 'USDC'::text) AND ((COALESCE(exp_asset_stats.asset_issuer,
                asset_contracts.asset_issuer))::text = 'GA5ZSEJYB37JRC5AVCIA5MOP4RHTM335X2KGX3IHOJAPP5RE34K4KZVN'::text))
                Rows Removed by Filter: 334606
                Buffers: shared hit=19618
                ->  Seq Scan on exp_asset_stats  (cost=0.00..27634.62 rows=382262 width=401) (actual time=0.002..68.281 rows=334607 loops=1)
                      Buffers: shared hit=19571
                ->  Hash  (cost=67.36..67.36 rows=2036 width=100) (actual time=0.715..0.715 rows=2033 loops=1)
                      Buckets: 2048  Batches: 1  Memory Usage: 280kB
                      Buffers: shared hit=47
                      ->  Seq Scan on asset_contracts  (cost=0.00..67.36 rows=2036 width=100) (actual time=0.006..0.296 rows=2033 loops=1)
                            Buffers: shared hit=47
          ->  Index Scan using contract_asset_stats_pkey on contract_asset_stats  (cost=0.27..0.31 rows=1 width=87) (actual time=0.021..0.021 rows=1 loops=1)
                Index Cond: (contract_id = asset_contracts.contract_id)
                Buffers: shared hit=7
  Planning:
    Buffers: shared hit=9
  Planning Time: 0.465 ms
  Execution Time: 130.288 ms

After

  Limit  (cost=14.13..14.13 rows=1 width=353) (actual time=2.645..2.647 rows=1 loops=1)
    Buffers: shared hit=798 read=2
    I/O Timings: shared read=2.058
    ->  Sort  (cost=14.13..14.13 rows=2 width=353) (actual time=2.644..2.647 rows=1 loops=1)
          Sort Key: exp_asset_stats.asset_code, exp_asset_stats.asset_issuer
          Sort Method: quicksort  Memory: 26kB
          Buffers: shared hit=798 read=2
          I/O Timings: shared read=2.058
          ->  Append  (cost=0.97..14.12 rows=2 width=353) (actual time=2.403..2.625 rows=1 loops=1)
                Buffers: shared hit=795 read=2
                I/O Timings: shared read=2.058
                ->  Limit  (cost=0.97..7.05 rows=1 width=488) (actual time=2.403..2.404 rows=1 loops=1)
                      Buffers: shared hit=396 read=2
                      I/O Timings: shared read=2.058
                      ->  Nested Loop Left Join  (cost=0.97..7.05 rows=1 width=488) (actual time=2.402..2.403 rows=1 loops=1)
                            Buffers: shared hit=396 read=2
                            I/O Timings: shared read=2.058
                            ->  Nested Loop Left Join  (cost=0.70..4.75 rows=1 width=434) (actual time=2.384..2.385 rows=1 loops=1)
                                  Join Filter: (exp_asset_stats.asset_type = asset_contracts.asset_type)
                                  Buffers: shared hit=389 read=2
                                  I/O Timings: shared read=2.058
                                  ->  Index Scan using exp_asset_stats_pkey on exp_asset_stats  (cost=0.42..2.44 rows=1 width=401) (actual time=0.306..0.306 rows=1
                                  loops=1)
                                        Index Cond: (((asset_code)::text = 'USDC'::text) AND ((asset_issuer)::text =
                                        'GA5ZSEJYB37JRC5AVCIA5MOP4RHTM335X2KGX3IHOJAPP5RE34K4KZVN'::text))
                                        Buffers: shared hit=388
                                  ->  Index Scan using asset_contracts_pkey on asset_contracts  (cost=0.28..2.30 rows=1 width=100) (actual time=2.076..2.076 rows=1
                                  loops=1)
                                        Index Cond: (((asset_code)::text = 'USDC'::text) AND ((asset_issuer)::text =
                                        'GA5ZSEJYB37JRC5AVCIA5MOP4RHTM335X2KGX3IHOJAPP5RE34K4KZVN'::text))
                                        Buffers: shared hit=1 read=2
                                        I/O Timings: shared read=2.058
                            ->  Index Scan using contract_asset_stats_pkey on contract_asset_stats  (cost=0.27..2.29 rows=1 width=87) (actual time=0.016..0.016 rows=1
                            loops=1)
                                  Index Cond: (contract_id = asset_contracts.contract_id)
                                  Buffers: shared hit=7
                ->  Limit  (cost=0.97..7.05 rows=1 width=218) (actual time=0.218..0.219 rows=0 loops=1)
                      Buffers: shared hit=399
                      ->  Nested Loop Anti Join  (cost=0.97..7.05 rows=1 width=218) (actual time=0.218..0.218 rows=0 loops=1)
                            Join Filter: (exp_asset_stats_1.asset_type = asset_contracts_1.asset_type)
                            Buffers: shared hit=399
                            ->  Nested Loop Left Join  (cost=0.55..4.60 rows=1 width=154) (actual time=0.018..0.019 rows=1 loops=1)
                                  Buffers: shared hit=10
                                  ->  Index Scan using asset_contracts_pkey on asset_contracts asset_contracts_1  (cost=0.28..2.30 rows=1 width=100) (actual
                                  time=0.010..0.010 rows=1 loops=1)
                                        Index Cond: (((asset_code)::text = 'USDC'::text) AND ((asset_issuer)::text =
                                        'GA5ZSEJYB37JRC5AVCIA5MOP4RHTM335X2KGX3IHOJAPP5RE34K4KZVN'::text))
                                        Buffers: shared hit=3
                                  ->  Index Scan using contract_asset_stats_pkey on contract_asset_stats contract_asset_stats_1  (cost=0.27..2.29 rows=1 width=87)
                                  (actual time=0.007..0.007 rows=1 loops=1)
                                        Index Cond: (contract_id = asset_contracts_1.contract_id)
                                        Buffers: shared hit=7
                            ->  Index Only Scan using exp_asset_stats_pkey on exp_asset_stats exp_asset_stats_1  (cost=0.42..2.44 rows=1 width=68) (actual
                            time=0.198..0.198 rows=1 loops=1)
                                  Index Cond: ((asset_code = 'USDC'::text) AND (asset_issuer = 'GA5ZSEJYB37JRC5AVCIA5MOP4RHTM335X2KGX3IHOJAPP5RE34K4KZVN'::text))
                                  Heap Fetches: 385
                                  Buffers: shared hit=389
  Planning:
    Buffers: shared hit=48 read=4
    I/O Timings: shared read=2.978
  Planning Time: 3.698 ms
  Execution Time: 2.696 ms

This reduces execution time for that request from 130.288 ms to 2.696 ms while changing the plan from full scans plus a full join to bounded index-backed branches.

Known limitations

• The merged query still performs a final sort, but only over the bounded branch outputs rather than the full joined asset set.
• The contract-only branch still performs an anti-join check against exp_asset_stats; this is much cheaper than the previous full scan, but it is not free.

[Copilot]

Pull request overview

Reworks /assets pagination to merge bounded, index-oriented query branches instead of using a non-sargable full outer join.

Changes:

• Splits asset retrieval into exp_asset_stats and contract-only CTEs.
• Merges, orders, and limits both result sets.
• Documents the performance improvement.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
internal/db2/history/asset_stats.go	Implements bounded CTE pagination.
CHANGELOG.md	Records the /assets performance improvement.

Comments suppressed due to low confidence (1)

internal/db2/history/asset_stats.go:517

• This row-valued ORDER BY likewise cannot be satisfied by asset_contracts' multi-column primary-key ordering, so broad requests sort the full contract-only candidate set before LIMIT. Use separate ordering expressions so this branch can stop after the first page.Limit index-ordered rows.

		OrderBy("(asset_contracts.asset_code, asset_contracts.asset_issuer) " + orderBy).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[hexdigest]

Hi all, I know Horizon is being decommissioned, but we’re currently seeing production latency issues from this class of /assets queries. In the worst cases, latency can spike to around 10 seconds, which is not acceptable for some of our clients. This change is intended to address that specific query shape by restoring index-backed pagination and filtering, and the before/after EXPLAIN ANALYZE results show a substantial improvement for the affected requests.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

[tamirms]

Nice work — the decomposition is the canonical fix for pagination over a FULL OUTER JOIN, and I verified the semantics are preserved: the NULL→zero-JSON swap matches what the Scan implementations already normalize, placeholder rebinding through SelectRaw is correct, and the existing TestGetAssetStatsFiltersAndCursor suite (which includes a contract-only asset) covers both branches, filters, cursors, and both orders.

One ask before merge: a test addition. The checklist marks tests as added but the diff contains none. Existing coverage is genuinely strong, with one gap: no case filters a contract-only asset by code (BTC only appears unfiltered or issuer-filtered). An assetCode: "BTC" case in TestGetAssetStatsFiltersAndCursor would exercise the NOT EXISTS branch under exactly the filter shape this PR optimizes.

[tamirms]

Consider selecting typed NULLs here instead of hand-written zero JSON:

Suggested change

	`'{"authorized":0,"authorized_to_maintain_liabilities":0,"claimable_balances":0,"liquidity_pools":0,"unauthorized":0}'::jsonb as accounts`,
	`'{"authorized":"0","authorized_to_maintain_liabilities":"0","claimable_balances":"0","liquidity_pools":"0","unauthorized":"0"}'::jsonb as balances`,
	`NULL::jsonb as accounts`,
	`NULL::jsonb as balances`,

The old FULL OUTER JOIN produced NULL accounts/balances for contract-only rows, and ExpAssetStatAccounts.Scan / ExpAssetStatBalances.Scan already normalize NULL to exactly these zero values (0 ints, "0" strings). Relying on that gives byte-for-byte parity with the previous behavior and removes two hard-coded copies of the struct field sets, which would silently drift if a field is ever added to ExpAssetStatAccounts/ExpAssetStatBalances.

[tamirms]

This block is dead code — results starts nil and sqlx only appends per scanned row, so zero rows already yields nil (clearSliceIfPossible preserves nil-ness too). That's why the existing expected: nil cases in TestGetAssetStatsFiltersAndCursor passed before this change, on the same SelectRaw path.

Suggested change

	if len(results) == 0 {
	return nil, nil
	}