Skip to content

Compute average_batch_r2 per cell type - #127

Merged
LuLeom merged 4 commits into
mainfrom
fix-average-batch-r2-celltype
Jul 30, 2026
Merged

Compute average_batch_r2 per cell type#127
LuLeom merged 4 commits into
mainfrom
fix-average-batch-r2-celltype

Conversation

@rcannood

Copy link
Copy Markdown
Member

Describe your changes

average_batch_r2 reports a per-cell-type score (average_batch_r2_ct), but the R² it computes is the donor-level one, pooled over all cell types.

In src/metrics/average_batch_r2/script.py the loop builds s1_view_ct / s2_view_ct, uses them for the 20-cell guard, and then hands batch_r2() the donor-level s1_view / s2_view:

for ct in ct_list:
    s1_view_ct = s1_view[s1_view.obs["cell_type"] == ct]
    s2_view_ct = s2_view[s2_view.obs["cell_type"] == ct]
    if s1_view_ct.shape[0] < 20 or s2_view_ct.shape[0] < 20:
        continue

    r2_list, marker_list = batch_r2(s1_view, s2_view)   # <-- not the _ct views

Two consequences:

  • The reported value is the global R², not the cell-type-aware one -- so it can't show the "good on global but not on cell type" case that the metric description is built around, and the ∑_{j=1}^{C} cell type sum in the description isn't what's computed.
  • Because the same donor-level value is appended once per surviving cell type, donors with more cell types above the 20-cell threshold get more weight in the final np.mean().

This PR passes the cell type subsets, and corrects the "less than 10 cells" comment next to the < 20 guard.

Note that this changes the value of average_batch_r2_ct for every method, so results from earlier runs are not comparable.

Found while reviewing the task ahead of the next full benchmark run -- see also #TODO_ISSUE_A and the sibling PRs.

Checklist before requesting a review

  • I have performed a self-review of my code

  • Check the correct box. Does this PR contain:

    • Breaking changes
    • New functionality
    • Major changes
    • Minor changes
    • Bug fixes
  • Proposed changes are described in the CHANGELOG.md

  • CI Tests succeed and look good!

* Pass the cell type subsets to batch_r2() instead of the donor-level views
* Correct the cell count in the skip comment
@rcannood
rcannood requested review from LuLeom and ghar1821 July 28, 2026 10:30
@LuLeom

LuLeom commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

LGTM

@LuLeom
LuLeom merged commit ae68ce3 into main Jul 30, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants