Skip to content

feat(evaluation): add ROC AUC metric - #411

Merged
tachyonicClock merged 2 commits into
adaptive-machine-learning:mainfrom
tachyonicClock:AUC
Sep 7, 2026
Merged

tachyonicClock merged 2 commits into
adaptive-machine-learning:mainfrom
tachyonicClock:AUC

Conversation

@tachyonicClock

@tachyonicClock tachyonicClock commented Sep 7, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Bump the bundled MOA jar (invoke.yml) to the version that adds a prequential ROC AUC to BasicClassificationPerformanceEvaluator (F1 fix + log loss + evaluate prequential delayed regression Waikato/moa#332).
  • Enable the new rocAUC option by default in ClassificationEvaluator, and expose it as roc_auc() on both ClassificationEvaluator and ClassificationWindowedEvaluator, so it's accessible from PrequentialResults the same way as accuracy, kappa, f1_score, etc.
  • Add the roc_auc -> "ROC AUC (cumulative)" name mapping used to translate between CapyMOA and MOA metric names.

Note: MOA computes ROC AUC prequentially for binary classification only, reporting NaN for multi-class problems (see adaptive-machine-learning/backlog#23, which had been blocked on an alternative approach — we ended up simply adding ROC AUC to the evaluator instead).

Test plan

  • pytest tests/test_evaluation.py passes (47 passed), including test_evaluation_api which generically checks every metric returned by metrics_header() is accessible through PrequentialResults.
  • Manually verified results.cumulative.roc_auc(), results.roc_auc() (delegated), and results.windowed.roc_auc() all return sensible values on ElectricityTiny with HoeffdingTree.

Assisted-by: claude-code:claude-sonnet-5

MOA's BasicClassificationPerformanceEvaluator now supports a
prequential ROC AUC (Waikato/moa#332). Enable it by default and
expose it as `roc_auc()` on ClassificationEvaluator and
ClassificationWindowedEvaluator, so it's accessible through
PrequentialResults like other metrics.

Closes adaptive-machine-learning/backlog#23

Assisted-by: claude-code:claude-sonnet-5
@tachyonicClock
tachyonicClock merged commit 3efc9a6 into adaptive-machine-learning:main Sep 7, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant