Skip to content

feat: wire scfit dataloader - #160

Open
selmanozleyen wants to merge 7 commits into
refactor/splitters-and-device-fixfrom
feat/scfit-dataloader
Open

feat: wire scfit dataloader#160
selmanozleyen wants to merge 7 commits into
refactor/splitters-and-device-fixfrom
feat/scfit-dataloader

Conversation

@selmanozleyen

Copy link
Copy Markdown
Collaborator

This pull request refactors and simplifies the data handling utilities in the core methods, improves the interface for slicing and matching batch data, and updates dependencies and API usage for improved compatibility and clarity. The most important changes are summarized below.

Data handling and API refactoring

  • Major simplification of src/sckitflow/core/_data_utils.py: removes several now-redundant functions (extract_step_data, extract_coupling_data, extract_distribution_data, etc.), and introduces a new, general-purpose subscript_step_data function for row-slicing both source and target fields in a StepData batch. This centralizes and clarifies batch manipulation logic. [1] [2] [3] [4]
  • Refactors all internal usage to rely on subscript_step_data, removing the previous _safe_subscript_obj and custom slicing logic from BaseMethod. This makes the codebase more maintainable and less error-prone. [1] [2] [3]

Documentation and interface improvements

  • Updates docstrings and comments throughout the codebase to reflect the new data loader responsibilities and clarify that StepData is now fully assembled by the data loaders, not by methods themselves. [1] [2] [3]

Dependency and configuration updates

  • Updates minimum versions of key dependencies in pyproject.toml (anndata>=0.13, scfit>=0.0.7) to support new features and GPU streaming, and adds optional GPU and torch dependencies for improved flexibility. [1] [2]

Notebooks and example usage

  • Updates example notebooks to use the new APIs: replaces optim_kwargs with optim_config, and updates batch size and prediction parameterization for consistency with the refactored interfaces. [1] [2] [3] [4]

Method registry and exports

  • Exposes OptimConfig and OptimizationManager in the core methods module for easier access and consistency with the rest of the API.

These changes collectively improve the clarity, maintainability, and extensibility of the codebase, especially around how batches of data are handled and manipulated during training and inference.

@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@selmanozleyen selmanozleyen changed the title Feat/scfit dataloader feat: wire scfit dataloader Aug 7, 2026
@selmanozleyen selmanozleyen self-assigned this Aug 7, 2026
@selmanozleyen
selmanozleyen force-pushed the feat/scfit-dataloader branch from 417ddc0 to 0523caa Compare August 7, 2026 13:28
@selmanozleyen
selmanozleyen force-pushed the feat/scfit-dataloader branch from 60ba0d6 to 942485c Compare August 7, 2026 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant