Skip to content

Unified aero recipe: optional shared reader cache - #2016

Draft
coreyjadams wants to merge 2 commits into
io-03-cache-readersfrom
io-04-recipe
Draft

coreyjadams wants to merge 2 commits into
io-03-cache-readersfrom
io-04-recipe

Conversation

@coreyjadams

@coreyjadams coreyjadams commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

PhysicsNeMo Pull Request

Description

Wires the optional reader cache into the unified external aerodynamics recipe. A dataloader.cache config block (off by default; the example points disk_dir under $TMPDIR), one shared DatasetCache built in build_dataloaders and passed to every dataset, MeshReaderWithGlobalData routing its external global-data load through it, and per-epoch cache.stats() in the existing benchmark_io mode. Two recipe tests.

This is the branch the I/O benchmark matrix runs on (main / pread / cache / both, benchmark_io=true, epoch-2 throughput).

Checklist

Stack

# PR Base
1 #1939 preadv in the mesh readers (Max Khitrov, from fork) main
2 #2014 DatasetCache core io-01-pread
3 #2015 reader integrations io-02-cache-core
4 #2016 unified aero recipe wiring io-03-cache-readers
5 #2017 docs io-04-recipe

Each PR shows only its own commits. Merge bottom-up; after each merge, retarget the next PR to main. Supersedes #1872.


🤖 Generated with Claude Code

coreyjadams and others added 2 commits September 23, 2026 13:02
Signed-off-by: Corey Adams <6619961+coreyjadams@users.noreply.github.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Corey Adams <6619961+coreyjadams@users.noreply.github.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 23, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions

Copy link
Copy Markdown
Contributor

CODEOWNERS review map

Current for commit 5fb0c7211847. An approval covers every file listed for that owner; one owner is sufficient for shared files.

⏳ @coreyjadams — 6 file(s)
  • examples/cfd/external_aerodynamics/unified_external_aero_recipe/conf/base.yaml
  • examples/cfd/external_aerodynamics/unified_external_aero_recipe/src/datasets.py
  • examples/cfd/external_aerodynamics/unified_external_aero_recipe/src/merge_global_data.py
  • examples/cfd/external_aerodynamics/unified_external_aero_recipe/src/train.py
  • examples/cfd/external_aerodynamics/unified_external_aero_recipe/tests/test_merge_global_data_cache.py
  • examples/cfd/external_aerodynamics/unified_external_aero_recipe/tests/test_reader_cache_wiring.py
⏳ @peterdsharpe — 6 file(s)
  • examples/cfd/external_aerodynamics/unified_external_aero_recipe/conf/base.yaml
  • examples/cfd/external_aerodynamics/unified_external_aero_recipe/src/datasets.py
  • examples/cfd/external_aerodynamics/unified_external_aero_recipe/src/merge_global_data.py
  • examples/cfd/external_aerodynamics/unified_external_aero_recipe/src/train.py
  • examples/cfd/external_aerodynamics/unified_external_aero_recipe/tests/test_merge_global_data_cache.py
  • examples/cfd/external_aerodynamics/unified_external_aero_recipe/tests/test_reader_cache_wiring.py

Comment /codeowners-info to refresh.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant