cloudytile

Is this satellite tile clear enough to read the lake?
A small CNN that answers yes or no, so a drainage classifier downstream never has to guess.

Lecture · Status & decisions · Evaluation protocol

---

Supraglacial lakes on the Greenland Ice Sheet are read from Sentinel-2 time series. Cloud ruins frames, and Sentinel-2's own cloud masks are unreliable over bright ice — snow and cloud look alike to them. cloudytile learns the distinction directly from labeled imagery and writes a per-timestep cloudy_seq flag into each lake's NetCDF, which the lake-vision drainage classifier consumes.

What the label means

The question is usability, not cloud presence.

useful clear	useful thin cloud — lake still reads	not useful obscured	not useful uniform deck

The second tile is visibly cloudy and still labeled useful — the lake reads through the thin layer, so the frame carries information. That is the whole difficulty of the task, and the reason a cloud mask is the wrong tool for it.

No-data frames are not part of the negative class. Frames that are ≥50% no-data are dropped by rule before a human ever sees them — they are decidable without a model, so labeling them would spend effort on a solved subproblem. The result: 97.8% of "not useful" frames contain under 1% no-data (median exactly 0). The negative class is cloud, not blank canvas, and the classifier is never rewarded for the easy discrimination.

The dataset

Frames	10,000 — 200 lakes × 25 timesteps × 2 melt seasons (2018, 2019)
Lakes	400, all in the CW (central-west) IMBIE basin
Balance	68.6% useful / 31.4% not useful (≈2.2:1, mild)
Split	280 train / 40 val / 80 test lakes — frozen, committed, never re-drawn
Bands	all six SDR bands stored per tile; the band subset is a training-time choice
Source	the ESSD SDR deposit (10.25740/sf350xp4038)

Timesteps are sampled at random across the melt season rather than consecutively, because adjacent frames of one lake are near-duplicates. Splits are grouped by lake, never by tile — an earlier tile-level split put every test lake in train as well, so nothing was actually held out.

Baselines worth beating

Fit on the 320 development lakes, scored on the 80 frozen test lakes:

baseline	accuracy	what it knows
majority class	67.5%	nothing
JPEG file-size threshold	82.2%	one byte count per image

JPEG size is an entropy meter: clear ice is detailed and compresses poorly, cloud is smooth and compresses well. Quote model accuracy against 82.2%, not against chance — the margin above a stat() call is the only part the network earns. And because accuracy sits so near the majority rate, headline numbers should be AUC plus precision/recall at the chosen operating point.

The model

CloudyTileCNN — six blocks of conv 3×3 → BatchNorm → ReLU → maxpool, widths [16, 32, 64, 64, 96, 128], then a head that collapses the feature block to a single logit.

Depth is what buys receptive field: 22 px at three blocks versus 190 px at six, about 37% of a 512 px tile. It is nearly free — blocks 4–6 run on 32×32 grids and smaller, costing 1.3× the MACs of a three-block stack, where doubling the first block would cost 4.6×. Grow channels late.

Three interchangeable heads all emit exactly 128 values into the same 8-unit MLP, so they differ in size by under 0.5% and any difference between them is inductive bias rather than capacity:

head	reduce	pool	flattened	params
gap	—	1×1	128	228,609
mixed	8 ch	4×4	128	229,657
spatial	2 ch	8×8	128	228,871

Every head is resolution-independent (AdaptiveAvgPool2d), so the model is byte-for-byte the same at any input size. Training and inference both run at 512 px — a model selected at one resolution and deployed at another is not the model that was selected.

Selected configuration: rgb+swir16 bands, six-block stack, spatial head, 80 epochs with cosine annealing, AdamW at 1e-3. SWIR is the finding — it beat RGB by +0.0058 AUC winning 20 of 20 folds, while NIR was null. See docs/STATUS.md for the evidence behind each choice and docs/CROSSVAL.md for how it was measured.

Quickstart

pip install -r requirements.txt
python3 -m pytest cloudytile/tests/          # 93 tests

The pipeline, end to end. Steps 1–4 are one-time; you almost certainly want to start at step 5.

1–2 · Build a labeled set (already done: labels/labels.csv)

# sample lake-diverse frames, dropping >=50% no-data before a human sees them
sbatch slurm/extract_label_frames.sh

# label them: one image, two keys, atomic CSV writes, automatic resume
python3 engine/label_gui.py --image_dir data/label_frames_2019
#   <- = 0 not useful  |  -> = 1 useful  |  backspace = back  |  space = skip  |  h = help

3–4 · Freeze the split, extract training tiles (already done)

# ONCE, before any model is fit. Writes lake-ID lists; commit them.
python3 engine/make_splits.py --labels_csv labels/labels.csv --out_dir splits/cloudytile_v1

# JPGs are for human eyes only; training reads six-band NetCDF
sbatch slurm/extract_training_nc.sh
sbatch slurm/compute_band_stats.sh

5 · Select a configuration — lake-grouped cross-validation over a small grid, folded over the 320 development lakes only, so selection never sees the 80 test lakes:

sbatch slurm/run_cv_bandhead.sh                                    # 4 band sets x 4 heads
sbatch slurm/run_cv_epochs.sh                                      # 3 heads x 5 horizons

python3 engine/run_cv_grid.py --out_dir <results> --summarize      # ranked table
python3 engine/upload_wandb.py --out_dir <results> --sync --table   # push to wandb

6 · Train the final model — the only run that touches the test lakes:

# set WINNER_BANDS and WINNER_HEAD at the top first; it refuses to run while they say TBD
sbatch slurm/run_final_model.sh

7 · Inference over the lake archive:

python3 engine/run_inference.py --model best_model.pth --input /path/to/nc_dir

Checkpoints are self-describing — the architecture is saved alongside the weights, so inference rebuilds the exact trained model and needs no architecture flags. Checkpoints predating September 2026 are bare state_dicts and still load, but must be told: --channels, --fc_layers, --head, --head_reduce, --nc_channels.

Layout

cloudytile/           importable package
  model.py              CloudyTileCNN — conv stack + gap/pool<N>/flatten heads
  data.py               datasets (NetCDF primary, JPG legacy)
  splits.py             lake-grouped splitting, frozen splits, file-size baseline
  training.py           train/eval loops, metrics, threshold selection
  inference.py          cloudy_seq prediction on lake NetCDF files
  labels.py             label CSV store with atomic writes
  preprocessing.py      NetCDF -> JPG rendering
  tests/                pytest suite

engine/               runnable scripts, flat and pure Python
  extract_label_frames.py   SDR .nc -> JPGs for labeling
  label_gui.py              browser labeling GUI (+ index.html)
  extract_training_nc.py    labels.csv -> per-tile six-band .nc
  compute_band_stats.py     per-band mean/std for normalization
  run_training.py           single training run
  run_cv_grid.py            lake-grouped CV over a config grid (+ --summarize)
  run_baselines.py          majority-class and file-size baselines
  upload_wandb.py           push results and offline runs to wandb
  run_inference.py          apply a model to lake NetCDF files

slurm/                Sherlock wrappers — paths, modules, and the reasoning for each choice
labels/               labels.csv (current) and labels_deprecated.csv (never merge)
splits/cloudytile_v1/ frozen lake-ID split + baselines.json
docs/                 lecture, STATUS.md, CROSSVAL.md, claudiary/

Conventions

• Frame key {lake_id}_t{timestep:03d} — .jpg for labeling, .nc for training.
• Label polarity 0 = not useful, 1 = useful. This is load-bearing downstream; do not flip it.
• Normalization per-band mean/std, then NaN → exactly 0.0. In that order — the reverse maps no-data to roughly −4σ, which is a strong bogus signal rather than a neutral one.
• Never merge labels_deprecated.csv into labels.csv; it labels a different frame population over only 60 lakes.

Where things live on OAK

resource	path under /oak/stanford/groups/cyaolai/JoshRines
SDR imagery (source of everything)	data/essd_sdr/data/CW{2018,2019}/
Labeling frames (JPG)	data/cloudytile/label_frames_{2018,2019}/
Training tiles (.nc)	data/cloudytile/training_nc_10k/
Band statistics	data/cloudytile/band_stats_10k.json
Models, logs, CV results	sherlock/sherlock_cloudytile/

Locally the same tree mounts at /Volumes/groups/cyaolai/JoshRines.

Contributing

Issues and PRs welcome at jharlanr/cloudy-tile.