Task-Driven SR Ensemble

PyTorch/srforge code and pretrained models accompanying Task-Driven Super-Resolution Ensemble for Multi-Image License Plate Recognition (ICIP 2026), by Jakub Sadel, Tomasz Tarasiewicz and Michal Kawulok.

The repository provides simplified RAMS and SBFBurst training and the final recognition ensemble: raw LR frames, RAMS, SBFBurst and MagNAt, followed by FastPlate OCR fusion. MagNAt is included for inference with pretrained weights.

Installation

The reference environment uses Linux, Python 3.12 and PyTorch 2.8 with CUDA 12.6. The full pipeline requires an NVIDIA GPU. Run the commands below from the repository root.

git clone https://github.com/jakubsadel/task-driven-sr-ensemble.git
cd task-driven-sr-ensemble
python3.12 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 \
  --index-url https://download.pytorch.org/whl/cu126
pip install -e .

For the full ensemble, install the matching graph extensions used by MagNAt:

pip install pyg-lib==0.5.0 torch-scatter==2.1.2 torch-sparse==0.6.18 \
  torch-cluster==1.6.3 torch-spline-conv==1.2.2 \
  -f https://data.pyg.org/whl/torch-2.8.0+cu126.html
pip install -e ".[ensemble]"

PWC uses a CuPy CUDA kernel. A compatible CUDA installation must be available for kernel compilation. The core install includes srforge; a separate checkout of the framework is unnecessary.

Checkpoints

Download the final SR models, PWC weights and four FastPlate models/configurations:

tdsr-checkpoints --download

For OCR-aware training, also download PARSeq:

tdsr-checkpoints --download --set training

checkpoints/manifest.yaml lists exact filenames, roles, download URLs and SHA-256 checksums. Files are placed in checkpoints/ and excluded from Git. The inference files occupy approximately 247 MiB; PARSeq adds 91 MiB.

Existing files in the same directory layout can be imported and verified offline:

tdsr-checkpoints --source-dir /path/to/models --set all
tdsr-checkpoints --verify --set all

Data

Obtain the datasets from their providers:

• RLPR: multi-frame plate restoration data for image-only warm-up.
• UFPR-SR-Plates: real LR/HR plate crops for task-driven fine-tuning. Follow the authors' data-access instructions.
• XLPSR: use the cropped license-plate sequences distributed by the challenge organizers for inference/evaluation.

Fill in your own dataset paths in configs/paths.yaml. You can keep personal settings in an ignored local copy:

cp configs/paths.yaml configs/paths.local.yaml
# Edit configs/paths.local.yaml, then pass --paths configs/paths.local.yaml to training.

Dataset files are not downloaded or included by this repository. Keep training and validation splits separate.

The training adapters expect these layouts:

RLPR/
└── sample_*/
    ├── Pseudo_GT.png
    └── Plate_crop/06.png ... 26.png

UFPR-SR-Plates/
├── split.txt
├── 1920_1080/{brazilian,mercosur}/plate-*/
│   ├── lr-001.png ... lr-005.png
│   └── hr-001.png ... hr-005.png
└── 1280_0960/{brazilian,mercosur}/plate-*/
    └── ...

UFPR's split records use relative/path;PLATETEXT;split. The simplified recipe uses the center HR frame. It does not require the private OCR-based HR-selection manifests used during the original experiments.

For RLPR, provide separate training and validation text files with one sample directory name per line (for example, sample_00001). The adapter reads frames 12–20 and downsamples the ×4 pseudo-GT before fitting the ×2 target canvas. Set both split-file paths in your local configuration.

Training

Train the image reconstruction stage, then fine-tune with OCR-aware supervision:

tdsr-train --config configs/training/rams_image.yaml
tdsr-train --config configs/training/rams_task.yaml

tdsr-train --config configs/training/sbfburst_image.yaml
tdsr-train --config configs/training/sbfburst_task.yaml

Training uses srforge's data routing, epoch runners and trainer. The image stage uses masked L1 and Sobel losses. The task stage adds SR–LR consistency and cross-entropy through a frozen PARSeq recognizer. Gradients pass through PARSeq to the SR model. PARSeq is not part of final ensemble inference.

Each run saves config_resolved.yaml, metrics.jsonl, checkpoint_best.pth, checkpoint_last.pth and the corresponding raw SR weights (rams_best.pth, rams_last.pth, or sbfburst_*.pth) under outputs/training/<recipe>/. The task-stage defaults load the best weights from the matching image stage. Override warm_start in your paths file to use another checkpoint.

Resume an interrupted run with its checkpoint and accompanying SR weight file in the same directory:

tdsr-train --config configs/training/rams_image.yaml --paths configs/paths.local.yaml \
  --resume outputs/training/rams_image/checkpoint_last.pth training.epochs=120

training.epochs is the total target epoch count, including completed epochs. Other settings can also be changed with key=value, for example training.batch_size=4.

Your paths and W&B account

Local logging is the default; W&B is optional. To use it, set your own values in configs/paths.local.yaml:

output_root: outputs/training
tracking:
  enabled: true
  project: YOUR_WANDB_PROJECT
  entity: null  # Your username/team, or null to use your account's default.
  name: null   # Optional run name.
  group: null  # Optional experiment group.
  mode: online # Use offline to save a run for later synchronization.

Authenticate with wandb login or set WANDB_API_KEY in your shell environment. Do not put API keys in YAML files. Start training with your local settings:

tdsr-train --config configs/training/rams_image.yaml --paths configs/paths.local.yaml

Set tracking.enabled: false to train without W&B. Dataset roots, output paths, checkpoint paths and tracking settings belong to the person running the code; the published configuration contains no account credentials.

These configurations are a simplified training recipe. The released paper checkpoints additionally used mixed-domain synthetic data and model-specific training choices; running the simplified recipe is not expected to recreate those checkpoints or their reported scores exactly.

Inference

Prepare one directory per sequence, with zero-padded frame names in temporal order:

input/
├── all_sequences.csv
├── seq_000/img_00.png ... img_09.png
└── seq_001/img_00.png ... img_09.png

all_sequences.csv contains a single sequence_id column:

sequence_id
seq_000
seq_001

Run the full ensemble, including MagNAt:

tdsr-infer --input-dir /path/to/input --output-dir outputs/predictions \
  --models-dir checkpoints --device cuda:0 --ocr-provider-mode cuda

To select another physical GPU, prefix the command with CUDA_VISIBLE_DEVICES=1 (or your GPU index) and keep --device cuda:0. The single-GPU pipeline uses logical GPU 0 consistently for PyTorch, CuPy and ONNX Runtime.

To evaluate an ensemble with your own trained SR models, prepare a separate --models-dir using the manifest's directory layout. Place rams_best.pth at rams/RAMS_MIXED_CHARCONS_best.pth and sbfburst_best.pth at sbfburst/SBFBurstRGBx2_sbf9_mix_best.pth, alongside the pretrained MagNAt, PWC and FastPlate files. The release checksums identify the original paper weights; custom training outputs will have different hashes.

Each output sequence directory contains sr.png and text.txt. The text uses uppercase letters, digits and _ for uncertain characters. The SR image is selected by agreement with the fused text.

The pipeline produces two 9-frame RAMS reconstructions, two 9-frame SBFBurst reconstructions and one MagNAt reconstruction. Four FastPlate recognizers process the SR images and raw frames. The seven most reliable raw frames enter the final fusion together with all five SR outputs:

Source	Weight
Each selected LR frame	1.00
RAMS s0 / s1	1.35 / 0.55
SBFBurst s0 / s1	0.55 / 0.30
MagNAt	1.30

Inference requires the complete model set and does not use ground-truth text. Missing model files or unavailable backends cause an error.

Evaluation

Given labels in a CSV with columns sequence_id,text:

tdsr-evaluate --predictions-dir outputs/predictions \
  --ground-truth /path/to/labels.csv --output-dir outputs/evaluation

Use --text-column license_plate if that is the label-column name in your file. Evaluation writes per-sequence predictions.csv and aggregated metrics.json. It reports mean wECR (+2 correct, 0 missing, −1 wrong), exact-match accuracy and positional character accuracy. Missing predictions cause an error rather than silently reducing the evaluation set.

The paper's final development ensemble reports 10.72 wECR, 38.46% exact match and 81.56% character accuracy on 39 sequences. Development data also supported adaptation/calibration. The public result of 7.66 wECR and 69.25% character accuracy belongs to the earlier RAMS + SBFBurst whole-text fusion submission.

Citation and acknowledgments

@inproceedings{sadel2026taskdriven,
  title={Task-Driven Super-Resolution Ensemble for Multi-Image License Plate Recognition},
  author={Sadel, Jakub and Tarasiewicz, Tomasz and Kawulok, Michal},
  booktitle={IEEE International Conference on Image Processing (ICIP)},
  year={2026}
}

See THIRD_PARTY_NOTICES.md and LICENSE for the model implementations, pretrained components and their respective terms.