Cluster box features to help choose images to annotate

[bw4sz]

As part of an active learning workflow, we'd want a way to run a detector over the boxes, and cluster box features using something like Dino v3 and then have the workflow select either 1) exemplar images from k clusters, 2) images that are similar to a user supplied image. This kind of image query helps with rapid annotation.

[musaqlain]

This is a great feature idea, I have looking in deepforest to work on architectural work from the start. I would love to contribute on it, once the issue is fully documented... also I have been exploring CropModel pipeline and DINOV2 feature extraction, happy to help design and implement this

[vickysharma-prog]

Interesting idea! A couple of thoughts while this gets documented:

The ongoing work on CropModel (#1313, #1347) is already making the crop pipeline more flexible with multi-architecture support and custom normalization this could naturally extend to plugging in a DINO-family backbone for per-box feature extraction rather than classification.
One design consideration: since the end goal is selecting images to annotate, the pipeline would need a strategy to go from box-level feature vectors to image-level selection whether that's picking images that maximize cluster diversity, or aggregating box features per image for direct image-level clustering.

Happy to wait for full documentation before diving deeper!

[InfamousFreak]

Hi @bw4sz ! This active learning workflow sounds incredibly useful for rapid annotation. I’ve been diving into the DeepForest codebase in preparation for my GSoC application and would love to take a swing at this.

If no one is actively coding this yet, could I be assigned? I'd be happy to spin up an initial draft PR setting up the feature extraction and a k-clusters selection script so we have some concrete code to discuss!

[vickysharma-prog]

Welcome to DeepForest @InfamousFreak! 👋

Great to see your interest in DeepForest

Just a heads up this issue is still in the ideas/documentation phase (notice the "To be documented" label). Ben hasn't fully scoped it yet, so jumping into a PR right now might lead to rework. If you're looking to get started with contributions, filtering by the "good first issue" or "bug" labels is a great way to build familiarity with the codebase while this one gets fleshed out!