Repository navigation
Backend portability: Band-value scaling is unspecified in openEO, and not consistently surfaced via STAC #589
Description
Gap 2 — Band-value scaling is unspecified in openEO, and not consistently surfaced via STAC
The same Sentinel-2 L2A collection is served by different backends with different numeric ranges:
- eopf_explorer: reflectance 0–1 (float)
- Copernicus Data Space: integer digital numbers 0–10000 (Sentinel-2 L2A native)
Any UDP that computes a threshold-based index (NDCI > 0.08, chl_a < 0.5 μg/L, FAI thresholds from the CyanoLakes paper, etc.) has to normalize first — but the scaling factor (1.0 vs 10000.0) is not published anywhere openEO clients read. It's not in GET /collections/{id}, it's not in a capability field, it's not in the dimension metadata.
STAC 1.1 already defines raster:scale / raster:offset per band (radiantearth/stac-spec#1213), and real-world Sentinel-2 STAC items use e.g. scale: 0.0001, offset: -0.1. openEO backends that expose STAC collection metadata could populate these, and openEO processes could be specified to transform accordingly, but none of this is currently required or enforced.
Suggested direction. Either:
- Require backends to either normalize to 0–1 reflectance server-side for well-known collections, or
- Require backends to publish
bands[].scale/bands[].offsetin the STAC collection metadata returned byGET /collections/{id}, and recommend that clients auto-apply them before user code runs, or - Add an openEO process (or
load_collectionflag) that forces the backend to return normalized values regardless of source format.
The first option is the cleanest for algorithm authors but may be rejected on performance grounds. The second is the most STAC-native and probably the smallest spec change.
Activity
[-]Backend portability: Gap 2 — Band-value scaling is unspecified in openEO, and not consistently surfaced via STAC[/-][+]Backend portability: Band-value scaling is unspecified in openEO, and not consistently surfaced via STAC[/+]m-mohr commentedon Apr 29, 2026
I'm starting with some context again: When we defined the specification, we did only have
load_collectionand collection metadata at the/collectionsendpoints. So the entry point was controlled and there was noload_stac. The intention was as follows:I think sometimes the backends don't really do a good job in describing the metadata though. Thinking about the Sentinel-2 data as UINT16 which require a scale/offset to get to the actual measurements: This needs to be described properly. If the datacube is containing these UINT16 values, the collection metadata should list scale/offset. If you get the actual floating point values after applying scale/offset, no scale/offset is listed. The metadata (e.g. description) should described whether the values are "digital numbers" / scaled or physical values. It could also add a unit property (although not quite applicable to S2, unless you make something up).
This "contract" probably should better be described in the specification (either API, but maybe even more in load_collection).
The issue is now that
load_stacdoesn't say anything at all about it. Here, it is a bit difficult because different file formats behave differently. Usually, I'd say we let users handle applying scale/offset andload_stacjust delivers what's in there. netCDF and related formats apply scale/offset automatically though. The question here is, should STAC list scale/offset when it's applied automatically per netCDF spec?So my thinking here is: Clarify this in the processes and API specification for collections. Whenever a scale/offset is present in metadata, users have to apply it manually. If no scale/offset is present, users have to do nothing. That would be consistent across all processes, I think. I think that's your second bullet point.
We could simplify the "normalization" process probably a bit, either by client tooling generating those constructs or a process addition/change.
soxofaan commentedon Apr 29, 2026
I think that's a no-go because that's a huge, non-manageable breaking change.
As mentioned, in the geopyspark implementation we originally do not auto-scale values for performance reasons. Not only to avoid the scaling operation itself, but also to preserve the original compact data type (e.g. int8, int16) as long as possible instead of having to jump straight off the bat into some arbitrary, more memory-intensive float/double representation.
emmanuelmathot commentedon Apr 29, 2026
Couldn't it be something the user could define to translate or not. Again, my proposal would be to use some backend parameterization (#590) to have the possibility to apply a scale offset operation (via a defined process) in the graph?
soxofaan commentedon Apr 29, 2026
I think there is a bit of conflict arising here when there is coupling between
On the one hand we want to align the openEO collection metadata as much with standard STAC metadata. For example: it should be possible to use the openEO collection metadata with non-openEO-aware STAC tooling and walk the STAC links to get STAC items and assets. But that probably means you want to have the correct scale/offset metadata so that these STAC tools properly handle the linked STAC assets.
But with the rules quoted above, that means we're locked in in always expecting the openEO user/client to apply the scaling,
and it eliminates the opportunity for doing things more user friendly (simpler process graphs) or more efficiently at the backend-side (e.g. cache pre-scaled data)
I think we need a more explicit and openEO-specific opt-in feature (so outside of the standard STAC metadata) to signal that auto-scaling is desired or happening. For example as suggested in
Personally, it feels more natural to make it a
load_collection/load_stacflag, so that the design properly reflects that this feature is tightly coupled to the data loading aspect.m-mohr commentedon Apr 29, 2026
I don't understand the conflict/issue you are describing. There's no coupling, openEO just does what the STAC metadata express explicitly. The only case that is not "easy" is when you want autoscaling in openEO, which it seems you do not want ("that's a no-go").
Such a flag in those processes would only work if scale/offset is defined in the metadata. When it's not, you can't scale. That doesn't feel ideal to me. So a user still needs to look up the metadata to know what he will get.
soxofaan commentedon Apr 29, 2026
It's a no-go to make this the default behavior on existing collections at the moment.
It's however a feature we and our users want in some way, but it should be an explicit opt-in thing for existing collections, and possibly enabled by default for newly added collections.
There is no coupling or conflict indeed in the absence of backend-side auto-scaling.
But with backend-side autoscaling (which is the most user friendly option here I think), there is a conflict about the STAC metadata of the openEO collections. (Note that I'm reasoning here from a point where openEO functionality is being added on top of existing STAC resources, with existing scale/offset metadata.)
So that's the conflict: you manipulate STAC metadata of existing STAC items/assets to express openEO-specific behavior (which is not directly linked from that STAC metada), and that possibly breaks the proper consumption of that STAC metadata outside of openEO.
soxofaan commentedon Apr 29, 2026
You make this sound like a bug, but I think this is actually a feature:
it allows the backend to do more advanced normalization that can not be expressed with a single scale/offset pair: e.g. when different items or assets have different scale/offset values, or the normalization is non-linear (expressed with custom metadata)