Indonesia-GeoMAD

Configuration files and workflows for generating GeoMAD (Geometric Median Absolute Deviation) statistics over Indonesia using odc-stats and Argo Workflows.

Overview

This repository contains:

• ODC-stats configuration files for GeoMAD statistics processing
• Argo Workflow definitions for orchestrating large-scale processing
• Testing scripts for validation and quality control
• Deployment templates and utilities

GeoMAD Statistics

This repository supports generation of multiple spectral statistics:

• Geomedian - Geometric median composite (multi-dimensional median)
• MAD - Median Absolute Deviation (measure of spectral variability)
• SMAD - Standardized MAD (normalized deviation)
• EMAD - Euclidean MAD (distance-based deviation)
• BCMAD - Bray-Curtis MAD (compositional deviation)
• Count - Valid observation counts
• Percentiles - Spectral percentile statistics

Manage Development Environment

ODC

To build Docker images and run containerised PostgreSQL/PostGIS and ODC locally, follow this quickstart and then in that env run make index-s2-gm-annual so the GeoMAD product has datasets. Export these so we can then connect to the local ODC-enabled database:

# Use the secrets you set up here: https://github.com/piksel-ina/piksel-core/blob/main/.env.example
export ODC_DEFAULT_DB_USERNAME=X
export ODC_DEFAULT_DB_DATABASE=X
export ODC_DEFAULT_DB_HOSTNAME=X
export ODC_DEFAULT_DB_PASSWORD=X
export ODC_DEFAULT_DB_PORT=X
export ODC_DEFAULT_INDEX_DRIVER=postgis
export AWS_PROFILE=X

Now you are ready to run commands such as to create the low resolution mosaic COGs and STAC.

Environment Setup with UV

1. Install System Dependencies

   Install PostgreSQL development libraries required by some Python packages:

   sudo apt-get update
   sudo apt-get install libpq-dev python3-dev

2. Install Project Dependencies (Notebook Environment)

   Create virtual environment and install all packages from pyproject.toml:

   uv sync --extra notebook

   Warning (GDAL version pinning):

   Our dependency lock pins a specific GDAL version to match the production/testing environment.
   If you’re running on a machine with a different system GDAL version, installation may fail or behave unexpectedly.
   In that case, adjust the GDAL pin to match your system (or remove the version constraint).

3. Activate Virtual Environment

   Activate the environment to access installed packages:

   source .venv/bin/activate

4. Register Jupyter Kernel

   Make the environment available in Jupyter notebooks:

   python -m ipykernel install --user --name indonesia-geomad --display-name "Indonesia GeoMAD"

Select Kernel in Notebook

1. Open your notebook (.ipynb file)
2. Click Kernel → Change Kernel → Indonesia GeoMAD
3. Verify the kernel name appears in the top-right corner

Add New Dependencies

If you need to install additional packages:

# Activate environment first
source .venv/bin/activate

# Add a new package
uv add package-name

# Add a new package to the notebook environment
uv add --group notebook package-name

# Sync to apply changes
uv sync --extra notebook

Acknowledgments

• Open Data Cube community
• odc-stats developers
• Digital Earth Australia for GeoMAD algorithm development

Low Resolution COGs

For visualisation, we make low resolution mosaic COGs of the S2 GeoMAD Annual product.