From c58cac9b2b46bc8f42d69dfe9083078086984c22 Mon Sep 17 00:00:00 2001 From: cadauxe Date: Tue, 2 Sep 2025 11:53:43 +0200 Subject: [PATCH 1/2] docs: added guidelines on slurp's usage --- docs/source/index.rst | 2 +- docs/source/installation.rst | 56 +----- docs/source/main_config_descr.md | 37 ++-- docs/source/slurp_config.rst | 10 +- docs/source/tutorial.rst | 286 ++++++++++++++++++++++++++++++- docs/source/usage_api.rst | 56 +++--- docs/source/usage_cli.rst | 101 +++-------- 7 files changed, 376 insertions(+), 172 deletions(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index e4223db9..78ca731c 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -17,10 +17,10 @@ obtain into a single multiclass mask. :maxdepth: 1 SLURP configuration + Tutorial CLI Usage API Usage API Documentation - Slurp tutorial .. toctree:: :caption: Contributing diff --git a/docs/source/installation.rst b/docs/source/installation.rst index c061ba02..d465d6f0 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -8,15 +8,19 @@ Installation Stable release -------------- -If deployed in Pypi, to install slurp, run this command in your terminal: +SLURP is deployed on Pypi. To install slurp, please create a python 3.10 virtual environment : .. code-block:: console - $ pip install slurp + $ python3.10 -m venv slurp_env + $ pip install slurp-masks This is the preferred method to install slurp, as it will always install the most recent stable release. -Consider using a virtualenv to separate and test the installation. +.. warning:: + + Depending on which computation you want to perform (cf. `Use cases `_ page), you may need to install OTB on your system as an additional dependencies. + Please refer to the OTB installation guide provided `here `_ for more details. If you don't have `pip`_ installed, this `Python installation guide`_ can guide you through the process. @@ -28,7 +32,7 @@ you through the process. From sources ------------ -The sources for slurp can be downloaded from the `Github repo`_. +The sources for SLURP can be downloaded from the `Github repo`_. You can either clone the public repository: @@ -51,50 +55,6 @@ Once you have a copy of the source, you can install it in a virtualenv with: $ make install $ source venv/bin/activate -Install (from the README.md) -------- - -You need to clone the repository and pip install SLURP. - -.. code-block:: console - - git clone git@gitlab.cnes.fr:pluto/slurp.git - -To install SLURP, you need OTB, -`EOScale `__ and some libraries -already installed on VRE OT. - -Otherwise, if you are are connected to TREX, or working on your personal -computer (Linux), you may set the environment as mentioned below. ### -Create a virtual env with all libraries (if you don’t use VRE OT) On -TREX, connect to a computing node to create & compile the virtual -environment (needed to compile rasterio at install time) - -.. code-block:: console - - sinter -A cnes_level2 -N 1 -n 8 --time=02:00:00 --mem=64G --x11 --pty bash - -Load OTB and create a virtual env with some Python libraries. Compile -and install EOScale and then SLURP - -.. code-block:: console - - module load otb/9.0.0-python3.8 - # Creates a virtual env base on Python 3.8.13 - python -m venv slurp_env - . slurp_env/bin/activate - # upgrade pip and install several libraries - pip install pip --upgrade - cd - pip install . - cd - pip install . - # for validation tests - pip install pytest - -Your environment is ready, you can compute SLURP masks with -slurp_watermask, slurp_urbanmask, etc. - .. _Github repo: https://github.com/CNES/slurp .. _tarball: https://github.com/CNES/slurp/tarball/master diff --git a/docs/source/main_config_descr.md b/docs/source/main_config_descr.md index d3435cf2..1db676ea 100644 --- a/docs/source/main_config_descr.md +++ b/docs/source/main_config_descr.md @@ -7,12 +7,13 @@ | |   hand | Str | Path of the global HAND file | Mandatory | | |   wsf | Str | Path to store the extracted WSF file | Mandatory | | **aux_layers** | | | Auxiliary layers used in processing | Mandatory | -| |   extracted_pekel | Str | Path to the extracted Pekel data | Mandatory | -| |   extracted_hand | Str | Path to the extracted HAND data | Mandatory | -| |   extracted_wsf | Str | Extracted World Settlement Footprint raster filename | Mandatory | | |   valid_stack | Str | Path to store the valid stack file | Optional | | |   file_ndvi | Str | Path to the NDVI layer | Mandatory | | |   file_ndwi | Str | Path to the NDWI layer | Mandatory | +| |   extracted_pekel | Str | Path to the extracted Pekel data | Mandatory | +| |   extracted_hand | Str | Path to the extracted HAND data | Mandatory | +| |   extracted_wsf | Str | Path to the extracted World Settlement Footprint | Mandatory | +| |   extracted_wbm | Str | Path to the extracted Water Body Mask | Optional | | |   file_texture | Str | Path to store the texture file | Mandatory | | |   mnh | Str | Path to the Mean Height of Nearest Neighbors (MN) layer | Optional | | |   file_cloud_gml | Str | GML file containing cloud masks (optional) | Optional | @@ -23,6 +24,7 @@ | |   shadowmask | Str | For shadowmask computation : Output classification filename. For stackmask: Shadow mask to stack. Otherwise, if given, big shadow areas will be marked as background | Mandatory | | |   stackmask | Str | Output classification filename | Mandatory | | **resources** | | | Resources configuration for parallel processing | Mandatory | +| |   debug | Bool | Flag for debugging mode (default : False) | Optional | | |   n_workers | Int | Number of CPU | Mandatory | | |   tile_max_size | Int | Maximum size of tiles to process | Mandatory | | |   multiproc_context | Str | Multiprocessing strategy: 'fork' or 'spawn' for EOScale | Optional | @@ -34,11 +36,12 @@ | |   nir | Int | NIR band index | Mandatory | | |   cloud_mask | Str | Path to the input cloud mask | Optional | | |   pekel_method | Str | Method for Pekel recovery : 'all' for global file and 'month' for monthly recovery | Mandatory | -| |   pekel | Str | Path of the global Pekel file | Mandatory | +| |   pekel | Str | Path of the global Pekel (Global Surface Water) file | Mandatory | | |   pekel_monthly_occurrence | Str | Path of the root of monthly occurrence Pekel files | Optional | -| |   pekel_obs | Str | Month of the desired Pekel file (pekel_method = month) | Optional | -| |   hand | Str | Path of the global HAND file | Mandatory | -| |   wsf | Str | Path of the global WSF file | Mandatory | +| |   pekel_obs | Str | Month of the desired Pekel (Global Surface Water) file (pekel_method = month) | Optional | +| |   hand | Str | Path of the global Height Above Nearest Drainage (HAND) file | Mandatory | +| |   wsf | Str | Path of the global World Settlement Footprint (WSF) file | Mandatory | +| |   wbm | Str | Path of the global Water Body Mask (WBM) file | Mandatory | | |   texture_rad | Int | Radius for texture (std convolution) computation | Mandatory | | |   dtm | Str | Digital Terrain Model, used only in sensor mode | Optional | | |   geoid_file | Str | Geoid file, used only in sensor mode | Mandatory | @@ -79,7 +82,6 @@ | |   non_veg_clusters | List[Int] | Labelize each 'non vegetation cluster' as 0, 1, 2 (..) instead of single label (0) | Optional | | |   nb_clusters_low_veg | Int | Nb of clusters considered as low vegetation (1-NB_CLUSTERS) | Mandatory | | |   max_texture_th | Float | Maximal texture value to consider a cluster as low vegetation (overload nb clusters choice) | Optional | -| |   debug | Bool | Flag for debugging mode | Mandatory | | **water** | | | Water detection settings | Mandatory | | |   files_layers | List[Str] | Add layers as additional features used by learning algorithm | Mandatory | | |   thresh_pekel | Int | Threshold for Pekel water occurrence detection | Mandatory | @@ -106,15 +108,18 @@ | |   building_erosion | Int | Supposed buildings will be eroded by this size in the marker step | Mandatory | | |   bonus_gt | Int | Bonus for pixels covered by GT, in the watershed regularization step (ex : +30 to improve discrimination between building and background) | Mandatory | | |   malus_shadow | Int | Value of the malus for pixels in shadow, in the watershed regularization step | Mandatory | -| |   value_classif_low_veg | Int | Output classification value for low vegetation | Mandatory | -| |   value_classif_high_veg | Int | Output classification value for high vegetation | Mandatory | -| |   value_classif_water | Int | Output classification value for water | Mandatory | -| |   value_classif_buildings | Int | Output classification value for buildings | Mandatory | -| |   value_classif_bare_ground | Int | Output classification value for bare ground | Mandatory | +| |   value_classif_low_veg | Int | Output classification value for low vegetation | Optional | +| |   value_classif_high_veg | Int | Output classification value for high vegetation | Optional | +| |   value_classif_water | Int | Output classification value for water | Optional | +| |   value_classif_buildings | Int | Output classification value for buildings | Optional | +| |   value_classif_bare_ground | Int | Output classification value for bare ground | Optional | +| |   value_classif_sea | Int | Output classification value for sea | Optional | +| |   value_classif_lake | Int | Output classification value for sea | Optional | +| |   value_classif_river | Int | Output classification value for sea | Optional | | |   value_classif_false_positive_buildings | Int | Output classification value for buildings false positive | Mandatory | | |   value_classif_background | Int | Output classification value for background | Mandatory | -| |   vegmask_max_value | Int | Maximum allowed value in the vegetation mask | Mandatory | +| |   vegmask_min_value | Int | Maximum allowed value in the vegetation mask | Mandatory | | |   binary_closing | Int | Size of disk structuring element | Mandatory | | |   binary_opening | Int | Size of disk structuring element | Mandatory | -| |   remove_small_objects | Int | The maximum area, in pixels, of a contiguous object that will be removed | Mandatory | -| |   remove_small_holes | Int | The maximum area, in pixels, of a contiguous hole that will be filled | Mandatory | \ No newline at end of file +| |   categorized_watermask | Bool | If true, stack_mask will infer water body category (lake, river, sea, unknown) from a general water body mask | Optional | +| |   minimal_size_water_area | Int | Minimal area (in pixels) of water bodies | Optional | \ No newline at end of file diff --git a/docs/source/slurp_config.rst b/docs/source/slurp_config.rst index f6be4b85..f17c4879 100644 --- a/docs/source/slurp_config.rst +++ b/docs/source/slurp_config.rst @@ -6,12 +6,14 @@ SLURP can be configured with JSON files that allows you to set various parameter The main configuration file is given to the SLURP's API or CLI through the `main_config` argument. -Below is an example of a main configuration file: +.. tip:: + + Some arguments defined by the JSON file can be overwritten by command line arguments or by API's arguments. + For more information, please refer to `API's documentation `_ or refer to the help of your CLI command (`cli_command_name --help`). + +Below is an example of a configuration file: .. include:: main_config_descr.md :parser: myst_parser.sphinx_ -Some arguments defined by the JSON file can be overwritten by command line arguments or by API's arguments. -For more information, please refer to `API's documentation `_ or refer to the help of your CLI command (`cli_command_name --help`). - diff --git a/docs/source/tutorial.rst b/docs/source/tutorial.rst index 7cfd6684..1f1c9b41 100644 --- a/docs/source/tutorial.rst +++ b/docs/source/tutorial.rst @@ -1,7 +1,287 @@ .. _tutorial: -slurp tutorial +SLURP tutorial ============== -.. include:: Tutorial.md - :parser: myst_parser.sphinx_ \ No newline at end of file +This document presents an example of the SLURP pipeline on an image extract from Strasbourg (France). + +.. figure:: _static/images/tutorials/strasbourg_vhr_image.png + :alt: VHR image + :width: 40% + :align: center + + VHR image + +----------- +First steps +----------- + + +Activate your environment containing SLURP +__________________________________________ + +.. warning:: + + Depending on which computation you want to perform (cf. `SLURP pipeline <#slurp-pipeline>`_ section), you may need to install OTB on your system as an additional dependencies. + Please refer to the OTB installation guide provided `here `_ for more details. + +.. code-block:: console + + $ source slurp_env/bin/activate + + +Fill in the configuration file +------------------------------ + +A template for the configuration file is available `here `_. +All parameters are explained in the `SLURP configuration `_ page. + +Create the output folders +------------------------- + +.. code-block:: console + + mkdir -p prepare + mkdir -p out + +-------------- +SLURP pipeline +-------------- + +Additional shell scripts are provided `here `_ to launch the steps of the +pipeline, but they can also be launched separately as explained below. + +Usage without sensor mode (Additional OTB steps may be necessary) +_________________________________________________________________ + +If your input data has undergone geometric processing (e.g. orthorectification), you may need to use OTB to reproject and crop the +auxiliary data (Pekel, Hand and WSF masks) to match your input image. + +.. note:: + + **Pekel** and **Hand** data are used to compute the water mask, **WSF** data is used to compute the urban mask. + Vegetation and shadow masks computation do not require any additional data. + + Therefore, **OTB is not required to compute vegetation and shadow masks.** + +To install OTB, please refer to the `guide `_ provided. + +Once installed, you can use the OTB command line interface (CLI) to superimpose Pekel, Hand and WSF data with your input image. + +.. code-block:: console + + # Superimpose Pekel, Hand and WSF with OTB + # /!\ Adapt path depending on where your global Pekel database (resp. HAND, WSF) is located + otbcli_Superimpose -inr path_to_input/input_file.tif -inm path_to_pekel/pekel_file.vrt -out "out/pekel.tif?&gdal:co:TILED=YES&gdal:co:COMPRESS=DEFLATE" uint8 -interpolator nn + otbcli_Superimpose -inr path_to_input/input_file.tif -inm path_to_hand/hand_file.vrt -out "out/hand.tif?&gdal:co:TILED=YES&gdal:co:COMPRESS=DEFLATE" + otbcli_Superimpose -inr path_to_input/input_file.tif -inm path_to_wsf/wsf_file.vrt -out "out/wsf.tif?&gdal:co:TILED=YES&gdal:co:COMPRESS=DEFLATE" uint8 -interpolator nn + + +Usage in sensor mode (OTB is not required) +------------------------------------------ + +If your input data image is in sensor geometry, please use SLURP with the ``sensor_mode`` option set to ``True``. +This mode allows to use images in sensor geometry, Pekel, Hand and WSF masks are directly processed during SLURP's preparation step. + + +SLURP's preparation step +------------------------ + +We need to prepare all the auxiliary data required to calculate the masks. This includes reprojection (when using ``sensor_mode`` option) and +cropping. + +Run the following command : + +.. code-block:: console + + slurp_prepare main_config.json + +Or use the Python API : + +.. code-block:: python + + import slurp-masks as slurp + + slurp_prepare(main_config = main_config.json) + +The outputs of the prepare script have been stored to the `prepare` directory. + +It contains : + + - the validity mask + - the NDVI and NDWI primitives + - the texture analysis + - the superimposed Pekel, HAND and WSF files (only if using `sensor_mode` option) + - the global config JSON file + +.. raw:: html + + + + + + + + + + + + + + + + +
+ NDVI image + + NDWI image + + Texture computation + + Pekel mask + + WSF mask +
NDVI of the VHR imageNDWI of the VHR imageSquared convolution with a kernel of onesReprojected and cropped Pekel maskReprojected and cropped WSF mask
+ +SLURP's masks computation +------------------------- + +Now, the different masks can be computed and finally stacked together. Use either the CLI or the Python API to launch each step. + +**Generate the water mask** + +.. code-block:: console + + slurp_watermask prepare/effective_used_config.json + +.. code-block:: python + + import slurp-masks as slurp + + slurp_watermask(main_config = prepare/effective_used_config.json) + + +**Generate the vegetation mask** + +.. code-block:: console + + slurp_vegetationmask prepare/effective_used_config.json + +.. code-block:: python + + import slurp-masks as slurp + + slurp_vegetationmask(main_config = prepare/effective_used_config.json) + +**Generate the shadow mask** + +.. code-block:: console + + slurp_shadowmask prepare/effective_used_config.json + +.. code-block:: python + + import slurp-masks as slurp + + slurp_shadowmask(main_config = prepare/effective_used_config.json) + + +**Generate the urban mask** + +.. code-block:: console + + slurp_urbanmask prepare/effective_used_config.json + +.. code-block:: python + + import slurp-masks as slurp + + slurp_urbanmask(main_config = prepare/effective_used_config.json) + +**Stack the masks** + +.. code-block:: console + + slurp_stackmasks prepare/effective_used_config.json + +.. code-block:: python + + import slurp-masks as slurp + + slurp_stackmasks(main_config = prepare/effective_used_config.json) + +The outputs masks have been stored to the `out` directory. + +It contains : + + - the water mask + - the shadow mask + - the urban mask + - the vegetation mask + - the stack mask + +Results without sensor mode +--------------------------- + +.. raw:: html + + + + + + + + + + + + + + + + +
+ Water mask + + Low/High vegetation and bare ground mask + + Shadow mask + + Urban probability + + Final mask +
Water mask with style conf/style_water.qmlVegetation mask with style conf/style_vegetation.qmlShadow mask with style conf/style_shadow.qmlUrban mask (building probability)Stack mask with style conf/style_stack.qml
+ + +Results with sensor mode +------------------------ + +.. raw:: html + + + + + + + + + + + + + + + + +
+ Water mask + + Low/High vegetation and bare ground mask + + Shadow mask + + Urban probability + + Final mask +
Water mask with style conf/style_water.qmlVegetation mask with style conf/style_vegetation.qmlShadow mask with style conf/style_shadow.qmlUrban mask (building probability)Stack mask with style conf/style_stack.qml
+ diff --git a/docs/source/usage_api.rst b/docs/source/usage_api.rst index 2d8d09c5..e404e99b 100644 --- a/docs/source/usage_api.rst +++ b/docs/source/usage_api.rst @@ -2,13 +2,17 @@ API Usage ========= -SLURP can be used as a Python library. Please start by installing it through pip. +SLURP uses some global data, such as Global Surface Water (Pekel) for +water detection or World Settlement Footprint (WSF) for building +detection. -.. code-block:: python - - pip install slurp-masks +Data preparation can be achieved with Orfeo ToolBox or other tools (for more information, please refer to `SLURP's tutorial `_ page), in +order to bring all necessary data in the same projection. You can either +build your mask step by step, or use a batch script to launch and build +the final mask automatically. -Then you can import it in your Python scripts or notebooks. +SLURP can be used as a Python library. Please refer to the `installation guide `_. +Then, you can import it in your Python scripts or notebooks. .. code-block:: python @@ -19,8 +23,15 @@ Data preparation Each mask needs some auxiliary files. They must be on the same projection, resolution and bounding box of the VHR input image to enable -mask computation. You can generate this data yourself or use the prepare -script available in SLURP. +mask computation. You can generate this data yourself or use the `prepare +script `_ available in SLURP. + +.. warning:: + + Depending on which computation you want to perform (cf. `SLURP pipeline <#slurp-pipeline>`_ section), you may need to install OTB on + your system. + + Please refer to the provided `tutorial `_ to know if additional steps are required. The prepare script enables : @@ -31,12 +42,10 @@ The prepare script enables : - Extraction of WSF file - Computation of texture file with a convolution -It requires an OTB installation. **To run the script** -1. Configure the JSON file. A template is available at - conf/main_config.json with default values. +1. Configure the JSON file. A template is available `here `_ with default values. 2. Update input, aux_layers, resources and prepare blocs inside the JSON file. 3. Use the python function: @@ -50,7 +59,7 @@ It requires an OTB installation. wsf, extracted_wsf, file_texture, texture_rad, analyse_glcm, land_cover_map, cropped_land_cover_map, n_workers, tile_max_size, multiproc_context) -Please check complete arguments description available `here `_. +Please check complete arguments description available `here `_. Beware that API arguments override the JSON arguments. Features @@ -66,8 +75,7 @@ post-processed to clean artefacts. **To compute the mask** -1. Configure the JSON file : a template is available at - conf/main_config.json with default values. +1. Configure the JSON file. A template is available `here `_ with default values. 2. Update input, aux_layers and masks blocs inside the JSON file. To go further you can modify resources, post_process and water blocs. 3. Run the command : @@ -86,7 +94,7 @@ post-processed to clean artefacts. value_classif, n_workers, tile_max_size, multiproc_context) -Please check complete arguments description available `here `_. +Please check complete arguments description available `here `_. Beware that API arguments override the JSON arguments. Vegetation mask @@ -101,8 +109,7 @@ texture denotes for low vegetation). **To compute the mask** -1. Configure the JSON file : a template is available at - conf/main_config.json with default values. +1. Configure the JSON file. A template is available `here `_ with default values. 2. Update input, aux_layers and masks blocs inside the JSON file. To go further you can modify resources and vegetation blocs. 3. Run the command : @@ -116,7 +123,7 @@ texture denotes for low vegetation). binary_dilation, remove_small_objects, remove_small_holes, vegetationmask, n_workers, tile_max_size, multiproc_context) -Please check complete arguments description available `here `_. +Please check complete arguments description available `here `_. Beware that API arguments override the JSON arguments. @@ -131,8 +138,7 @@ probability” layer ([0..100]) that can be used by the stack algorithm. **To compute the mask** -1. Configure the JSON file : a template is available at - conf/main_config.json with default values. +1. Configure the JSON file. A template is available `here `_ with default values. 2. Update input, aux_layers and masks blocs inside the JSON file. To go further you can modify resources and urban blocs. 3. Run the command : @@ -145,7 +151,7 @@ probability” layer ([0..100]) that can be used by the stack algorithm. gt_binary_erosion, save_mode, nb_samples_urban, nb_samples_other, max_depth, nb_estimators, n_jobs, urbanmask, n_workers, tile_max_size, multiproc_context) -Please check complete arguments description available `here `_. +Please check complete arguments description available `here `_. Beware that API arguments override the JSON arguments. @@ -160,8 +166,7 @@ in the regularization step. **To compute the mask** -1. Configure the JSON file : a template is available at - conf/main_config.json with default values. +1. Configure the JSON file. A template is available `here `_ with default values. 2. Update input, aux_layers and masks blocs inside the JSON file. To go further you can modify resources, post_process and shadow blocs. 3. Run the command : @@ -172,7 +177,7 @@ in the regularization step. th_rgb, th_nir, absolute_threshold, percentile, binary_opening, remove_small_objects, shadowmask, n_workers, tile_max_size, multiproc_context) -Please check complete arguments description available `here `_. +Please check complete arguments description available `here `_. Beware that API arguments override the JSON arguments. Stack and regularize buildings @@ -189,8 +194,7 @@ gradient image, thus regularizing buildings shapes. **To compute the mask** -1. Configure the JSON file : a template is available at - conf/main_config.json with default values. +1. Configure the JSON file. A template is available `here `_ with default values. 2. Update input, aux_layers and masks element inside the JSON file. To go further you can modify resources, post_process and stack blocs. 3. Run the command : @@ -205,7 +209,7 @@ gradient image, thus regularizing buildings shapes. value_classif_false_positive_buildings, value_classif_background, n_workers, tile_max_size, multiproc_context) -Please check complete arguments description available `here `_. +Please check complete arguments description available `here `_. Beware that API arguments override the JSON arguments. diff --git a/docs/source/usage_cli.rst b/docs/source/usage_cli.rst index 6bb4d85e..ca0a93ce 100644 --- a/docs/source/usage_cli.rst +++ b/docs/source/usage_cli.rst @@ -2,27 +2,11 @@ CLI Usage ========= -By default, a fake command line is deployed with argparse: - -To use it: - -.. code-block:: console - - $ slurp -h - - -To use slurp in a project: - -.. code-block:: python - - import slurp - - SLURP uses some global data, such as Global Surface Water (Pekel) for water detection or World Settlement Footprint (WSF) for building detection. -Data preparation can be achieved with Orfeo ToolBox or other tools, in +Data preparation can be achieved with Orfeo ToolBox or other tools (for more information, please refer to `SLURP's tutorial `_ page), in order to bring all necessary data in the same projection. You can either build your mask step by step, or use a batch script to launch and build the final mask automatically. @@ -31,15 +15,15 @@ the final mask automatically. - - - - - - + + + + + + - + @@ -49,53 +33,30 @@ the final mask automatically.
Initial VHR imageWater maskVegetation maskShadow maskUrban probabilityFinal maskInitial VHR imageWater maskVegetation maskShadow maskUrban probabilityFinal mask
Bring your own VHR 4 bands (R/G/B/NIR) imageBring your own VHR 4 bands
(R/G/B/NIR) image		 Learn 'Pekel' water occurrence Detect low/high vegetation Detect shadows (avoid water confusion)
- -Getting Started ---------------- - -Once your environment has been set up, you can run SLURP. - -A tutorial (with and without OTB) is available : -`Tutorial.md `__. - -Use SLURP on TREX ------------------ - -On TREX, you can directly use SLURP by sourcing the following -environment. - -.. code-block:: console - - source /work/CAMPUS/users/tanguyy/PLUTO/slurp_demo/init_slurp.sh - -This will load OTB 9.0 and all Python dependencies - -You can also use a shell script with SLURM to launch different masks -algorithms on your images. - -.. code-block:: console - - sbatch --export="PHR_IM=/work/scratch/tanguyy/public/RemyMartin/PHR_image_uint16.tif,OUTPUT_DIR=/work/scratch/tanguyy/public/RemyMartin/,CLUSTERS_VEG=4,CLUSTERS_LOW_VEG=2" /softs/projets/pluto/demo_slurp/compute_all_masks.pbs - -Two scripts (to calculate all the masks and the scores) are available in -conf/ directory. - +Once your environment has been set up, you can run SLURP. A `tutorial `_ is available. Log Display Options ------------------- By default, logs are displayed in the console. -When the CLI argument `--logs_to_file` is specified, -logs will instead be saved to the file `logs/out_logs.log` located in your current working directory. +When the CLI argument ``--logs_to_file`` is specified, +logs will instead be saved to the file ``logs/out_logs.log`` located in your current working directory. Data preparation ---------------- Each mask needs some auxiliary files. They must be on the same projection, resolution and bounding box of the VHR input image to enable -mask computation. You can generate this data yourself or use the prepare -script available in SLURP. +mask computation. You can generate this data yourself or use the `prepare +script `_ available in SLURP. + +.. warning:: + + Depending on which computation you want to perform (cf. `SLURP pipeline <#slurp-pipeline>`_ section), you may need to install OTB on + your system. + + Please refer to the provided `tutorial `_ to know if additional steps are required. The prepare script enables : @@ -106,12 +67,9 @@ The prepare script enables : - Extraction of WSF file - Computation of texture file with a convolution -It requires an OTB installation. - **To run the script** -1. Configure the JSON file. A template is available at - conf/main_config.json with default values. +1. Configure the JSON file. A template is available `here `_ with default values. 2. Update input, aux_layers, resources and prepare blocs inside the JSON file. 3. Run the command : @@ -144,8 +102,7 @@ post-processed to clean artefacts. **To compute the mask** -1. Configure the JSON file : a template is available at - conf/main_config.json with default values. +1. Configure the JSON file : A template is available `here `_ with default values. 2. Update input, aux_layers and masks blocs inside the JSON file. To go further you can modify resources, post_process and water blocs. 3. Run the command : @@ -177,8 +134,7 @@ texture denotes for low vegetation). **To compute the mask** -1. Configure the JSON file : a template is available at - conf/main_config.json with default values. +1. Configure the JSON file : A template is available `here `_ with default values. 2. Update input, aux_layers and masks blocs inside the JSON file. To go further you can modify resources and vegetation blocs. 3. Run the command : @@ -208,8 +164,7 @@ probability” layer ([0..100]) that can be used by the stack algorithm. **To compute the mask** -1. Configure the JSON file : a template is available at - conf/main_config.json with default values. +1. Configure the JSON file : A template is available `here `_ with default values. 2. Update input, aux_layers and masks blocs inside the JSON file. To go further you can modify resources and urban blocs. 3. Run the command : @@ -239,8 +194,7 @@ in the regularization step. **To compute the mask** -1. Configure the JSON file : a template is available at - conf/main_config.json with default values. +1. Configure the JSON file : A template is available `here `_ with default values. 2. Update input, aux_layers and masks blocs inside the JSON file. To go further you can modify resources, post_process and shadow blocs. 3. Run the command : @@ -272,8 +226,7 @@ gradient image, thus regularizing buildings shapes. **To compute the mask** -1. Configure the JSON file : a template is available at - conf/main_config.json with default values. +1. Configure the JSON file : A template is available `here `_ with default values. 2. Update input, aux_layers and masks element inside the JSON file. To go further you can modify resources, post_process and stack blocs. 3. Run the command : @@ -310,7 +263,7 @@ masks with ``pytest -m computation`` or validate them with two last modes require OTB installation. You can change the default configuration for the tests by modifying the -JSON file “tests/config_tests”. +`test configuration `_ JSON file. Documentation ------------- From b9aa916d9036561fa08a9ea00b2461f9d5b2142b Mon Sep 17 00:00:00 2001 From: cadauxe Date: Wed, 3 Sep 2025 17:14:50 +0200 Subject: [PATCH 2/2] docs: added doc --- docs/source/tutorial.rst | 99 ++++++++++++++++++++++----------------- docs/source/usage_api.rst | 2 +- docs/source/usage_cli.rst | 2 +- 3 files changed, 59 insertions(+), 44 deletions(-) diff --git a/docs/source/tutorial.rst b/docs/source/tutorial.rst index 1f1c9b41..600c0717 100644 --- a/docs/source/tutorial.rst +++ b/docs/source/tutorial.rst @@ -3,6 +3,11 @@ SLURP tutorial ============== +SLURP is designed to compute a simple land cover map (water, low/high vegetation, bare groud, buildings) from a VHR (Very High Resolution) image with +4 bands (Red, Green, Blue, Near-Infrared). SLURP is based on few or unsupervised algorithms (random forest, clustering, segmentation) that need some auxiliary data for training step. + +It had been validated with Pleiades and tested with WordView, CO3D, Pleiades NEO images. It shall also work with images with lower resolution (ex : SPOT 6/7). + This document presents an example of the SLURP pipeline on an image extract from Strasbourg (France). .. figure:: _static/images/tutorials/strasbourg_vhr_image.png @@ -30,20 +35,62 @@ __________________________________________ $ source slurp_env/bin/activate +Get the necessary input data +---------------------------- + +In addition to the VHR input image, SLURP requires some auxiliary data to compute the different masks. + +.. raw:: html + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Auxiliary dataStep / usageComments
Pekel
(Global Surface Water - uint8)		Water mask prediction : Water occurrence [0–100] during the last 30 years, used to learn a water prediction model (MANDATORY)The global map is mandatory but you can also use some data by month
Hand MERIT
(float32)		Water mask prediction : Map of height above nearest drainage used to optimize choice of "non water" samples in the training step (OPTIONAL)Free after registration (other kind of HAND maps exist)
WSF 2019
(World Settlement Footprint - uint8)		Urban mask prediction : Global buildings map used to learn a building prediction model (MANDATORY)Could be replaced by a better resolution map if available (e.g., OSM buildings)
ESA WorldCover
(uint8)		Vegetation mask configuration : Global land cover map (10m resolution) used to customize vegetation clustering (OPTIONAL)Very helpful to parameterize balance between non-vegetation / low and high vegetation clusters (see vegetation mask algorithm)
Copernicus WBM
(float32 or int16)		Stack mask : Water Body Mask from Copernicus, used to classify each water body from the watermask (OPTIONAL)If used, the final map will contain river, lake, sea classes
+

+ + Fill in the configuration file ------------------------------ A template for the configuration file is available `here `_. All parameters are explained in the `SLURP configuration `_ page. -Create the output folders -------------------------- - -.. code-block:: console - - mkdir -p prepare - mkdir -p out - -------------- SLURP pipeline -------------- @@ -220,41 +267,9 @@ It contains : - the vegetation mask - the stack mask -Results without sensor mode ---------------------------- - -.. raw:: html - - - - - - - - - - - - - - - - -
- Water mask - - Low/High vegetation and bare ground mask - - Shadow mask - - Urban probability - - Final mask -
Water mask with style conf/style_water.qmlVegetation mask with style conf/style_vegetation.qmlShadow mask with style conf/style_shadow.qmlUrban mask (building probability)Stack mask with style conf/style_stack.qml
- -Results with sensor mode ------------------------- +Results +------- .. raw:: html diff --git a/docs/source/usage_api.rst b/docs/source/usage_api.rst index e404e99b..7a7e3bbd 100644 --- a/docs/source/usage_api.rst +++ b/docs/source/usage_api.rst @@ -28,7 +28,7 @@ script `_ available in SLURP. .. warning:: - Depending on which computation you want to perform (cf. `SLURP pipeline <#slurp-pipeline>`_ section), you may need to install OTB on + Depending on which computation you want to perform, you may need to install OTB on your system. Please refer to the provided `tutorial `_ to know if additional steps are required. diff --git a/docs/source/usage_cli.rst b/docs/source/usage_cli.rst index ca0a93ce..9d235813 100644 --- a/docs/source/usage_cli.rst +++ b/docs/source/usage_cli.rst @@ -53,7 +53,7 @@ script `_ available in SLURP. .. warning:: - Depending on which computation you want to perform (cf. `SLURP pipeline <#slurp-pipeline>`_ section), you may need to install OTB on + Depending on which computation you want to perform, you may need to install OTB on your system. Please refer to the provided `tutorial `_ to know if additional steps are required.