Data normalization

Normalize raster.

remote_sensing_processor.normalize.min_max(input_path, minimum, maximum, output_path=None, clip_values=False, nodata=None, write_stac=True)[source]

Applies min-max normalization to an input file.

Recommended if you want your data in the 0-1 range.

Parameters:

  • input_path (string or STAC Item) – Path to input file, directory or STAC dataset or a STAC Item (e.g., from Planetary Computer).

  • minimum (int or float) – Min value.

  • maximum (int or float) – Max value.

  • output_path (string (optional)) – Path to an output file, directory, or STAC dataset. If not set, then will overwrite the input files. Must be set if input is a remote STAC Item.

  • clip_values (bool (default = False)) – If True, limits output values to min-max range. Otherwise, it will just normalize the values and if there are values outside the min-max range, they would be larger than 1 or smaller than 0.

  • nodata (int or float (optional)) – Nodata value. If not set, then is read from inputs.

  • write_stac (bool (default = True)) – If True, then output metadata is saved to a STAC file.

Returns:

Path where output raster is saved.

Return type:

pathlib.Path

Examples

>>> import remote_sensing_processor as rsp
>>> rsp.normalize.min_max(
...     input_path="/home/rsp_test/mosaics/sentinel/B1.tif",
...     minimum=0,
...     maximum=10000,
...     output_path="/home/rsp_test/mosaics/sentinel/B1_norm.tif",
... )
'/home/rsp_test/mosaics/sentinel/B1_norm.json'
remote_sensing_processor.normalize.z_score(input_path, mean, stddev, output_path=None, nodata=None, write_stac=True)[source]

Applies z-score normalization to an input file.

Recommended if you want all of your data to have the same distribution, but the data will not be limited to 0-1 range.

Parameters:

  • input_path (string or STAC Item) – Path to input file, directory or STAC dataset or a STAC Item (e.g., from Planetary Computer).

  • mean (int or float) – Mean value.

  • stddev (int or float) – Standard deviation value.

  • output_path (string (optional)) – Path to an output file, directory, or STAC dataset. If not set, then will overwrite the input files. Must be set if input is a remote STAC Item.

  • nodata (int or float (optional)) – Nodata value. If not set, then is read from inputs.

  • write_stac (bool (default = True)) – If True, then output metadata is saved to a STAC file.

Returns:

Path where output raster is saved.

Return type:

pathlib.Path

Examples

>>> import remote_sensing_processor as rsp
>>> rsp.normalize.z_score(
...     input_path="/home/rsp_test/mosaics/sentinel/B1.tif",
...     mean=302,
...     stddev=173,
...     output_path="/home/rsp_test/mosaics/sentinel/B1_norm.tif",
... )
'/home/rsp_test/mosaics/sentinel/B1_norm.json'
remote_sensing_processor.normalize.dynamic_world(input_path, percentile1, percentile2, output_path=None, clip_values=True, nodata=None, write_stac=True)[source]

Applies log-transform + sigmoid normalization to an input file.

This normalization method is similar to a method described in https://doi.org/10.1038/s41597-022-01307-4. Recommended if you want to have your data in 0-1 range and handle outliers well.

Parameters:

  • input_path (string or STAC Item) – Path to input file, directory or STAC dataset or a STAC Item (e.g., from Planetary Computer).

  • percentile1 (int or float) – First log-transformed data percentile.

  • percentile2 (int or float) – Second log-transformed data percentile.

  • output_path (string (optional)) – Path to an output file, directory, or STAC dataset. If not set, then will overwrite the input files. Must be set if input is a remote STAC Item.

  • clip_values (bool (default = True)) – If True, will convert negative values in input data to small positive values, otherwise negative values will become nodata because ln(neg) is a nan.

  • nodata (int or float (optional)) – Nodata value. If not set, then is read from inputs.

  • write_stac (bool (default = True)) – If True, then output metadata is saved to a STAC file.

Returns:

Path where output raster is saved.

Return type:

pathlib.Path

Examples

>>> import remote_sensing_processor as rsp
>>> rsp.normalize.dynamic_world(
...     input_path="/home/rsp_test/mosaics/sentinel/B1.tif",
...     percentile1=5.49,
...     percentile2=5.78,
...     output_path="/home/rsp_test/mosaics/sentinel/B1_norm.tif",
... )
'/home/rsp_test/mosaics/sentinel/B1_norm.json'

Denormalize raster.

remote_sensing_processor.denormalize.min_max(input_path, minimum=None, maximum=None, output_path=None, nodata=0, write_stac=True)[source]

Recovers original values from min-max normalized raster.

Parameters:

  • input_path (string or STAC Item) – Path to input file, directory or STAC dataset or a STAC Item (e.g., from Planetary Computer).

  • minimum (int or float) – Min value that was used for normalization.

  • maximum (int or float) – Max value that was used for normalization.

  • output_path (string (optional)) – Path to an output file, directory, or STAC dataset. If not set, then will overwrite the input files. Must be set if input is a remote STAC Item.

  • nodata (int or float (default = 0)) – Nodata value to be used in output data. If not set, then is read from inputs.

  • write_stac (bool (default = True)) – If True, then output metadata is saved to a STAC file.

Returns:

Path where output raster is saved.

Return type:

pathlib.Path

Examples

>>> import remote_sensing_processor as rsp
>>> rsp.denormalize.min_max(
...     input_path="/home/rsp_test/mosaics/sentinel/B1_norm.tif",
...     minimum=0,
...     maximum=10000,
...     output_path="/home/rsp_test/mosaics/sentinel/B1_orig.tif",
... )
'/home/rsp_test/mosaics/sentinel/B1_orig.json'
remote_sensing_processor.denormalize.z_score(input_path, mean, stddev, output_path=None, nodata=0, write_stac=True)[source]

Recovers original values from z-score normalized raster.

Parameters:

  • input_path (string or STAC Item) – Path to input file, directory or STAC dataset or a STAC Item (e.g., from Planetary Computer).

  • mean (int or float) – Mean value.

  • stddev (int or float) – Standard deviation value.

  • output_path (string (optional)) – Path to an output file, directory, or STAC dataset. If not set, then will overwrite the input files. Must be set if input is a remote STAC Item.

  • nodata (int or float (default = 0)) – Nodata value to be used in output data. If not set, then is read from inputs.

  • write_stac (bool (default = True)) – If True, then output metadata is saved to a STAC file.

Returns:

Path where output raster is saved.

Return type:

pathlib.Path

Examples

>>> import remote_sensing_processor as rsp
>>> rsp.denormalize.z_score(
...     input_path="/home/rsp_test/mosaics/sentinel/B1_norm.tif",
...     mean=302,
...     stddev=173,
...     output_path="/home/rsp_test/mosaics/sentinel/B1_orig.tif",
... )
'/home/rsp_test/mosaics/sentinel/B1_orig.json'
remote_sensing_processor.denormalize.dynamic_world(input_path, percentile1, percentile2, output_path=None, nodata=0, write_stac=True)[source]

Recovers original values from dynamic world normalized raster.

Parameters:

  • input_path (string or STAC Item) – Path to input file, directory or STAC dataset or a STAC Item (e.g., from Planetary Computer).

  • percentile1 (int or float) – First log-transformed data percentile.

  • percentile2 (int or float) – Second log-transformed data percentile.

  • output_path (string (optional)) – Path to an output file, directory, or STAC dataset. If not set, then will overwrite the input files. Must be set if input is a remote STAC Item.

  • nodata (int or float (default = 0)) – Nodata value to be used in output data. If not set, then is read from inputs.

  • write_stac (bool (default = True)) – If True, then output metadata is saved to a STAC file.

Returns:

Path where output raster is saved.

Return type:

pathlib.Path

Examples

>>> import remote_sensing_processor as rsp
>>> rsp.denormalize.dynamic_world(
...     input_path="/home/rsp_test/mosaics/sentinel/B1_norm.tif",
...     percentile1=5.49,
...     percentile2=5.78,
...     output_path="/home/rsp_test/mosaics/sentinel/B1_orig.tif",
... )
'/home/rsp_test/mosaics/sentinel/B1_orig.json'

Get normalization parameters.

remote_sensing_processor.get_normalization_params.min_max(inputs, nodata=None)[source]

Calculates optimal parameters for min-max normalization.

Parameters:

  • inputs (string or list of strings or STAC Item or list of STAC Items) – Paths to input files, directories or STAC datasets or STAC Items (e.g., from Planetary Computer).

  • nodata (int or float (optional)) – Nodata value. If not set, then is read from inputs.

Returns:

  • minimum (int or float) – Minimum value found in input datasets.

  • maximum (int or float) – Minimum value found in input datasets.

Examples

>>> import remote_sensing_processor as rsp
>>> rsp.get_normalization_params.min_max(
...     "/home/rsp_test/mosaics/sentinel/B1.tif",
... )
(0, 9849)
remote_sensing_processor.get_normalization_params.z_score(inputs, nodata=None)[source]

Calculates optimal parameters for z-score normalization.

Parameters:

  • inputs (string or list of strings or STAC Item or list of STAC Items) – Paths to input files, directories or STAC datasets or STAC Items (e.g., from Planetary Computer).

  • nodata (int or float (optional)) – Nodata value. If not set, then is read from inputs.

Returns:

  • mean (int or float) – Mean value for input datasets.

  • stddev (int or float) – Mean standard deviation value for input datasets.

Examples

>>> import remote_sensing_processor as rsp
>>> rsp.get_normalization_params.z_score(
...     "/home/rsp_test/mosaics/sentinel/B1.tif",
... )
(302, 173)
remote_sensing_processor.get_normalization_params.percentile(inputs, percentiles, nodata=None)[source]

Calculates percentiles for normalization.

Parameters:

  • inputs (string or list of strings or STAC Item or list of STAC Items) – Paths to input files, directories or STAC datasets or STAC Items (e.g., from Planetary Computer).

  • percentiles (list of ints) – Percentiles to be computed. Must be in the 0-100 range.

  • nodata (int or float (optional)) – Nodata value. If not set, then is read from inputs.

Returns:

percentiles – Dict where keys are percentiles and values are corresponding percentile values.

Return type:

dict of ints, floats

Examples

>>> import remote_sensing_processor as rsp
>>> rsp.get_normalization_params.percentile(
...     "/home/rsp_test/mosaics/sentinel/B1.tif",
...     [30, 70],
... )
{30: 129.1, 70: 475.6}
remote_sensing_processor.get_normalization_params.dynamic_world(inputs, percentiles, nodata=None)[source]

Calculates optimal parameters for dynamic world normalization.

Parameters:

  • inputs (string or list of strings or STAC Item or list of STAC Items) – Paths to input files, directories or STAC datasets or STAC Items (e.g., from Planetary Computer).

  • percentiles (list of ints) – Percentiles to be computed. Must be in the 0-100 range.

  • nodata (int or float (optional)) – Nodata value. If not set, then is read from inputs.

Returns:

percentiles – Dict where keys are percentiles and values are corresponding percentile values.

Return type:

dict of ints, floats

Examples

>>> import remote_sensing_processor as rsp
>>> rsp.get_normalization_params.dynamic_world(
...     "/home/rsp_test/mosaics/sentinel/B1.tif",
...     [30, 70],
... )
{30: 5.49, 70: 5.78}