Digital Farming with Amazon SageMaker Geospatial Capabilities - Part I

This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.

In this notebook, we will explore some of most common tasks for processing geospatial data in the Digital Farming domain, by working with Amazon SageMaker geospatial capabilities.

Environment Set-Up

We will start by importing a few libraries required, the necessary dependencies are already pre-installed in the Geospatial kernel.

We will now define a few variables for which we need to create sessions in the SageMaker and Boto3 SDKs.

We will also create the client for SageMaker geospatial capabilities by creating a Botocore session...

Note, at the time of writting this notebook sagemaker-geospatial is only available in the region "us-west-2".

Make sure you have the proper policy allowed and a trust relationship added to your role for "sagemaker-geospatial", as specified in the Get Started with Amazon SageMaker Geospatial Capabilties documentation.

Common geospatial processing tasks for Digital Farming

Now that we have the client setup, we are ready to work with SageMaker Geospatial.

We can in example check the raster collections available...

We will save the ARN of our collection of interest. In our example we will work with satellite imagery data from the Sentinel-2-L2A collection...

First, we will define the input configuration with the polygon of coordinates for our area of interest and the time range we are interested on.

We could graphically visualize this area by using the SageMaker Studio Geospatial extension, but in our case we will use an open-source library for programatically embedding a map in the notebook.

Typically, we are interested on working with images that are not covered by much clouds over our area of interest. For exploring this in our notebook, we will define some additional parameters like e.g. the ranges for cloud cover we want to consider (less than 2% in our example).

We can now search for satellite images of the area of interest and visualize, as an example, the True Color Image (TCI) tiles provided by SageMaker, or any other band of interest.

Note these images are directly provided by the built-in data sources in SageMaker geospatial capabilities, like e.g. AWS Data Exchange in our case, and the query will return the locations of these image tiles in Amazon S3.

Cloud Gap Filling - Earth Observation Job

Currently, SageMaker Geospatial supports different types of built-in processing Earth Observation Jobs (EOJs), such as:

• Cloud removal
• Temporal or zonal statistics
• Stacking of bands
• Geo-Mosaic combinations
• Re-sampling
• Segmentation with built-in models

etc.

Let us explore some of these techniques with an example.

In some cases we might be interested on removing the cloudy pixels of the images, this could be e.g. if we don't have any clear observation available in the data for our area and time-range of interest. We can do this through a Cloud Removal EOJ, which would replace or fill these cloudy pixels using any of the techniques available.

For this we will setup the cloud removal parameters. This can e.g. be set to a fixed value to run your own gap filling algorithm/logic, or alternatively use a built-in interpolation type supported in the service.

Finally, we are ready to start the EOJ with the given config. We will define a function for this purpose that will be handy for the other tasks in this notebook.

Note the EOJ processing takes some minutes. We can check the status programatically by getting the EOJ with the SageMaker Geospatial client, or graphically by using the Geospatial extension for SageMaker Studio.

Once completed, note the resulting imagery is stored in a service bucket and it can be used for chaining this to another EOJ. Also, for visualizing these images we can, if desired, copy the results of the EOJ to our own S3 location in our account by exporting it.

Check the "Exporting the Results" section of this notebook for details.

Geo Mosaic - Earth Observation Job

A common process when working with tiles of Satellite images is combining these geographically, for having a consolidated view of the whole area of interest. We can perform this through the use of a Geo Mosaic job, applicable to the output of most of the examples provided with other processing EOJs in this notebook.

For this purpose we can set the configuration as follows.

Note the EOJ processing takes some minutes. We can check the status programatically by getting the EOJ with the SageMaker Geospatial client, or graphically by using the Geospatial extension for SageMaker Studio.

Band-Math - Earth Observation Job

Follwing our example, we will now perform a band-math over predefined vegetation and soil indices supported by SageMaker geospatial capabilities. Note SageMaker geospatial capabilities can perform the band-math over one or many of these indices, and supports the use of equations.

For our case let's obtain the moisture index by using the equation below.

Note the EOJ processing takes some minutes. We can check the status programatically by getting the EOJ with the SageMaker Geospatial client, or graphically by using the Geospatial extension for SageMaker Studio.

Exporting the Results

As mentioned before, the results of our EOJs are stored in the service and are available for chaining as input for another EOJ, but can also export these to Amazon S3 for visualizing the imagery directly.

We will define a function for exporting the results of our EOJs.

Let's go through the EOJs created before for checking it's status and exporting accordingly. Keep in mind each EOJ takes some minutes to complete, so we will add a check on the status every 30 seconds...

Now that we have all our EOJs exported, let's visualize a few of the images obtained in S3.

For this we will use the open library "rasterio".

For our Cloud Removal, we can check in example some of the images for the "blue" band

For the Geo Mosaic, let's visualize the consolidated image shown as "visual".

For the Band Math, let's visualize a few of the output images for the Moisture Index.

Here, take into account the legend for the resulting moisture index would be similar to the below.

You can continue with the folling notebook in this series for exploring other common geospatial processing tasks.

Notebook CI Test Results

This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.