Segmenting aerial imagery using geospatial GPU notebook

This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.

Introduction

This notebook demonstrates how you can use the geospatial kernel in combination with open-source geospatial libraries to perform prompt-based segmentation on satellite or aerial imagery. This example uses data from National Agriculture Imagery Program (NAIP). The National Agriculture Imagery Program acquires aerial imagery during the agricultural growing seasons in the continental United States.

Prerequisites

This notebook runs with the Geospatial 1.0 kernel with a ml.g5.xlarge instance. Note that the following policies need to be attached to the execution role that you used to run this notebook:

• AmazonSageMakerFullAccess
• AmazonSageMakerGeospatialFullAccess

You can see the policies attached to the role in the IAM console under the permissions tab. If required, add the roles using the 'Add Permissions' button.

In addition to these policies, ensure that the execution role's trust policy allows the SageMaker-GeoSpatial service to assume the role. This can be done by adding the following trust policy using the 'Trust relationships' tab:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "sagemaker.amazonaws.com",
                    "sagemaker-geospatial.amazonaws.com"
                ]
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

GPU Support

When you're running this notebook on a ml.g5.xlarge instance, you have access to a NVIDIA A10G GPU. The NVIDIA System Management Interface (nvidia-smi) is a command line utility, which monitors and manages NVIDIA GPUs. You can run the following command to get meaningful insights about the instance's GPU.

The Geospatial 1.0 kernel comes preinstalled with PyTorch and Tensorflow, and you can check the availability of the GPU also via the following steps.

Setup SageMaker geospatial capabitilies and install additional dependencies

In this example, you'll use the segment-geospatial library to perform the segmentation. This library is combining Segment Anything with Grounding DINO to detect and segment based on text inputs. The following steps are installing the additional dependencies and setting up the necessary imports.

Get NAIP example data

To query NAIP aerial imagery, you can use the search_raster_data_collection API, which allows to run spatial-temporal queries against various raster data collections.

The following code example uses the ARN associated with the NAIP data collection, arn:aws:sagemaker-geospatial:us-west-2:378778860802:raster-data-collection/public/37ndema229vwa987.

After performing the query, we obtained two scenes which matched the search criteria. We can inspect the results in the following cell.

Download and visualize example scene

We will download the image asset of the first scene to use it as input for the segmentation model. To prepare the input for the inference, we'll clip the scene to a particular area of interest and visualize the prepared input.

Perform segmentation on spatial data

Initialize Segmentation model

Run inference & visualize segmentation mask

After the segmentation model has been initialized, we'll perform the inference by running the predict method.

The model prediction step includes setting appropriate thresholds for object detection and associating text with the identified objects. These threshold values, which range from 0 to 1, are specified when invoking the predict method of the LangSAM class.

box_threshold: This parameter is used for object detection in the image. A higher box threshold causes the model to be more selective, identifying only object instances with the highest confidence. This selectivity may lead to a reduction in the total number of detections. On the other hand, a lower box threshold renders the model more permissive, resulting in a greater number of detections, including those of potentially lower confidence.

text_threshold: A higher text threshold demands a stronger correlation between the object and the text prompt, which can lead to more precise but fewer associations. Conversely, a lower text threshold facilitates looser associations, possibly increasing the number of associations but at the risk of including less accurate matches.

We can also show the annotations as a binary mask.

As a final step, we'll convert the binary mask to a vector format, such as shapefile. You can also use any other vector format supported by geopandas, such as GeoJSON and GeoPackage. In the following cells, we extract the mask as shapefile and visualize it on an interactive map.

Notebook CI Test Results

This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.