Sm Feature Store Introduction

Amazon SageMaker Feature Store: Introduction to Feature Store

This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.

This notebook demonstrates how to get started with Feature Store, create feature groups, and ingest data into them. These feature groups are stored in your Feature Store.

Feature groups are resources that contain metadata for all data stored in your Feature Store. A feature group is a logical grouping of features, defined in the feature store to describe records. A feature group’s definition is composed of a list of feature definitions, a record identifier name, and configurations for its online and offline store.

Overview

1. Set up
2. Creating a feature group
3. Ingest data into a feature group

Prerequisites

This notebook uses both boto3 and Python SDK libraries, and the Python 3 (Data Science) kernel. This notebook works with Studio, Jupyter, and JupyterLab.

Library dependencies:

• sagemaker>=2.100.0
• numpy
• pandas

Role requirements:

IMPORTANT: You must attach the following policies to your execution role:

• AmazonS3FullAccess
• AmazonSageMakerFeatureStoreAccess

Set up

Inspect your data

In this notebook example we ingest synthetic data. We read from ./data/feature_store_introduction_customer.csv and ./data/feature_store_introduction_orders.csv.

Below is an illustration on the steps the data goes through before it is ingested into a Feature Store. In this notebook, we illustrate the use-case where you have data from multiple sources and want to store them independently in a feature store. Our example considers data from a data warehouse (customer data), and data from a real-time streaming service (order data).

Create a feature group

We first start by creating feature group names for customer_data and orders_data. Following this, we create two Feature Groups, one for customer_data and another for orders_data

Instantiate a FeatureGroup object for customers_data and orders_data.

Append EventTime feature to your data frame. This parameter is required, and time stamps each data point.

Load feature definitions to your feature group.

Below we call create to create two feature groups, customers_feature_group and orders_feature_group respectively

To confirm that your FeatureGroup has been created we use DescribeFeatureGroup and ListFeatureGroups APIs to display the created FeatureGroup.

Add metadata to a feature

We can add searchable metadata fields to FeatureGroup features by using the UpdateFeatureMetadata API. The currently supported metadata fields are description and parameters.

To view feature metadata, we can use DescribeFeatureMetadata to display that feature.

Feature metadata fields are searchable. We use search API to find features with metadata that matches some search criteria.

Ingest data into a feature group

We can put data into the FeatureGroup by using the PutRecord API. It will take < 1 minute to ingest data.

Using an arbitrary customer record ID, 573291 we use get_record to check that the data has been ingested into the feature group.

We use batch_get_record to check that all data has been ingested into two feature groups by providing customer IDs.

Add features to a feature group

If we want to update a FeatureGroup that has done the data ingestion, we can use the UpdateFeatureGroup API and then re-ingest data by using the updated dataset.

Verify the FeatureGroup has been updated successfully or not.

Inspect the new dataset.

Append EventTime feature to your data frame again.

Ingest the new dataset.

Use batch_get_record again to check that all updated data has been ingested into customers_feature_group by providing customer IDs.

Clean up

Here we remove the Feature Groups we created.

Next steps

In this notebook you learned how to quickly get started with Feature Store and now know how to create feature groups, and ingest data into them.

For an advanced example on how to use Feature Store for a Fraud Detection use-case, see Fraud Detection with Feature Store.

For detailed information about Feature Store, see the Developer Guide.

Programmers note

In this notebook we used a variety of different API calls. Most of them are accessible through the Python SDK, however some only exist within boto3. You can invoke the Python SDK API calls directly on your Feature Store objects, whereas to invoke API calls that exist within boto3, you must first access a boto client through your boto and sagemaker sessions: e.g. sagemaker_session.boto_session.client().

Below we list API calls used in this notebook that exist within the Python SDK and ones that exist in boto3 for your reference.

Python SDK API Calls

• describe()
• ingest()
• delete()
• create()
• load_feature_definitions()
• update()
• update_feature_metadata()
• describe_feature_metadata()

Boto3 API Calls

• list_feature_groups()
• get_record()
• batch_get_record()
• search()

Notebook CI Test Results

This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.