Huggingface Sagemaker - Vision Transformer

Image Classification with the google/vit on cifar10

1. Introduction
2. Development Environment and Permissions
   1. Installation
   2. Permissions
3. Processing
   1. convert features and transform images
   2. Uploading data to sagemaker_session_bucket
4. Fine-tuning & starting Sagemaker Training Job
   1. Creating an Estimator and start a training job

Introduction

Welcome to our end-to-end binary Image-Classification example. In this demo, we will use the Hugging Faces transformers and datasets library together with Amazon SageMaker to fine-tune a pre-trained vision transformers on image classification.

The script and notebook is inspired by NielsRogges example notebook of Fine-tune the Vision Transformer on CIFAR-10. Niels was also the contributor of the Vision Transformer into transformers.

NOTE: You can run this demo in Sagemaker Studio, your local machine or Sagemaker Notebook Instances

Development Environment and Permissions

Use at least a t3.large instance otherwise preprocessing will take ages.

Installation

Note: we only install the required libraries from Hugging Face and AWS. You also need PyTorch or Tensorflow, if not already installed

Permissions

If you are going to use Sagemaker in a local environment, you need access to an IAM Role with the required permissions for Sagemaker. You can find out more about this here

Preprocessing

We are using the datasets library to download and preprocess the fashion-mnist dataset. After preprocessing, the dataset will be uploaded to our sagemaker_session_bucket to be used within our training job. The cifar10 are labeled subsets of the 80 million tiny images dataset. They were collected by Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.

Note from Nils: "that in the ViT paper, the best results were obtained when fine-tuning at a higher resolution. For this, one interpolates the pre-trained absolute position embeddings".

Convert Features and transform images

We are downsampling dataset to make it faster to preprocess.

Uploading data to sagemaker_session_bucket

After we processed the datasets we are going to use the new FileSystem integration to upload our dataset to S3.

num_train_epochs# Fine-tuning & starting Sagemaker Training Job

In order to create a sagemaker training job we need a HuggingFace Estimator. The Estimator handles end-to-end Amazon SageMaker training and deployment tasks. In an Estimator, we define which fine-tuning script should be used as entry_point, which instance_type should be used, which hyperparameters are passed in .....

/opt/conda/bin/python train.py --num_train_epochs 1 --model_name google/vit-base-patch16-224-in21k --per_device_train_batch_size 16

Creating an Estimator and start a training job

We are defining the hyperparameter use_auth_token with our token from huggingface.co/settings to automatically upload our model to the Hugging Face Model Hub afterwards. The train.py makes us of the .push_to_hub() of the Trainer API to automatically upload model to hf.co/models.

Upload to hub

Since we have done the preprocessing in advance we need to upload the image_processor separately. You can this by creating a preprocessor_config.json file in the UI on huggingface.co or using the huggingface_hub python library.

The file needs to contain the configuration of the image_processor