🤗 x 🦾: Training ACT with LeRobot Notebook

Welcome to the LeRobot ACT training notebook! This notebook provides a ready-to-run setup for training imitation learning policies using the 🤗 LeRobot library.

In this example, we train an ACT policy using a dataset hosted on the Hugging Face Hub, and optionally track training metrics with Weights & Biases (wandb).

⚙️ Requirements

• A Hugging Face dataset repo ID containing your training data (--dataset.repo_id=YOUR_USERNAME/YOUR_DATASET)
• Optional: A wandb account if you want to enable training visualization
• Recommended: GPU runtime (e.g., NVIDIA A100) for faster training

⏱️ Expected Training Time

Training with the ACT policy for 100,000 steps typically takes about 1.5 hours on an NVIDIA A100 GPU. On less powerful GPUs or CPUs, training may take significantly longer.

Example Output

Model checkpoints, logs, and training plots will be saved to the specified --output_dir. If wandb is enabled, progress will also be visualized in your wandb project dashboard.

Install conda

This cell uses condacolab to bootstrap a full Conda environment inside Google Colab.

Install LeRobot

This cell clones the lerobot repository from Hugging Face, installs FFmpeg (version 7.1.1), and installs the package in editable mode.

Weights & Biases login

This cell logs you into Weights & Biases (wandb) to enable experiment tracking and logging.

Start training ACT with LeRobot

This cell runs the train.py script from the lerobot library to train a robot control policy.

Make sure to adjust the following arguments to your setup:

1. --dataset.repo_id=YOUR_HF_USERNAME/YOUR_DATASET:
   Replace this with the Hugging Face Hub repo ID where your dataset is stored, e.g., pepijn223/il_gym0.

2. --policy.type=act:
   Specifies the policy configuration to use. act refers to configuration_act.py, which will automatically adapt to your dataset’s setup (e.g., number of motors and cameras).

3. --output_dir=outputs/train/...:
   Directory where training logs and model checkpoints will be saved.

4. --job_name=...:
   A name for this training job, used for logging and Weights & Biases.

5. --policy.device=cuda:
   Use cuda if training on an NVIDIA GPU. Use mps for Apple Silicon, or cpu if no GPU is available.

6. --wandb.enable=true:
   Enables Weights & Biases for visualizing training progress. You must be logged in via wandb login before running this. Set to False if you do not plan on using Weights & Biases.

Login into Hugging Face Hub.

Now after training is done login into the Hugging Face hub and upload the last checkpoint.

Configure Hugging Face Token

Add your HF_TOKEN (AKA Secret) to Google Colab to enable Colab to access your HF repositories. This is optional and might need modification if you are using another cloud provider.

Download files from Hugging Face Hub into Colab

Now, you can use hf_hub_download to pull the files directly to your Colab environment. You can then download them from Colab to your local machine. This is optional.

Verify train_config.json existence locally. This is needed for restarting training and testing of the policy.

Create a new repository on Hugging Face Hub

This command will create a new repository under your Hugging Face account.