In this colab, we'll see an example of adding dropout to a PyTorch model and observe the effect dropout has on the model's performance by tracking our models in Weights & Biases.

You can read more about using dropout in PyTorch here.

Setup

Download dataset and prepare DataLoaders

Visualize data

Define training

Define testing

Training the unregularized model

Initialize Model, Loss and Optimizer

Train

Training a model with dropout regularization

Initialize Model, Loss and Optimizer

Train

Visualize and debug model pipelines with W&B

Think of W&B like GitHub for machine learning models — save everything you need to debug, compare and reproduce your models — architecture, hyperparameters, weights, model predictions, GPU usage, git commits, and even datasets — with a few lines of code.

W&B lightweight integrations work with any Python script, and all you need to do is sign up for a free W&B account to start tracking and visualizing your models.

Used by the likes of OpenAI, Lyft, Github and researchers at top machine learning labs across the world, W&B is part of the new standard of best practices for machine learning.

How W&B can help you optimize your machine learning workflows:

• Debug model performance in real time
• Automatically tracked GPU, CPU usage and other system metrics
• Powerful custom charts
• Share model insights interactively
• Efficient hyperparameter optimization
• Dataset and model pipeline tracking and production model management

W&B is free for individuals, academics and open source projects.