🧹 Weights & Biases Sweep + ‍🌊 TensorFlow 2.x

Use Weights & Biases for machine learning experiment tracking, dataset versioning, and project collaboration.

Use Weights & Biases Sweeps to automate hyperparameter optimization and explore the space of possible models, complete with interactive dashboards like this:

🤔 Why Should I Use Sweeps?

• Quick setup: With just a few lines of code you can run W&B sweeps.
• Transparent: We cite all the algorithms we're using, and our code is open source.
• Powerful: Our sweeps are completely customizable and configurable. You can launch a sweep across dozens of machines, and it's just as easy as starting a sweep on your laptop.

Check out the official documentation $\rightarrow$

What this notebook covers

• Simple steps to get started with W&B Sweep with custom training loop in TensorFlow.
• We will find best hyperparameters for our image classification task.

Note: Sections starting with Step are all you need to perform hyperparameter sweep in existing code. The rest of the code is there to set up a simple example.

🚀 Install, Import, and Log in

Step 0️⃣: Install W&B

Step 1️⃣: Import W&B and Login

Side note: If this is your first time using W&B or you are not logged in, the link that appears after running wandb.login() will take you to sign-up/login page. Signing up is as easy as a few clicks.

👩‍🍳 Prepare Dataset

🧠 Define the Model and Training Loop

🏗️ Build a Simple Classifier MLP

🔁 Write a Training Loop

Step 3️⃣: Log metrics with wandb.log

Step 4️⃣: Configure the Sweep

This is where you will:

• Define the hyperparameters you're sweeping over
• Provide your hyperparameter optimization method. We have random, grid and bayes methods.
• Provide an objective and a metric if using bayes, for example to minimize the val_loss.
• Use hyperband for early termination of poorly-performing runs

Check out more on Sweep Configs $\rightarrow$

Step 5️⃣: Wrap the Training Loop

You'll need a function, like sweep_train below, that uses wandb.config to set the hyperparameters before train gets called.

Step 6️⃣: Initialize Sweep and Run Agent

You can limit the number of total runs with the count parameter, we will limit a 10 to make the script run fast, feel free to increase the number of runs and see what happens.

👀 Visualize Results

Click on the Sweep URL link above to see your live results.

🎨 Example Gallery

See examples of projects tracked and visualized with W&B in our Gallery →

📏 Best Practices

1. Projects: Log multiple runs to a project to compare them. wandb.init(project="project-name")
2. Groups: For multiple processes or cross validation folds, log each process as a runs and group them together. wandb.init(group='experiment-1')
3. Tags: Add tags to track your current baseline or production model.
4. Notes: Type notes in the table to track the changes between runs.
5. Reports: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects.

🤓 Advanced Setup

1. Environment variables: Set API keys in environment variables so you can run training on a managed cluster.
2. Offline mode: Use dryrun mode to train offline and sync results later.
3. On-prem: Install W&B in a private cloud or air-gapped servers in your own infrastructure. We have local installations for everyone from academics to enterprise teams.