Classification with PyTorch/TensorFlow

Lab Assignment from AI for Beginners Curriculum.

Part 1: Iris Classification

Iris Dataset contains 150 records of 3 different classes of irises. Each record contains 4 numeric parameters: sepal length/width and petal length/width. It is an example of a simple dataset, for which you do not need a powerful neural network.

Getting the Dataset

Iris dataset is build into Scikit Learn, so we can easily get it:

Visualize the Data

In many cases, it makes sense to visualize the data to see if they look separable - it would assure us that we should be able to build good classification model. Because we have a few features, we can build a series of pairwise 2D scatter plots, showing different classes by different dot colors. This can be automatically done by a package called seaborn:

Normalize and Encode the Data

To prepare data to neural network training, we need to normalize inputs in the range [0..1]. This can be done either using plain numpy operations, or Scikit Learn methods.

Also, you need to decide if you want target label to be one-hot encoded or not. PyTorch and TensorFlow allow you feed in class number either as an integer (from 0 to N-1), or as one-hot encoded vector. When creating neural network structure, you need to specify loss function accordingly (eg. sparse categorical crossentropy for numeric representation, and crossentropy loss for one-hot encoding). One-hot encoding can also be done using Sklearn, or by using this piece of code:

n_values = np.max(labels) + 1
labels_onehot = np.eye(n_values)[labels]

Split the Data into Train and Test

Since we do not have separate train and test dataset, we need to split it intro train and test dataset using Sklearn

Define and Train Neural Network

Now you are ready to go, import your preferred framework, define the neural network and start training, observing the behavior of train and validation accuracy.

Experiment

Now you can experiment with different network architectures to see how it affects the result. Try:

1. One-layer network with 3 neurons (equal to the number of classes)
2. Two-layer network with small/medium/large hidden layer
3. Using more layers

Make sure you observe overfitting when you are using rich model with lots of neurons (parameters).

Part 2: MNIST Training

Both Keras and PyTorch contain MNIST as built-in dataset, so you can easily get it with a couple of lines of code (Keras, PyTorch). You will also be able to load both train and test datasets without manually splitting them.

Now you need to perform the steps above to make sure dataset is normalized (it would probably already be), defining and training a neural network.

Takeaway

1. Neural networks can be used for traditional machine learning tasks. However, they are in many cases too powerful, and can cause overfitting.
2. It is important in this assignment that you observe the overfitting behavior, and try to avoid it.
3. With frameworks like Keras, sometimes training a neural network is quite straightforward. But you need to understand what goes on.