Safe lab preview

Classification of Oxford Pets using Transfer Learning

This is a sanitized, read-only preview. Nothing executes in this page.

Classification of Oxford Pets using Transfer Learning

Use this guided lab to apply the lesson concepts.

Task

Imagine you need to develop an application for pet nursery to catalog all pets. One of the great features of such an application would be automatically discovering the breed from a photograph. In this assignment, we will use transfer learning to classify real-life pet images from Oxford-IIIT pets dataset.

The Dataset

We will use the original Oxford-IIIT pets dataset, which contains 35 different breeds of dogs and cats.

To download the dataset, use this code snippet:

Read-only

Notebook preview

Oxford Pets

Download notebook

Markdown cell[1]

> **Integrated runtime note:** This preview uses a small deterministic, rights-safe offline fixture for reproducible learning. Full-scale results require the lesson's documented dataset or model in an approved external environment. # Classification of Pet's Real-Life Images Lab Assignment from AI for Beginners Curriculum. Now it's time to deal with more challenging task - classification of the original [Oxford-IIIT Dataset](https://www.robots.ox.ac.uk/~vgg/data/pets/). Let's start by loading and visualizing the dataset.

Code cell[2]

# Generated pet images for deterministic, rights-safe execution.
from pathlib import Path
from PIL import Image
import numpy as np

image_root = Path("images")
image_root.mkdir(parents=True, exist_ok=True)
for breed_index, breed in enumerate(("Abyssinian", "Maine_Coon", "beagle", "pug")):
    for image_index in range(1, 4):
        yy, xx = np.mgrid[:64, :64]
        pixels = np.zeros((64, 64, 3), dtype=np.uint8)
        pixels[..., 0] = (xx * (2 + breed_index) + 23 * image_index) % 256
        pixels[..., 1] = (yy * (3 + image_index) + 31 * breed_index) % 256
        pixels[..., 2] = ((xx + yy) * 2 + 47 * breed_index) % 256
        Image.fromarray(pixels).save(image_root / f"{breed}_{image_index}.jpg")
print("Prepared 12 generated pet images")

Markdown cell[3]

We will define generic function to display a series of images from a list:

Code cell[4]

import matplotlib.pyplot as plt
import os
from PIL import Image
import numpy as np

def display_images(images, titles=None, fontsize=12):
    images = list(images)[:24]
    if not images:
        return
    columns = min(len(images), 6)
    rows = (len(images) + columns - 1) // columns
    fig, axes = plt.subplots(rows, columns, squeeze=False, figsize=(columns * 2.5, rows * 2.5))
    flat_axes = axes.reshape(-1)
    for index, image in enumerate(images):
        flat_axes[index].imshow(image)
        flat_axes[index].axis("off")
        if titles is not None and index < len(titles):
            flat_axes[index].set_title(titles[index], fontsize=fontsize)
    for axis in flat_axes[len(images):]:
        axis.axis("off")
    plt.tight_layout()
    plt.show()

Markdown cell[5]

You can see that all images are located in one directory called `images`, and their name contains the name of the class (breed):

Code cell[6]

fnames = os.listdir('images')[:5]
display_images([Image.open(os.path.join('images',x)) for x in fnames],titles=fnames,fontsize=30)

Markdown cell[7]

To simplify classification and use the same approach to loading images as in the previous part, let's sort all images into corresponding directories:

Code cell[8]

for fn in os.listdir('images'):
    cls = fn[:fn.rfind('_')].lower()
    os.makedirs(os.path.join('images',cls),exist_ok=True)
    os.replace(os.path.join('images',fn),os.path.join('images',cls,fn))

Markdown cell[9]

Let's also define the number of classes in our dataset:

Code cell[10]

num_classes = len(os.listdir('images'))
num_classes

Markdown cell[11]

## Preparing dataset for Deep Learning To start training our neural network, we need to convert all images to tensors, and also create tensors corresponding to labels (class numbers). Most neural network frameworks contain simple tools for dealing with images: * In Tensorflow, use `keras.utils.image_dataset_from_directory` * In PyTorch, use `torchvision.datasets.ImageFolder` As you have seen from the pictures above, all of them are close to square image ratio, so we need to resize all images to square size. Also, we can organize images in minibatches.

Code cell[12]

# PREPARE THE DATASET

Markdown cell[13]

Now we need to separate dataset into train and test portions:

Code cell[14]

# SPLIT INTO TRAIN-TEST DATASETS

Markdown cell[15]

Now define data loaders:

Code cell[16]

# DEFINE DATA LOADERS if needed

Code cell[17]

# [OPTIONAL] Plot the dataset

Markdown cell[18]

## Define a neural network For image classification, you should probably define a convolutional neural network with several layers. What to keep an eye for: * Keep in mind the pyramid architecture, i.e. number of filters should increase as you go deeper * Do not forget activation functions between layers (ReLU) and Max Pooling * Final classifier can be with or without hidden layers, but the number of output neurons should be equal to number of classes. An important thing is to get the activation function on the last layer + loss function right: * In Tensorflow, you can use `softmax` as the activation, and `sparse_categorical_crossentropy` as loss. The difference between sparse categorical cross-entropy and non-sparse one is that the former expects output as the number of class, and not as one-hot vector. * In PyTorch, you can have the final layer without activation function, and use `CrossEntropyLoss` loss function. This function applies softmax automatically. > **Hint:** In PyTorch, you can use `LazyLinear` layer instead of `Linear`, in order to avoid computing the number of inputs. It only requires one `n_out` parameter, which is number of neurons in the layer, and the dimension of input data is picked up automatically upon first `forward` pass.

Code cell[19]

# DEFINE NEURAL NETWORK ARCHITECTURE

Markdown cell[20]

## Train the Neural Network Now we are ready to train the neural network. During training, please collect accuracy on train and test data on each epoch, and then plot the accuracy to see if there is overfitting.

Code cell[21]

# TRAIN THE NEURAL NETWORK

Code cell[22]

# PLOT THE RESULT: Train and Test Accuracy

Markdown cell[23]

Even if you have done everything correctly, you will probably see that the accuracy is quite low. ## Transfer Learning To improve the accuracy, let's use pre-trained neural network as feature extractor. Feel free to experiment with VGG-16/VGG-19 models, ResNet50, etc. > Since this training is slower, you may start with training the model for the small number of epochs, eg. 3. You can always resume training to further improve accuracy if needed. We need to normalize our data differently for transfer learning, thus we will reload the dataset again using different set of transforms:

Code cell[24]

# LOAD THE DATASET
# Perform standard transformations for VGG-16/VGG-19 if needed

Markdown cell[25]

Let's load the pre-trained network:

Code cell[26]

# vgg = ...

Markdown cell[27]

Now define the classification model for your problem: * In PyTorch, there is a slot called `classifier`, which you can replace with your own classifier for the desired number of classes. * In TensorFlow, use VGG network as feature extractor, and build a `Sequential` model with VGG as first layer, and your own classifier on top

Code cell[28]

# BUILD MODEL for your problem with your own linear layers

Markdown cell[29]

Make sure to set all parameters of VGG feature extractor not to be trainable

Code cell[30]

# MAKE VGG Layers not trainable

Markdown cell[31]

Now we can start the training. Be very patient, as training takes a long time, and our train function is not designed to print anything before the end of the epoch.

Code cell[32]

# TRAIN THE MODEL

Markdown cell[33]

It seems much better now! ## Optional: Calculate Top 3 Accuracy We can also computer Top 3 accuracy using the same code as in the previous exercise.

Code cell[34]

# CALCULATE TOP-3 Accuracy of the model

Outputs, execution counts, widgets, and active content were removed during import. Run notebooks only in an external environment you trust.

Record your practice

Optional self-reporting helps you remember what you practiced and never gates course completion.

Back to lesson

Task

# Generated pet images for deterministic, rights-safe execution. from pathlib import Path from PIL import Image import numpy as np image_root = Path("images") image_root.mkdir(parents=True, exist_ok=True) for breed_index, breed in enumerate(("Abyssinian", "Maine_Coon", "beagle", "pug")): for image_index in range(1, 4): yy, xx = np.mgrid[:64, :64] pixels = np.zeros((64, 64, 3), dtype=np.uint8) pixels[..., 0] = (xx * (2 + breed_index) + 23 * image_index) % 256 pixels[..., 1] = (yy * (3 + image_index) + 31 * breed_index) % 256 pixels[..., 2] = ((xx + yy) * 2 + 47 * breed_index) % 256 Image.fromarray(pixels).save(image_root / f"{breed}_{image_index}.jpg") print("Prepared 12 generated pet images")

import matplotlib.pyplot as plt import os from PIL import Image import numpy as np def display_images(images, titles=None, fontsize=12): images = list(images)[:24] if not images: return columns = min(len(images), 6) rows = (len(images) + columns - 1) // columns fig, axes = plt.subplots(rows, columns, squeeze=False, figsize=(columns * 2.5, rows * 2.5)) flat_axes = axes.reshape(-1) for index, image in enumerate(images): flat_axes[index].imshow(image) flat_axes[index].axis("off") if titles is not None and index < len(titles): flat_axes[index].set_title(titles[index], fontsize=fontsize) for axis in flat_axes[len(images):]: axis.axis("off") plt.tight_layout() plt.show()

Classification of Oxford Pets using Transfer Learning

Classification of Oxford Pets using Transfer Learning

Task

The Dataset

Notebook preview

Oxford Pets

Record your practice

Classification of Oxford Pets using Transfer Learning

Classification of Oxford Pets using Transfer Learning

Task

The Dataset

Notebook preview

Oxford Pets

Record your practice

Start with the notebook

Takeaway

Lab instructions

Classification of Oxford Pets using Transfer Learning

Task

The Dataset

Notebook preview

Oxford Pets

Record your practice

Lab instructions

Classification of Oxford Pets using Transfer Learning

Task

The Dataset

Notebook preview

Oxford Pets

Record your practice

Start with the notebook

Takeaway