{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.4"
},
"colab": {
"name": "keras_basics_notebook.ipynb",
"provenance": [],
"collapsed_sections": []
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "6chnwJxscdcb",
"colab_type": "text"
},
"source": [
"# CS568:Deep Learning Spring 2020 "
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true,
"id": "hr5Yd5bXcdc4",
"colab_type": "text"
},
"source": [
"In this notebook, we will see the basic building blocks of keras. \n",
"\n",
"1. **Load dataset** \n",
" + difference between **flow**, **flow_from_directory** and **flow_from_dataframe**\n",
" \n",
"2. **Define model**\n",
" + difference between **sequential** and **functional** models \n",
" \n",
"3. **Compile model**\n",
"\n",
"4. **Fit model**\n",
" + difference between **fit** and **fit_generator** functions\n",
" \n",
"5. **Evaluate model** \n",
" + difference between **evaluate** and **evaluate_generator** functions\n",
" \n",
"6. **Predict model**\n",
" + difference between **predict**, **predict_classes** and **predict_generator** functions"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "k27hmA9scddD",
"colab_type": "text"
},
"source": [
"## 1: Load Datasets"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "TRxu9dDvcddK",
"colab_type": "text"
},
"source": [
"Download and extract your dataset in a folder. Make sure your dataset folder has three subfolders train, val and test.\n",
"\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "mSBbV3EJcddS",
"colab_type": "code",
"colab": {}
},
"source": [
"train_data_dir = '/dataset/dogs-vs-cats/train/'\n",
"validation_data_dir = '/keras/dataset/dogs-vs-cats/val/'\n",
"test_dir = '/keras/dataset/dogs-vs-cats/test/'"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "R9w8-NMwcddw",
"colab_type": "text"
},
"source": [
"Now, import ImageDataGenerator"
]
},
{
"cell_type": "code",
"metadata": {
"id": "D5G5jetTcdd2",
"colab_type": "code",
"colab": {}
},
"source": [
"from keras.preprocessing.image import ImageDataGenerator"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "pcYjMXYGcdeM",
"colab_type": "text"
},
"source": [
"Keras ImageDataGenerator is used to \n",
"+ take a batch of images from disk or memory.\n",
"+ apply random transformations to each image in the batch.\n",
"+ replace the original batch of images with a new randomly transformed batch.\n",
"+ train a deep learning model on this transformed batch.\n",
"\n",
"For more information about ImageDataGenerator, see this [link](https://keras.io/preprocessing/image/). "
]
},
{
"cell_type": "code",
"metadata": {
"id": "5-OWfcTrcdeS",
"colab_type": "code",
"colab": {}
},
"source": [
"# create an instance of the ImageDataGenerator class\n",
"data_generator = ImageDataGenerator(\n",
" featurewise_center=False, \n",
" featurewise_std_normalization=False, \n",
" rotation_range=10,\n",
" width_shift_range=0.1,\n",
" height_shift_range=0.1,\n",
" zoom_range=.1,\n",
" horizontal_flip=True)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "SDG7cKwOcdel",
"colab_type": "text"
},
"source": [
"**Example of data augmentation**"
]
},
{
"cell_type": "code",
"metadata": {
"id": "Tzs3Ev3s3zKo",
"colab_type": "code",
"outputId": "7be17abd-c3fe-46b4-f719-9152c8e9572a",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 122
}
},
"source": [
"from google.colab import drive\n",
"drive.mount('/content/drive')"
],
"execution_count": 0,
"outputs": [
{
"output_type": "stream",
"text": [
"Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly\n",
"\n",
"Enter your authorization code:\n",
"··········\n",
"Mounted at /content/drive\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "KKvTqZArcdey",
"colab_type": "code",
"colab": {}
},
"source": [
"from keras.preprocessing.image import ImageDataGenerator\n",
"from keras.preprocessing.image import img_to_array, load_img\n",
"import numpy as np\n",
"\n",
"\n",
"src_path = '/content/drive/My Drive/CS568-DeepLearning-(Spring2020)/dataset/image/hinton.jpg'\n",
"des_path = '/content/drive/My Drive/CS568-DeepLearning-(Spring2020)/dataset/augmented_images/'\n",
"\n",
"\n",
"image = load_img(src_path)\n",
"image = img_to_array(image)\n",
"image = np.expand_dims(image, axis=0)\n",
"\n",
"\n",
"# define ImageDataGenerator class\n",
"data_augmentation = ImageDataGenerator(\n",
" rotation_range=90,\n",
" zoom_range=0.5,\n",
" width_shift_range=0.2,\n",
" height_shift_range=0.2,\n",
" shear_range=0.15,\n",
" horizontal_flip=True,\n",
" fill_mode=\"nearest\")\n",
"\n",
"# apply ImageDataGenerator to input image\n",
"img_generator = data_augmentation.flow(image, batch_size=1, save_to_dir=des_path, save_prefix=\"image\", save_format=\"jpg\")\n",
"\n",
"\n",
"nb_image = 10\n",
"count = 0\n",
"for e in img_generator:\n",
" if (count == nb_image):\n",
" break\n",
" count += 1\n",
" \n",
"# check augmented images in the des_path folder"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "v_hdp0W9cdfd",
"colab_type": "text"
},
"source": [
"Once constructed, an iterator can be created for an image dataset.\n",
"\n",
"Keras ImageDataGenerator class provides three different types of iterators to load the image dataset:\n",
"\n",
"+ **flow()**: an iterator that loads the complete image dataset in memory and generates batches of augmented images on each iteration.\n",
"+ **flow_from_directory()**: an iterator that loads a batch of images in memory from disk\n",
"+ **flow_from_dataframe()**: an iterator that loads a batch of images in memory from disk"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ElC-Qy4ccdfo",
"colab_type": "text"
},
"source": [
"The method **flow_from_directory()** assumes:\n",
"\n",
"+ the root directory contains **three (3) folders** for **train**, **val**.and **test**.\n",
"+ the train folder should contain **k (number of classes)** sub-directories each containing images of respective classes.\n",
"+ the test folder should contain a single folder, which stores all **test images**.\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "XMxrn-t9cdfx",
"colab_type": "code",
"outputId": "2cc718e7-a92e-46ea-c675-a72cbfe2ea2a",
"colab": {}
},
"source": [
"# define ImageDataGenerator class \n",
"train_datagen = ImageDataGenerator(\n",
" rescale=1 / 255.0,\n",
" rotation_range=20,\n",
" zoom_range=0.05,\n",
" width_shift_range=0.05,\n",
" height_shift_range=0.05,\n",
" shear_range=0.05,\n",
" horizontal_flip=True,\n",
" fill_mode=\"nearest\")\n",
"\n",
"# define ImageDataGenerator class for testing images\n",
"test_datagen = ImageDataGenerator(rescale=1 /255.0)\n",
"\n",
"# automatically retrieve images and their classes for train and validation sets\n",
"batch_size = 4\n",
"train_generator = train_datagen.flow_from_directory(\n",
" directory=train_path, # directory must be set to the path where your ‘k’ classes of folders are present.\n",
" target_size=(100, 100), # target_size is the size of your input images, every image will be resized to this size.\n",
" color_mode=\"rgb\", # grayscale or rgb\n",
" batch_size=batch_size, # batch size\n",
" class_mode=\"binary\", # Set “binary” for two classes, set “categorical” for k classes and for regression task set “input” \n",
" shuffle=True, # want to shuffle images or not\n",
" seed=42 # random seed for applying random image augmentation and shuffling the order of the image.\n",
")"
],
"execution_count": 0,
"outputs": [
{
"output_type": "stream",
"text": [
"Found 90 images belonging to 2 classes.\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "zSXn6HjEcdgc",
"colab_type": "code",
"colab": {}
},
"source": [
"# for k number of classes\n",
"train_generator = train_datagen.flow_from_directory(\n",
" directory=train_path, \n",
" target_size=(100, 100), \n",
" color_mode=\"rgb\", \n",
" batch_size=batch_size, \n",
" class_mode=\"categorical\",\n",
" shuffle=True, \n",
" seed=42 \n",
")"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "VHDfe7U0cdg7",
"colab_type": "code",
"colab": {}
},
"source": [
"# for specific classes\n",
"train_generator = train_datagen.flow_from_directory(\n",
" directory=train_path, \n",
" target_size=(100, 100), \n",
" color_mode=\"rgb\", \n",
" batch_size=batch_size, \n",
" classes=['class1','class2']\n",
" class_mode=\"categorical\",\n",
" shuffle=True, \n",
" seed=42 \n",
")"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "wcdGYlzkcdhg",
"colab_type": "text"
},
"source": [
"The method **flow_from_dataframe()** is useful when the images of different classes are reside in one folder.\n",
"\n",
"In this case, the text file contains information about class labels. \n",
"We make dataframe using pandas (library) and text file to classify images."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5BUQmE0pcdhq",
"colab_type": "text"
},
"source": [
"## 2. Define Model"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WFGYjXQGcdhy",
"colab_type": "text"
},
"source": [
"There are two ways to build Keras models: sequential and functional. \n",
"\n",
"**Sequential Model**\n",
"\n",
"The sequential API allows creating models layer-by-layer for most problems. It does not allow to create models that share layers or have multiple inputs or outputs."
]
},
{
"cell_type": "code",
"metadata": {
"id": "ygU5KGn5cdh5",
"colab_type": "code",
"colab": {}
},
"source": [
"from keras.models import Sequential\n",
"from keras.layers import Dense\n",
"\n",
"model = Sequential()\n",
"model.add(Dense(2, input_dim=1))\n",
"#model.add(Dense())\n",
"model.add(Dense(1))"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "zgeHs7JccdiY",
"colab_type": "text"
},
"source": [
"**Functional Model**\n",
"\n",
"The functional API allows creating models where layers connect to more than just the previous and next layers. You can connect layers to any other layer. "
]
},
{
"cell_type": "code",
"metadata": {
"id": "U3G1eFGncdie",
"colab_type": "code",
"colab": {}
},
"source": [
"from keras.models import Model\n",
"from keras.layers import Input\n",
"from keras.layers import Dense\n",
"\n",
"# Define the input\n",
"input_layer = Input(shape=(2,)) \n",
"# Connecting layers\n",
"hidden_layer = Dense(2)(input_layer) \n",
"# Create the model\n",
"model = Model(inputs=input_layer, outputs=hidden_layer)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "0q4lgkgZcdi7",
"colab_type": "text"
},
"source": [
"## 3. Compile Model\n",
"\n",
"This step will create a Python object which builds the CNN. This is done by building the computation graph on the Keras backend.\n",
"\n",
"In this step, we define loss function and type of optimizer."
]
},
{
"cell_type": "code",
"metadata": {
"id": "djdJuC8FcdjA",
"colab_type": "code",
"colab": {}
},
"source": [
"# compile the model\n",
"model.compile(loss='binary_crossentropy', optimizer='sgd', metrics=['accuracy'])\n",
"\n",
"# Now we have a Python object that has model and all its parameters with its initial values."
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "unR22-G7cdjT",
"colab_type": "text"
},
"source": [
"## 4. Fit Model\n",
"\n",
"In this step we train the model so that the parameters get tuned to provide the correct outputs for a given input. This can be done by feeding the inputs at the input layer and then getting an output, calculate the loss function using the output and then use backpropagation to tune the model parameters. This step will **fit** the model parameters to the data.\n",
"\n",
"There are two ways to fit the Keras model\n",
"#### 1. **fit()**"
]
},
{
"cell_type": "code",
"metadata": {
"id": "ECbUcsYOcdjY",
"colab_type": "code",
"colab": {}
},
"source": [
"model.fit(train_X, train_Y, batch_size=32, epochs=50) # pass complete training data (train_X, train_Y) at once in the fit function. "
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "47VibtOmcdjl",
"colab_type": "text"
},
"source": [
"This function is suitable for small datasets only. However, real-world data sets are usually too large to fit in memory. "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BNQu2i7Ocdjp",
"colab_type": "text"
},
"source": [
"#### 2. **fit_generator()**"
]
},
{
"cell_type": "code",
"metadata": {
"id": "zzpvjTi1cdjt",
"colab_type": "code",
"colab": {}
},
"source": [
"model.fit_generator(\n",
" train_generator, # (tuple) data (inputs, targets)\n",
" steps_per_epoch = None, # (integer) value when the second data epoch ends with an epoch and executes the next epoch\n",
" epochs = 10, # (integer) number of rounds of data iteration\n",
" verbose = 1, # verbose = 0 (silent), verbose = 1 (animated progress bar of current status), verbose = 2 (the no of epoch)\n",
" callbacks = None, # \n",
" validation_data = None, # validation data tuple\n",
" validation_steps = None, # validation data steps valdation_data_samples// train_batch\n",
" class_weight = None, # define a dictionary with class labels and their associated weights cl_weight = {0: 1, 1: 50}.\n",
" max_queue_size = 0, # maximum capacity of the generator queue\n",
" workers = 1,\n",
" use_multiprocessing = False,\n",
" shuffle = True, \n",
" initial_epoch = 0) # start training from this epoch (used to continue training)\n",
"\n",
"# steps_per_epoch is used to indicate that one epoch is completed. The value is usually set by dividing the total no of training \n",
"# samples by the batch size. The result is no of steps per epoch we use. "
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "GSq1UraLcdj5",
"colab_type": "text"
},
"source": [
"In **fit_generator()**, we don't pass the training data directly, instead they come from a generator. The fit_generator() function accepts the batch of data, performs backpropagation, and updates the weights in our model."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4BtPQWn6cdj9",
"colab_type": "text"
},
"source": [
"## 5. Evaluate Model"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "lpIjVxRYcdkA",
"colab_type": "text"
},
"source": [
"Evaluate your model using **evaluate()** and **evaluate_generator()** functions to evaluate the performance of network on the test dataset.\n",
"\n",
"These functions will return a list with two values. The first will be the loss of the model and the second will be the accuracy of the model on the test dataset."
]
},
{
"cell_type": "code",
"metadata": {
"id": "P3EoLtEocdkI",
"colab_type": "code",
"colab": {}
},
"source": [
"# evaluate the keras model\n",
"_, accuracy = model.evaluate(X, y)\n",
"print('Accuracy: %.2f' % (accuracy*100))\n",
"\n",
"_, accuracy = model.evaluate_generator(test_generator, steps=None, callbacks=None, max_queue_size=10, workers=1, use_multiprocessing=False, verbose=0)\n",
"print('Accuracy: %.2f' % (accuracy*100))"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "hbhagJw-cdkg",
"colab_type": "text"
},
"source": [
"## 6. Make predictions "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0FZaBv8Tcdkp",
"colab_type": "text"
},
"source": [
"Make predictions on the test dataset. The functions **predict()**, **predict_classes()** and **predict_generator()** are used\n",
"to make predictions on test dataset. \n",
"\n",
"- **predict()** will return 0.6 class1 and 0.4 class2 \n",
"- **predict_classes()** will return the actual class. For example, class1\n",
"- **predict()** will return the result value in case of regression"
]
},
{
"cell_type": "code",
"metadata": {
"id": "Lwbz74nRcdku",
"colab_type": "code",
"colab": {}
},
"source": [
"# make probability predictions with the model\n",
"predictions = model.predict(X) # model.predict for regression problems\n",
"for i in range(len(X)):\n",
" print(\"X=%s, Predicted=%s\" % (X[i], predictions[i]))\n",
"\n",
"# make the class predictions with the model\n",
"predictions = model.predict_classes(X) # model.predict for classification problems\n",
"for i in range(len(X)):\n",
" print(\"X=%s, Predicted=%s\" % (X[i], predictions[i]))"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "NrYqWlEjcdk6",
"colab_type": "text"
},
"source": [
"# How to save a Keras model?"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "mlqEl70ycdlB",
"colab_type": "text"
},
"source": [
"Save model using **save()** function. This function make a single HDF5 file which contains:\n",
"\n",
"+ the architecture of the model, allowing to re-create the model.\n",
"+ the weights of the model.\n",
"+ the training configuration (loss, optimizer).\n",
"+ the state of the optimizer, allowing to resume training exactly where you left off."
]
},
{
"cell_type": "code",
"metadata": {
"id": "-YchKfoFcdlG",
"colab_type": "code",
"colab": {}
},
"source": [
"model.save(\"model.h5\")"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "Jr2ugWnjcdlX",
"colab_type": "text"
},
"source": [
"# How to load a Keras model?"
]
},
{
"cell_type": "code",
"metadata": {
"id": "RxgCoZSQcdlb",
"colab_type": "code",
"colab": {}
},
"source": [
"from keras.models import load_model\n",
"\n",
"model = load_model('model.h5')"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "0O-p1VVQcdlq",
"colab_type": "text"
},
"source": [
"# How to save weights after some epochs in Keras?\n",
"\n",
"Deep learning models can take hours, days or even weeks to train. If the run is stopped unexpectedly, you can lose a lot of work. So, the ideal solution is to save weights after every or some epochs.\n",
"\n",
"In keras ModelCheckpoint callback class helps us to save our model weights for each epoch. "
]
},
{
"cell_type": "code",
"metadata": {
"id": "BAg5ZDntcdlu",
"colab_type": "code",
"colab": {}
},
"source": [
"from keras.callbacks import ModelCheckpoint\n",
"\n",
"checkpoint = ModelCheckpoint(filepath, # path to save the model file\n",
" monitor='val_acc', # \n",
" verbose=1, \n",
" save_best_only=False, # will save best weights only\n",
" save_weights_only=True, # if True (model.save_weights(filepath)) else save weights with model (model.save(filepath))\n",
" mode='auto', # {auto, min, max} \n",
" period=10) # difference between two checkpoints\n",
"callbacks_list = [checkpoint]\n",
"# now use this checkpoint in step 4 (fit model)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "RvL_9xIHcdmC",
"colab_type": "text"
},
"source": [
"# Split validation data automatically using Keras ImageDataGenerator"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "r2hfBqwYcdmF",
"colab_type": "text"
},
"source": [
"Use **validation_split** parameter that takes a floating-point number between 0 and 1, which is used to specify a certain proportion of data in the training set as the verification set. \n",
"\n",
"**validation_split** takes all input data and splits it between train and validation sets. \n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "0_-dEANDcdmI",
"colab_type": "code",
"colab": {}
},
"source": [
"model.fit(train_X, train_Y, batch_size=16, epochs=10, verbose=2, validation_split=0.2) "
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "-uKc1RJRcdmV",
"colab_type": "text"
},
"source": [
"**validation_data** requires valX, valY explicitly. "
]
},
{
"cell_type": "code",
"metadata": {
"id": "XzF-mlH8cdmY",
"colab_type": "code",
"colab": {}
},
"source": [
"model.fit(train_X, train_Y, validation_data=(testX,testY), epochs=10, batch_size=10)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "_gvNTaS-cdmw",
"colab_type": "text"
},
"source": [
"How to split data in case of **ImageDataGenerator**?"
]
},
{
"cell_type": "code",
"metadata": {
"id": "FGGAnR6scdm1",
"colab_type": "code",
"colab": {}
},
"source": [
"train_datagen = ImageDataGenerator(rescale=1./255,\n",
" validation_split=0.2) # set validation split\n",
"\n",
"train_generator = train_datagen.flow_from_directory(\n",
" train_data_dir,\n",
" target_size=(img_height, img_width),\n",
" batch_size=batch_size,\n",
" class_mode='binary',\n",
" subset='training') # set as training data\n",
"\n",
"validation_generator = train_datagen.flow_from_directory(\n",
" train_data_dir, # same directory as training data\n",
" target_size=(img_height, img_width),\n",
" batch_size=batch_size,\n",
" class_mode='binary',\n",
" subset='validation') # set as validation data\n",
"\n",
"model.fit_generator(\n",
" train_generator,\n",
" steps_per_epoch = train_generator.samples // batch_size,\n",
" validation_data = validation_generator, \n",
" validation_steps = validation_generator.samples // batch_size,\n",
" epochs = nb_epochs)"
],
"execution_count": 0,
"outputs": []
}
]
}