{ "cells": [ { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "XoNilKhg-58H" }, "source": [ "#
CS568:Deep Learning
Spring 2020
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Assignment 2\n", "\n", "**Total Marks: 100** \n", "\n", "**Assigned **: Friday, April 17, 2020 \n", "\n", "**Due **: Friday, April 24, 2020 before **8.00 AM**\n", "\n", "## a) Initialization \n", "\n", "Train a two layer neural network for binary classification and fill the table given below. **(20 marks)**\n", "+ Dataset = MNIST\n", "+ Classes = 1 and 2 digits from MNIST\n", "+ No. of layers = 2 \n", "+ Activation function on each hidden layer = sigmoid \n", "+ Neurons in each layer = 5\n", "+ Batch size = 32 \n", "+ Epochs = 100 \n", "+ Learning rate = 1e-3 \n", "+ Optimizer = Stochastic gradient descent\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", " ** Initialization **\n", " \n", " **Train accuracy**\n", " \n", " **Test accuracy**\n", "
\n", " Zeros initialization\n", " \n", " 0%\n", " \n", " 0%\n", "
\n", " Random initialization and c=0.01\n", " \n", " 0%\n", " \n", " 0%\n", "
\n", " Random initialization and c=100\n", " \n", " 0%\n", " \n", " 0%\n", "
\n", " Xavier initialization \n", " \n", " 0%\n", " \n", " 0%\n", "
\n", " \n", " \n", "\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## b) Optimizer\n", "\n", "Train a two layer neural network for binary classification and fill the table given below. **(50 marks)**\n", "+ Dataset = MNIST\n", "+ Classes = 1 and 2 digits from MNIST\n", "+ Initialization = Xavier\n", "+ No. of layers = 2 \n", "+ Activation function on each hidden layer = sigmoid \n", "+ Neurons in each layer = 5\n", "+ Batch size = 32 \n", "+ Epochs = 100 \n", "+ Learning rate = 1e-3 \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", " **Optimization method**\n", " \n", " **Train accuracy**\n", " \n", " **Test accuracy**\n", "
\n", " Gradient Descent\n", " \n", " 0%\n", " \n", " 0%\n", "
\n", " Stochastic Gradient Descent\n", " \n", " 0%\n", " \n", " 0%\n", "
\n", " Momentum\n", " \n", " 0%\n", " \n", " 0%\n", "
\n", " Adam\n", " \n", " 0%\n", " \n", " 0%\n", "
\n", " " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "## add adam code here (30 marks)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## c) Regularization\n", "\n", "Train a two layer neural network for binary classification and fill the table given below. **(30 marks)**\n", "+ Dataset = MNIST\n", "+ Classes = 1 and 2 digits from MNIST\n", "+ No. of layers = 2\n", "+ Initialization = Xavier\n", "+ Optimizer = SGD\n", "+ Activation function on each hidden layer = sigmoid \n", "+ Neurons in each layer = 5\n", "+ Batch size = 32 \n", "+ Epochs = 100 \n", "+ Learning rate = 1e-3 \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", " **Regularization method**\n", " \n", " **Train accuracy**\n", " \n", " **Test accuracy**\n", "
\n", " l2-weight\n", " \n", " 0%\n", " \n", " 0%\n", "
\n", " Dropout\n", " \n", " 0%\n", " \n", " 0%\n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# add l2-weight and dropout code here (You can copy your fporp and bprop functions here) (20 marks)" ] } ], "metadata": { "colab": { "collapsed_sections": [], "name": "pytorch_mnist.ipynb", "provenance": [] }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.9" } }, "nbformat": 4, "nbformat_minor": 1 }