{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "XoNilKhg-58H"
},
"source": [
"#
\n",
" \n",
" \n",
" ** Initialization **\n",
" | \n",
" \n",
" **Train accuracy**\n",
" | \n",
" \n",
" **Test accuracy**\n",
" | \n",
"
\n",
" \n",
" Zeros initialization\n",
" | \n",
" \n",
" 0%\n",
" | \n",
" \n",
" 0%\n",
" | \n",
" \n",
" \n",
" Random initialization and c=0.01\n",
" | \n",
" \n",
" 0%\n",
" | \n",
" \n",
" 0%\n",
" | \n",
"
\n",
" \n",
" \n",
" Random initialization and c=100\n",
" | \n",
" \n",
" 0%\n",
" | \n",
" \n",
" 0%\n",
" | \n",
"
\n",
" \n",
" \n",
" Xavier initialization \n",
" | \n",
" \n",
" 0%\n",
" | \n",
" \n",
" 0%\n",
" | \n",
"
\n",
"
\n",
" \n",
" \n",
"\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## b) Optimizer\n",
"\n",
"Train a two layer neural network for binary classification and fill the table given below. **(50 marks)**\n",
"+ Dataset = MNIST\n",
"+ Classes = 1 and 2 digits from MNIST\n",
"+ Initialization = Xavier\n",
"+ No. of layers = 2 \n",
"+ Activation function on each hidden layer = sigmoid \n",
"+ Neurons in each layer = 5\n",
"+ Batch size = 32 \n",
"+ Epochs = 100 \n",
"+ Learning rate = 1e-3 \n",
" \n",
" \n",
" \n",
" \n",
" **Optimization method**\n",
" | \n",
" \n",
" **Train accuracy**\n",
" | \n",
" \n",
" **Test accuracy**\n",
" | \n",
"
\n",
" \n",
" Gradient Descent\n",
" | \n",
" \n",
" 0%\n",
" | \n",
" \n",
" 0%\n",
" | \n",
" \n",
" \n",
" Stochastic Gradient Descent\n",
" | \n",
" \n",
" 0%\n",
" | \n",
" \n",
" 0%\n",
" | \n",
"
\n",
" \n",
" \n",
" Momentum\n",
" | \n",
" \n",
" 0%\n",
" | \n",
" \n",
" 0%\n",
" | \n",
"
\n",
" \n",
" \n",
" Adam\n",
" | \n",
" \n",
" 0%\n",
" | \n",
" \n",
" 0%\n",
" | \n",
"
\n",
"
\n",
" "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"## add adam code here (30 marks)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## c) Regularization\n",
"\n",
"Train a two layer neural network for binary classification and fill the table given below. **(30 marks)**\n",
"+ Dataset = MNIST\n",
"+ Classes = 1 and 2 digits from MNIST\n",
"+ No. of layers = 2\n",
"+ Initialization = Xavier\n",
"+ Optimizer = SGD\n",
"+ Activation function on each hidden layer = sigmoid \n",
"+ Neurons in each layer = 5\n",
"+ Batch size = 32 \n",
"+ Epochs = 100 \n",
"+ Learning rate = 1e-3 \n",
" \n",
"