Deep Reinforcement Learning (CS-566)

Department of Computer Science
University of The Punjab

Instructor: Nazar Khan
Semester: Spring, 2026
Credit Hours: 3
Level: Graduate
Location: FCIT, Allama Iqbal Campus, AlKhwarizmi Lecture Theater
Class Times: Mon. and Wed., 12:15 PM - 2:15 PM

Google Classroom

Textbook: DRL
drl_cover
Deep Reinforcement Learning
Aske Plaat
Springer Nature, 2022
Preprint

Reference 1: RL
rlai_cover
Reinforcement Learning: An Introduction, 2nd Edition
Richard Sutton and Andrew Barto
MIT Press, 2018
PDF

Reference 2: DRLFRA
drlfra_cover
Deep Reinforcement Learning: Fundamentals, Research and Applications
Springer, 2020

In recent years, Deep Reinforcement Learning (Deep RL) has emerged as one of the most exciting and rapidly advancing fields in artificial intelligence. Combining the sequential decision-making framework of reinforcement learning with the powerful function approximation capabilities of deep neural networks, Deep RL has enabled breakthroughs in areas ranging from game playing—such as AlphaGo and OpenAI Five—to robotics, autonomous driving, operations research, financial systems, and large language models.

Lectures

Lecture Topic Readings Tutorials & Evaluations
1 Introduction DRL: Ch 1
2 Markov Decision Process (MDP) DRL: Sec 2.1 - 2.2.2 Quiz 1
3 MDP Terminology - I DRL: Sec 2.2.3.1
4 MDP Terminology - II DRL: Sec 2.2.3.2 - 2.2.3.3 Quiz 2
5 MDP Objective Function DRL: Sec 2.2.3.4 - 2.2.3.5
6-7 Known Model-Based Learning DRL: Sec 2.2.4.1 Quiz 3
8 Tutorial: Value Iteration on GridWorld
Python Notebook 		DRL: Sec 2.2.4.1 Assignment 1
Random vs. Value Iteration
9 Model-Free Learning - I: Reward Sampling DRL: Sec 2.2.4.2 Quiz 4
10 Model-Free Learning - II: Action Selection DRL: Sec 2.2.4.3 Assignment 2
11 Model-Free Learning - III: Learning from Rewards DRL: Sec 2.2.4.4 - 5
12 Deep Value-Based Agents DRL: Sec 3.1 - 3.2.2 Quiz 5, Quiz 6, Quiz 7
13 Stable Deep Value-Based Learning DRL: Sec 3.2.3 - 3.2.4
14 Policy-Based Learning DRL: Sec 4.1 - 4.2.2
Cartpole Balancing via REINFORCE .ipynb, .html
Gaussian Density .ipynb, .html
Mountain Car via REINFORCE .ipynb, .html
REINFORCE_cartpole_episodes
15 Improved Policy-Based Learning I: Actor-Critic Methods DRL: Sec 4.2.3 - 4.2.4
16 Improved Policy-Based Learning II: Trust Region Methods DRL: Sec 4.2.5
17 Improved Policy-Based Learning III: Soft Actor-Critic (SAC) DRL: Sec 4.2.6
18 Improved Policy-Based Learning IV: Deterministic Policy Gradient DRL: Sec 4.2.7
19 Learned Model-Based Learning DRL: Sec 5.1

Grading

Assignments Project Quizzes Midterm Exam Final Exam
Weight 12% 8% 5% 35% 40%