| date |
day |
topic |
Assignment due |
| 9-Jan |
Tuesday |
Intro, logistics, requirements, expectations |
|
| 11-Jan |
Thursday |
Introduction |
Read all of Chapter 1; 2 thought questions |
| 16-Jan |
Tuesday |
Bandit Methods |
Read the nonstarred sections of Chapter 2 plus 2.8;
2 thought questions |
| 18-Jan |
Thursday |
Bandit Methods |
jeopardy quiz followup; thought questions for
chapters 1 and 2 if you haven't done them yet |
| 23-Jan |
Tuesday |
Markov Decision Processes |
Exercises 2.1, 2.5, and 2.55; 2.8 is extra credit |
| 25-Jan |
Thursday |
Markov Decision Processes |
Read all of Chapter 3; 2 thought questions |
| 30-Jan |
Tuesday |
Value Functions |
|
| 1-Feb |
Thursday |
RL-Glue, RL-Library |
Exercises 3.4, 3.5, (3.6 is extra credit), 3.8 (omit
final part re eq 3.10), 3.9, 3.10, 3.11, 3.15, 3.17 |
| 6-Feb |
Tuesday |
State |
Party Problem Assignment #1 |
| 8-Feb |
Thursday |
Dynamic Programming |
State Exercise. Read Chapter 4; 2 thought questions |
| 13-Feb |
Tuesday |
Dynamic Programming |
Exercises 4.1, 4.2, 4.3, 4.5, 4.9 |
| 15-Feb |
Thursday |
Monte Carlo Methods |
Party Problem Assignment #2; Read Chapter 5; 2
thought questions |
| 27-Feb |
Tuesday |
Monte Carlo Control |
Exercises 5.1, 5.2, 5.5 |
| 1-Mar |
Thursday |
Temporal Difference Learning |
Read Chapter 6; 2 thought questions |
| 6-Mar |
Tuesday |
Temporal Difference Learning |
Exercises 6.1,6.2,6.3,6.8,6.9,6.10,6.12 |
| 8-Mar |
Thursday |
Midterm Exam |
|
| 13-Mar |
Tuesday |
Integrating Monte Carlo and Temporal-difference
Methods |
Read Chapter 7; 2 thought questions |
| 15-Mar |
Thursday |
Eligibility Traces |
Exercises 7.2 and 7.6 |
| 20-Mar |
Tuesday |
Function Approximation |
Read Chapter 8; 2 thought questions; blackjack programming assignment
|
| 22-Mar |
Thursday |
Function Approximation |
Exercises 8.1, 8.2, 8.6 and 8.7 |
| 27-Mar |
Tuesday |
Function Approximation |
first function approx programming assignment |
| 29-Mar |
Thursday |
Function Approximation |
first function approx programming assignment |
| 3-Apr |
Tuesday |
Integrating Learning and Planning: Dyna |
Read Chapter 9; 2 thought questions |
| 5-Apr |
Thursday |
Model-based backups |
Exercises 9.1,9.2,9.3,9.5 (9.6 is extra credit) |
| 10-Apr |
Tuesday |
student presentations |
Read Chapter 10, 2 thought questions |
| 12-Apr |
Thursday |
student presentations |
2nd function approx programming assignment due |
19-Apr
|
Thursday
|
final exam |
|