This page has not yet been updated to the second edition
Below are links to a variety of software related to examples and
exercises in the book,
organized by chapters (some files appear in multiple places). See
Mountain Car code.
Most of the rest of the code is written in Common Lisp and requires
routines available here. For the
graphics, you will need the the packages for G and in some
cases my graphing
Even if you can not run this code, it still may clarify
some of the details of the experiments. However, there is no guarantee
the examples in the book were run using exactly the software given.
This code also has not been extensively tested or documented and is
being made available "as is".
If you have corrections, extensions, additions or improvements of any
kind, please send
them to me at firstname.lastname@example.org for inclusion here.
code for nearly all the examples and excercises in the book has
been contributed by John Weatherwax. Thanks John!
- Chapter 1: Introduction
- Chapter 2: Evaluative Feedback
- Chapter 3: The Reinforcement Learning Problem
- Chapter 4: Dynamic Programming
- Chapter 5: Monte Carlo Methods
- Chapter 6: Temporal-Difference Learning
- Chapter 7: Eligibility Traces
- Chapter 8: Generalization and Function Approximation
- Chapter 9: Planning and Learning
- Chapter 10: Dimensions of Reinforcement Learning
- Chapter 11: Case Studies
For other RL software see the
Reinforcement Learning Repository at Michigan State University and