Code for:

Reinforcement Learning: An Introduction
by Richard S. Sutton and Andrew G. Barto

This page has not yet been updated to the second edition

Below are links to a variety of software related to examples and exercises in the book, organized by chapters (some files appear in multiple places). See particularly the Mountain Car code. Most of the rest of the code is written in Common Lisp and requires utility routines available here. For the graphics, you will need the the packages for G and in some cases my graphing tool. Even if you can not run this code, it still may clarify some of the details of the experiments. However, there is no guarantee that the examples in the book were run using exactly the software given. This code also has not been extensively tested or documented and is being made available "as is". If you have corrections, extensions, additions or improvements of any kind, please send them to me at for inclusion here.

Matlab code for nearly all the examples and excercises in the book has been contributed by John Weatherwax. Thanks John!

For other RL software see the Reinforcement Learning Repository at Michigan State University and here.