 |
Reinforcement Learning and
Artificial
Intelligence (RLAI)
|
Description
of CMPUT 609, Winter 2010:
Reinforcement
Learning in Artificial Intelligence
|
The
ambition
of this web
page is to provide basic, background information describing CMPUT 609,
a course at the
University of Alberta. The central web page for the course is here.
Instructor: Rich Sutton,
(sutton@cs.ualberta.ca) (http://www.cs.ualberta.ca/~sutton)
Office:
Athabasca 3-13 Office hours:
after class
Teaching
assistants: TBD
Class
Times: Tuesday and Thursday, 3:30-4:50 Class
Room: CSC B-41
Description: This course will provide a comprehensive
introduction to reinforcement learning as an approach to artificial
intelligence, emphasizing the design of complete agents interacting
with
stochastic, incompletely known environments. Reinforcement learning has
adapted
key ideas from machine learning, operations research, psychology, and
neuroscience to produce some strikingly successful engineering
applications.
The focus is on algorithms for learning what actions to take, and when
to take
them, so as to optimize long-term performance. This may involve
sacrificing
immediate reward to obtain greater reward in the long-term or just to
obtain
more information about the environment. The course will cover Markov
decision
processes, dynamic programming, temporal-difference learning, Monte
Carlo
reinforcement learning methods, eligibility traces, the role of
function
approximation, and the integration of learning and planning. The course
will
emphasize the development of intuition relating the mathematical theory
of
reinforcement learning to the design of human-level artificial
intelligence.
Textbook: Reinforcement
Learning: An Introduction,
by Richard S. Sutton and Andrew G. Barto. Although
a version of the textbook is available online, students are strongly
encouraged to get
their hands on the physical textbook. Much
of the
readings and questions will come directly from
the book. The textbook is available in the
bookstore.
Prerequisites: Interest in learning approaches to
artificial
intelligence; basic probability theory; computer programming ability.
You
should be comfortable with statistical ideas such as probability
distributions
and expected values.
Familiarity with
linear algebra would be helpful but is not required.
Written
Exercises: There will be a set
of exercises for most chapters. These will be due at the beginning of
the second
day on which the chapter is covered in class. All exercises will be
marked and
returned to you. Answer sheets for each week's exercises will be made
available
at the class on the day on which the exercises are due, so your
exercises must
be turned in on time.
Grading will be on the
basis of (with relative weighting):
1
Readings, research diary entries
2 Written exercises
2 Mid-term
2 Term project
