||Reinforcement Learning and
The ambition of this page is
provide a location to coordinate the Subjective Robotics meeting.
you attend this meeting, please subscribe
to this page.
This meeting is held each Tuesday at 1pm in CSC 249 at the University
We thought we would continue discussing
David Pierce's work, including a look at his thesis. His work
clearly has many of the same goals and perspectives as we do. We
should look carefully to see if we can utilize any of his ideas.
(rich probably will not attend because his furniture arrives this day from NJ)
We discussed the following topics:
- Is it possible to implement the subjective-robot description of
the lawnmower task as presented by Rich last week and still have
What is a map, really, and are TD Nets maps?
- Last week Rich proposed a 2d angle-centric view of a lawn where
a mower could be driven around the perimeter of the lawn and
then subjectively represent each point in its path as two coordinates:
the angles between
itself and two stakes set at arbitrary positions outside the lawn while
the mower is facing a third such stake.
- Michael pointed out a problem with that representation: in the
real task, there are more than 3 stakes, and if there are three angle
dimensions then the lawn is a 2-dimensional manifold within that
3-space and there's no way the robot can tell whether the it is
"inside" or "outside" the perimeter of the lawn without having seen the
entire lawn. The problem would presumably get worse with more
- This week Mark asked whether there might be a way to have a
"primary" map, as Rich described it, which is then merely augmented
with information from the other stakes to help reduce noise and make
finer distinctions/predictions. That way, the mower could always
its approximate location based on the primary map, including whether it
was inside or outside the perimeter. As it traversed the lawn,
it possibly, somehow, in some way integrate angle changes from
additional stakes to distinguish positions in the primary map?
- It wasn't clear from the discussion whether or not this
approach could be realized without "cheating" and giving the robot too
much additional objective information.
- Michael wondered about the relationship between maps and TD
- Maps seem to do at least three things:
Clearly, TD Nets are designed to do (1). It's not clear
whether (3) is particularly important for robotics at all. So the
issue of discussion became (2). In particular, is there a way to
whether an agent using a perfect TD Net, if started in a random state
with a random state vector, will eventually converge to a perfect state
vector that represents the actual state of the agent? After much
thinking, we decided this was non-trivial and possibly fairly
important. The discussion carried over to the TD Nets meeting
For next week we will be reading the Pierce, Kuipers paper
- allow a robot to predict what will happen if it takes certain
- allow a robot to locate itself within its environment by
suggesting specific tests
- provide distance relationships between all areas of the map,
which may or may not be directly related to the robot's actions.
We will be discussing the lawnmower problem in a
The paper regarding the problem by Huang and
can be found here:
Localization System Solely Based on Vision for Autonomous Lawnmowers