Home Reinforcement Learning and Artificial Intelligence (RLAI)

The agent hypothesis
--Rich Sutton, Feb 22 2006

The ambition of this web page is to state, refine, clarify and, most of all, promote discussion of, the following scientific hypothesis:

For the purposes of artificial intelligence, psychology, control theory and related fields, the universe is well thought of as consisting of exactly two subsystems that exchange signals over time, where the signals in one direction are thought of as choices and the signals in the other direction are thought of as informing the choices.

The subsystem emitting the choices is the agent and the subsystem receiving the choices is the environment.  The word "choice" suggests of course that the agent has a goal of some sort (see reward hypothesis).
return to hypotheses
Anna:
This general statement I agree with entirely, but I think it also useful to further divide the subsystem that informs the choice.  
Brian:
Really?  "well thought of" and "exactly two".  What are we saying here?  If you want to claim that this is the most general way to think of the Universe, I'd agree.  But "well thought of" is so fluffy.  What of sometimes thinking about the universe as 3 systems is exactly the right thing to do... how does that affect this hypothesis.

Isn't this just more of a proposal or statement than it is something that could ever be tested or decided?  
Anna:
I think this relates to Brian's complaint.

"Well thought of" is what makes the hypothesis impossible to falsify. This hypothesis is not falsified by coming up with a system that is better thought of as three (or five) systems, because the hypothesis only claims that things are "well thought of" in this way. Of course, I can be proven wrong by providing hypothetical falsifying criteria.

I claim that the agent hypothesis can be a terrible way to think of the universe, if it is the only way the universe is thought of. In some cases, thinking of a collection of subsystems making choices is more productive than either lumping every choice-making subsystem together into one agent or lumping all choice-making subsystems but one into the environment.

But I'm not falsifying the agent hypothesis, because it still could in some ways be "well thought of" as one or both of the above lumpings.  
Rich:
First, let me re-invite you all to write it better than i have.  thats why we are all here.  i am not entirely happy with the way it is now.  much too long.  and "universe" is too pretentious.

but "well thought of" doesn't bother me much.  you will find that phrase in many of the hypotheses.  It is graded, not absolute, but this i am thinking is appropriate, perhaps even necessary.  it will be true or false to the extent that this does prove to be a useful way of thinking.  and this is what we need to know now -- what are good ways of thinking about the problem.  it may be twenty years from now, but i think we will eventually know how true or false the hypothesis has proved to be.
Extend this Page   How to edit   Style   Subscribe   Notify   Suggest   Help   This open web page hosted at the University of Alberta.   Terms of use  2123/0