Tip:
Highlight text to annotate it
X
Here's an example of tracking the Predict Update Cycle;
and this is in a world in which the actions are guaranteed to work, as advertised--
that is, if you start to clean up the current location,
and if you move right or left, the wheels actually turn; and you do move.
But we can call this the kindergarten world because there are little toddlers
walking around who can deposit Dirt in any location, at any time.
So if we start off in this state, and execute the Suck action,
we can predict that we'll end up in one of these 2 states.
Then, if we have an observation--well, we know what that observation's going to be
because we know the Suck action always works, and we know we were in A;
so the only observation we can get is that we're in A--and that it's Clean--
so we end up in that same belief state.
And then, if we execute the Right action--
well, then lots of things could happen;
because we move Right, and somebody might have dropped Dirt in the Right location,
and somebody might have dropped Dirt in the Left location--or maybe not.
So we end up with 4 possibilities,
and then we can update again when we get the next observation--
say, if we observed that we're in B and it's Dirty, then we end up in this belief state.
And we can keep on going--specifying new belief states--
as a result of success of predicts and updates.
Now, this Predict Update Cycle gives us a kind of calculus of belief states
that can tell us, really, everything we need to know.
But there is one weakness with this approach--
that, as you can see here, some of the belief states start to get large;
and this is a tiny little world.
Already, we have a belief state with 4 world states in it.
We could have one with 8, 16, 10, 24--or whatever.
And it seems that there may be more succinct representations of a belief state,
rather than to just list all the world states.
For example, take this one here:
If we had divided the world up--not into individual world states,
but into variables describing that state,
then this whole belief state could be represented just by: Vacuum is on the Right.
So the whole world could be represented by 3 states--or 3 variables:
One, where is the Vacuum--is it on the Right, or not?
Secondly, is there Dirt in the Left location?
And third, is there Dirt in the Right location?
And we could have some formula, over those variables, to describe states.
And with that type of formulation,
some very large states--in terms of enumerating the world states--
can be made small, in terms of the description.