Tip:
Highlight text to annotate it
X
[Narrator] Here's a state space
diagram for a simple problem.
It involves a room with 2 locations.
The left we call A, and the right we call B,
and in that environment
there's a vacuum cleaner, and there
may or may not be dirt in either of the 2 locations,
and so that gives us 8 total states.
Dirt is here or not, here or not, and
the vacuum cleaner is here or here.
So that's 2 times 2 times 2
is 8 possible states, and I've drawn
here the states based diagram
with all the transitions
for the 3 possible actions, and the actions are moving right.
So we'd go from this state to this state.
Moving left, we'd go from this state to this state,
and sucking up dirt, we'd go from this state
to this state for example, and
in this state space diagram,
if we have a fully deterministic,
fully observable world, it's easy to plan.
Say we start in this state, and we want to be--
end up in a goal state where both sides are clean.
We can execute the suck-dirt action
and get here and then move right,
and then suck dirt again,
and now we end up in a goal state
where everything is clean.
Now suppose our robot vacuum cleaner's
sensors break down, and so the robot
can no longer perceive either
which location its in
or whether there's any dirt.
So we now have an unobservable
or sensor-less world rather
than a fully observable one,
and how does the agent then represent the state of the world?
Well it could be in any one of these 8 states,
and so all we can do to represent
the current state is draw a big circle
or box around everything, and say,
"I know I'm somewhere inside here."
Now that doesn't seem like it helps very much.
What good is it to know that
we don't really know anything at all?
But the point is that we can search in the state
space of the least states rather
than in the state space of actual spaces.
So we believe that we're in 1 of these 8 states,
and now when we execute an action,
we're going to get to another belief state.
Let's take a look at how that works.