Tip:
Highlight text to annotate it
X
Hi, in this last lecture about Markov processes, what I want to talk about
is exaptation. I want to talk about how we can use the Markov model in context,
and for problems we never would have thought of. And we do this in two ways.
First I'm gonna talk about taking it whole hog, taking the entire process and
modeling other things, just like we modeled the process of states becoming
free or dictatorial. Second, what I wanna do is, I wanna just take part of the Markov
model, its transition probability matrix, and use that to understand some things
that are really kinda surprising and interesting. So let's just remind
ourselves of what a Markov process is. Fixed set of states, and then there's
fixed transition probabilities between those states. Now if it's possible to get
from any state to any other through a sequence of transitions, then remember, we
have this Markov convergence theorem that says the process is going to go to a
unique equilibrium. So history doesn't matter. Initial conditions don't matter.
All that sort of stuff. Let's think about where else we might apply that. So we, one
place right away is voter turnout. So you can think about, there's a set of voters
at time t, and there's a set a of non-voters at time t, and what we can do is
draw a little matrix and say, how many of those are going to vote at time t+1,
and how many of these are not going to vote at time t+1? And it could be that 80
percent of voters at time t vote in t+1, twenty percent don't. It could be of
nonvoters that 40 percent vote and 60 percent don't vote. This would be our
Markov transition matrix and if you apply this, what you'd get is a unique
equilibrium that should tell you the number of people you'd expect to vote in
any election. Now it's not going to be the same people, right, because the process is
going to churn. It's a statistical equilibrium, not a fixed point. But this
model, if it's right, if these transition probabilities stay fixed, this would tell
us what turnouts should be. Even though it won't tell us who votes. Where else
would I use it? Well, we can use it for school enrollment. Same sort of thing, right?
[laugh] You can imagine, here's kids who go to school, and here's the ones that
don't go to school. And then we could ask, okay at time t+1, how many go, and how many
don't go? It could be that of those that go, 90 percent go the next day, and ten
percent don't. Of those that don't go, it could be that there's only a 50/50 chance
that they come the next day. Well again, if you work through the logic here, you'll
get some sort of percentage of people, students, who show up each day, and some
percentage that don't, there'll still be a churn, but this model will give you an
estimate of what total enrollment should be on a given day. What percentage of
students show up. These two applications are very standard applications. They're
not unlike the one we looked at, in terms of alert and bored students, and they're
not unlike the one with free countries versus dictatorial countries. What I wanna
do next is sort of go way outside the box. I wanna just take part of the Markov
model. The Markov transition matrix. And I want to think about what that tells us.
What it tells us is, if this the state of time t, what are the likelihoods beyond
these other states of time t+1? So think about this for a second. There's all
sorts of things you could use this framework for, where something happening at
time t, and then it transitions into something at time t+1. I'm gonna just talk
about three uses of this matrix, three very surprising uses. First one, to
identify writers. I mean, [inaudible] use this idea, this transition matrix
again, to figure out who wrote a book. So suppose some anonymous person writes a
book, and, you're trying to figure out, did this person write it? Did Bob write
it? Or you trying to figure out, did you know, Carlos write it? And you can't tell.
Well, what you can do is the following: You can figure out transition
probabilities. What do I mean? Well, take this book. Take the book written by an
anonymous author. And then say, okay, every time this book uses the word "for"--
You're loading the whole book in the computer. What percentage of the time does
it follow the word "for", with "the record"? What percentage of the time does
it follow the word "for" with "example"? And what percentage of the time does it follow
the word "for" with "the sake of". And what I'm doing is I'm creating, just, a giant
matrix. So if it, in some sense, like, at time t, I'm using "for". And then I'm
saying, what's the probability that I follow it with "the record", "example" or "sake"? And I'm
putting a .17, .9, .11, so I'm just trying to get a
big transition matrix. What you can do is you can take some key words, create these
giant transition matrices, and then figure out what does it look like. Does this
transition matrix look like Bob's transition matrix, or does it look like
Carlos' transition matrix? Now how do I know what Bob's transition matrix is and
what Carlos's transition matrix is? Well, that's easy. I just take one of their
other books, load it into the computer and figure out what their transition matrix is
for the other book. Once I got that in there I can figure out, what, does this
look more like Carlos or does this look more like Bob? Now, this actually gets
used. And let me tell an interesting story. So this is Arlene Saxenhouse. She's
one a of my colleagues at the University of Michigan. And when she was a young
graduate student she found, in the library at Yale, this book of, it included four
essays by someone who she thought was a young Thomas Hobbes. Book was published in
1620. She thought, 1620, and she thought, oh my gosh, these are essays by Hobbes.
And the thing is, she's a young grad student. Who's necessarily going to
believe her? How does she prove it? So she couldn't prove it. She had a strong
instinct that it was true. Well eventually she found someone who knew how to do this
stuff, knew how to do these transition probabilities, and they took the essays
and they put in some of Hobbes' other writings, and what they showed was that it
seems fairly clear that three of the four essays were actually written by Hobbes, and now
those three essays are actually considered part of Hobbes' work. Now, even though
appeal, you know, it just sort of felt to her like they were written by Hobbes. That
comes down to a matter of opinion. They were having a model, they were having
these transition probabilities models. Right? And by able to take that, and take
other Hobbes work and this work, you can show statistically that it seems very,
very likely that Hobbes wrote the work. So this combination of things. The model
alone doesn't do it. The combination of the model, plus her intuition, plus the
intuition of others, gives us a common understanding now, at least we think, that
Hobbes wrote those particular volumes. It's really cool. Let me give you another
example, medical diagnosis. So, if you think about giving someone a treatment for
some disease, typically, there's a sequence of reactions to that treatment.
Whether it's a drug protocol, or it's an exercise or diet regimen, what you can do
is, you can write down transition probabilities. Now, these can be
multi-stage. So it could be, for example, that if it's going to be successful, if
this treatment is going to be successful, that you go through the following
transition: you first feel some pain, then you're slightly depressed, then more pain,
but then you get better. Alternatively, if it's not successful, it could be that,
initially, you're depressed. Then there's mild pain. Then there's no pain. And then
the system fails. Not only the system fails, but you fail, the drug fails, the regimen
fails. So what does this mean? This means that, if I give someone the treatment, and
then I see this sequence of pain and depression, I can say to them, you know
you're feeling pain. You're feeling depression, but guess what, that's
consistent with a, a regimen that's gonna be successful. Whereas alternatively
someone else could say, well you know, I feel depressed but now I'm definitely not
feeling much pain. You can say to them, okay, even though you're not feeling much
pain, this probably isn't a good sign. It doesn't look like the treatment is gonna
work. So by gathering all sorts of data and past experiences you can use that
transition probability, to figure out early on in a treatment protocol, whether it looks like
it's working or not working. Another example: Lead up to war. Suppose you got
two countries and there's a little bit of tension. So what you get is the following:
you get, let's say first you get some political statements on each side. Then
that leads to trade embargoes. Then that leads to military buildup. So now you got
this sequence of three things. You have these transitions between these three
things. You can ask, historically, when I've had those three transitions, what's
the likelihood that I've had war, and the likelihood that I haven't had war. And it
could be that there's a twenty percent chance of war, and an 80 percent chance of
not having war. So, if you're just sort of on the ground watching what going on, you
say, oh boy, look at this! First, it was political statements, now there's a trade
embargo, now we're seeing military buildups. Looks like things are gonna go to
war. If we actually gather a lot of data and, and calculate these transition
probability matrices, you could figure out, you know, this actually happens a
lot, and only twenty percent of the time does actually lead up to war. So again, we're
not using the full power of the Markov model. We're not saying these transition
probabilities necessarily stay fixed. We're not worried about solving for the
equilibrium. All we're trying to do is just use this matrix to organize the data
in such a way that we can think more clearly about what's likely to happen. So
that's Markov processes. Markov process is a fixed set of states, fixed transition
probabilities. You can get from any one state to any other, and then you get an
equilbrium. So that equilibrium doesn't depend on where you start, it doesn't
depend on interventions. And it doesn't depend on history in any way. The model is
really powerful. And so if you wanna argue history matters. Or if you wanna argue
interventions matter. If someone gonna argue that this isn't a transition, that
this isn't a Markov process. Or that you've gotta argue that you're changing
the transition probabilities. Now that isn't impossible. And in fact, policies that
really make a difference, interventions that really make a difference, do change
transition probabilities. So what's really cool about this Markov process is,
is that it's given us this model. Is it's given us a new lens to look at the world.
When we think about it, I want to take this action. It's gonna make things
better. We have to be saying, it's changing the transition probabilities, not just
that it's changing the state. So if I make my students a little more alert for three
seconds by screaming or something, that's not going to change the long running
equilibrium of alert and bored students. If I change my teaching style, or if I add more
interesting examples in class or something, then it's possible I can change
those condition probabilities and end up with more alert students. We've also seen
in this last lecture that we don't even need to use the full Markov process model,
just the transition probabilities. Just that idea, the matrix of transition
probabilities, and we can find out all sorts of interesting things, like who
wrote a book? Is there likely to be more? Or is this medical treatment working?
So that framework, the transition probability framework and that matrix of
transition probabilities is a really powerful tool to keep in your pocket when
you confront some sort of dynamic process and you're trying to figure out, what do I think is
likely to happen. Thank you.