Tip:
Highlight text to annotate it
X
Craig Steinmaus: Okay. Let me grab my
pointer so I can point.
All right. Okay. So that's a cohort study. So
there's some aspect of time. Right? We're following people
over time. Whether it's we establish things in the past
retrospective cohort or whether we're establishing thing now
and foe lowing them forward. Cross sectional studies there's
not. Cross sectional studies is what's going on right now.
All right? So in this room who has the exposure of interest
and who has the disease of interest? If I'm studying benzene
and leukemia right now in this room in my group of study
subjects who is exposed to benzene right now and who has
leukemia right now.
So there's really no aspect of time. You are not
following people over a period of time. That's one major
aspect of cross sectional studies.
Again see this word? Now. All right. It's even in
red. You can't miss it.
Okay. So that's one aspect of cross sectional studies.
One key aspect. The second key aspect is you are looking at
disease or outcome prevalence. We're not looking at the number
of new case over a period of time like we are in cohort
studies. We're looking at how many people have disease now.
Okay. Now maybe you got it this morning. Maybe you got it a
week ago. But you still have it. So who has disease right
now? Not the number of new cases in the last one minute, but
who has the outcome status right now? See the difference? Now
versus following the number of new cases over a period of time.
So cross sectional studies we look at, remember I use
this word disease for outcome. We are always looking at some
outcome. So prevalence that's a key factor with cross
sectional studies. And the difference. You are going to want
to know the difference between prevalence and incidence.
Incidence is something, you guys have covered this. Incidence
is number of new cases over a period of time.
And that's the key. That period of time. Versus
prevalence is the number of people with the disease now divided
by the total number of people that we're looking at. Again,
the number of people in this room that have a cold right now.
Okay. So the difference between incidence and prevalence. And
again, cross sectional studies is all about prevalence. Okay.
Now let me go ahead and give you a nice easy example so I think
everybody can sort of understand. This easy example it's a
made up study. I made this study up in 2008. That's why we're
pretending we're in 2008 right now. And my question is, is
gender associated with back injuries.
So, in other words if you are male do you happen, is it
easier for you to injure your back? Do you have a higher rate
or risk or prevalence of back injuries as opposed to being
female? Let's say in my study we have a factory and 200
factory workers. Half are men and half are women. We have a
hundred men. What's our exposure variable or predictor
variable is gender. Let's say being male, does that increase
the risk of back injuries? If we were going to set up a two by
two table it would look like this. I'll write it big so
everybody can see.
We have an exposure variable. And it would be male and
female. And then whether you have a back injury or not.
Okay. Like that. That's basically what we're looking
at in this particular made up study.
Okay. So we went ahead and I started my factory
in 2000, in the year 2000. That's when I started my factory.
Over that time I had a hundred male workers. Over that time
since I started my factory five of them ended up with back
injuries. But, again, I'm doing a cross sectional study and
I'm doing my cross sectional study in 2008. Let's say
August 1st, 2008, I come into the factory and I'm doing my
study.
So, during that single point in time I don't have five
back injuries. This guy got better and this guy his back
injury hadn't started yet, 2008. I really only have these
three people that have back injuries. So, let's find, I need a
bigger chalkboard.
I want to keep that. What's my prevalence of back
injuries amongst males in this study?
>>>: Three.
Craig Steinmaus: Three divided by a hundred.
In 2008 I have three people that have an injured back. Doesn't
matter when they started or when they ended. Just so that they
had the back injury that day. Okay. So you're right. In
males three over 100. All right. Nice and simple. Right?
So here's my females. My hundred females. Four of
them ended up with a back injury over this sort of period of
time. Again, I came in in 2008 and what's my prevalence in
females?
>>>: Two.
Craig Steinmaus: That's males. That's females.
Okay. That's the prevalence. .03 and .02. Okay?
Prevalence. All right. Now there's a bunch of
different ways we can look at the outcome of cross sectional
studies. We've already calculated the prevalence in men and
calculated the prevalence in women. And some people use a
prevalence ratio as their outcome metric. That would be the
prevalence, I'm calling males my exposed group. I'm comparing
males to females. I'll call females my unexposed group. The
prevalence ratio is simple. It's this divided by that. It
would be this. Nice and simple. Right? Easy enough. The
prevalence in the exposed group divided by the prevalence in
the unexposed group. Very simple. And prevalence difference.
What's that?
The difference between the exposed group and the
unexposed group. So what would that number be?
>>>: 1/100.
Craig Steinmaus: Yeah. It would be this minus
this. Nice and simple. It's the difference between the two.
It's not the ratio but the difference.
Now, in my experience I don't see, I see a lot of
articles that give prevalence. I don't see the prevalence
ratio and the prevalence difference used all that much in my
experience. You may see them every once in a while. There's
the numbers. So in case you didn't quite get what we talked
about over the last couple of minutes you can look at those
numbers and you can figure out what those mean. What I usually
see in cross sectional studies is more people do the prevalence
odds ratio. Have you talked about odds ratios? You know odds
ratios usually are in case control studies but they are also in
cross sectional studies. When you are thinking odds ratios in
our two by two table, if you are going to be an epidemiologist
for the rest of your life learn to love the two by two table.
Whenever I'd start designing a study or start thinking about a
study I always start thinking about it in terms of a two by two
table. Where I have people with the outcome, people without
the outcome. People with the exposure of interest and people
without the exposure of interest. Eventually my studies may
get more complicated and I may not end up with these types of
analysis, but I always start here. It's a very simple way of
describing a study.
And I can't tell you enough if you are going to be an
epidemiologist that a key principle in epidemiology and this
will save you a lot of time and trouble and this will get you
grants.
These, this word. Keep it simple stupid. Okay? I'm
on a couple study sections. Several study sections which I'm
sure Dr. Reingold is as well. I'm sure he will tell you as
well people write research grants to do epidemiology studies.
When they get too complicated, us reviewers we get frustrated.
Guess what happens when us reviewers get frustrated? You don't
get your grants. You want to keep things simple. And one
simple way of describing an epidemiologic study is to put it in
terms of a two by two table. That's my unsolicited advice for
everybody.
So we got back injuries, we got males, we got females.
Okay. Prevalence odds ratio.
So prevalence odds ratio again is just like a case
control odds ratio. We can look at the odds of exposure or we
can look at the odds of exposure in cases or non-cases of
people with the outcome being cases. Cases. People without
the outcome of interest being non-cases.
Case control study these people are called controls.
We're not talking about case control studies. Cases,
non-cases. You put it in terms of that. The odds of exposure
is the number of exposed people divided by the number of
unexposed people. In whatever group you are looking at. If
you are looking at the cases then the odds of exposure among
the cases would be the number of cases with the exposure
divided by the number of cases without the exposure. The
number of people, number of cases. So we're talking about
these people with the exposure of interest that I'm calling
males divided by the number of people without the exposure of
interest. Again, the exposure of interest being male. Does
everybody get that? The odds of exposure in cases.
And it's the same thing with the odds of exposure in
the controls. In this group. I'm sorry, let's call them the
non-cases. People without the disease. It would be these
people divided by these people. All right?
So usually two by two table we have A, our B, our C and
our D.
So our prevalence odds ratio would be just like our
case control odds ratio. It would be the A divided by C
divided by the B divided by the D. The odds in these people
divided by the odds in these people. Does that make sense?
Okay.
Go like this if it doesn't make sense. All right.
Great. Okay.
I'm assuming you probably have this odds ratio slammed
down your throats, but there it is. All right.
So, oh. What is our odds ratio for our study? Okay.
So what's our odds? Let's say, go back to the slides.
Remember this? Remember we had a prevalence in males of three
over a hundred and females of two over a hundred. Three cases,
2 cases. So what's our A?
>>>: Three.
Craig Steinmaus: Three. And how many males did
we have?
>>>: 100.
Craig Steinmaus: A hundred. So what's our B?
>>>: 97.
Craig Steinmaus: How many cases do we have
amongst the females?
>>>: Two.
Craig Steinmaus: And how many females do we have?
>>>: 100.
Craig Steinmaus: A hundred. And so what's our D?
What? 98. So what's our prevalence odds ratio? It's our AD
divided by BC equals three times 98 divided by what? B times
C. And that equals whatever it equals. I'm not sure. 1.5 or
something like that. Okay. Simple. So that's the main,
that's the most, you can assess these cross sectional study
data a whole bunch of different ways. Most commonly it seems
like this is the outcome metric at least that I see most often
is this prevalence odds ratio. So very simple to calculate.
All right.
All right. So that's basically all you need to know.
So if you leave now, you're good to go. All right? But I'm
going to go ahead and give you a couple more examples. One
person is taking me up on it. I'll give you a couple of
examples. So this is an Art Reingold example that I stole from
him so everybody has to pay attention.
>>>: Can you reinforce what's the interpretation
of the odds ratio?
Craig Steinmaus: It's the same as the
interpretation of a rate ratio or a risk ratio or a case
control study odds ratio. When you are thinking about what
these things mean, think about what it is if there's no
association. If my exposure is completely unrelated to my
outcome, what am I going to see? What is my rate ratio going
to be my risk ratio going to be my odds ratio and my prevalence
odds ratio going to be? One. So basically we're trying to see
whether this is greater than one. Or less than one. Is it not
one? Is there an association? If there's no association it's
going to be pretty close to one. If there is an association it
will be. All right?
So, all those things are some indication of likelihood
of your outcome. Right? So let's say this is around 1.5. I
think it is 1.5. Now it just gives you an indication that your
outcome is more prevalent in your exposed versus your
unexposed. That's how I would look at it.
Okay. This is a study. Look who did it. Warren
Winkelstein. Everybody knows him? He was a professor here for
years and years. This was done quite a while ago when *** was
first becoming a major problem. So what he looked at was he
looked at men in San Francisco and he looked at was having ***
correlated with the number of *** partners that the men had
and he did it in a cross sectional study design. So what he
did was took men of this age group, 25 to 54 throughout San
Francisco and they tested *** in these men. And then asked
them about how many *** partners do you have? And here's
the number of partners. Here's the number of men. Here's the
number of men in each group. 17 had none. And 195 had greater
than 50 partners. I think over yeah, a two-year period. All
right. And then here's the number of these men who were ***
positive. Out of the 17 with no partners in the last 2 years 3
were *** positive.
So if you take three divided by 17 that's your
prevalence. Okay?
And 17.6 would be your prevalence and the prevalence
for each group here and the total prevalence for everybody
overall. So you can calculate a prevalence odds ratio on these
data. So why don't we go ahead and do that?
(Singing). Wasting your time, wasting your time. Here
we go.
All right. So let's calculate the prevalence odds
ratio comparing this group to this group. Okay. The lowest
group to the highest group. Okay. So this would be our
exposure. What's our exposure? Yeah. It would be the number
of partners. So 50. And what's our unexposed group? What?
0. Okay.
All right. And what's our outcome?
>>>: ***.
Craig Steinmaus: *** positive or negative. Okay.
***. Okay. So, what's our A? What? So we're looking at, oh,
I did this wrong, didn't I? How come nobody told me? It would
be better if we put 50 here and 0 here. Exposed, unexposed.
So our A would be what?
>>>: 138.
Craig Steinmaus: 138. And our B, let's see, so
our total in that group is what? 198.
>>>: 195.
Craig Steinmaus: 195. So our B would be what?
>>>: 57.
Craig Steinmaus: Okay. And our C, *** positive?
>>>: Three.
Craig Steinmaus: This is 17. So this would be
14. Right. Makes sense. So our prevalence odds ratio would
be our AD divided by BC equals 138 times 14 divided by 57 times
three. Right? Is that right?
>>>: Right.
Craig Steinmaus: I won't go through those numbers
but you get it. I've hit you over the head enough calculating
prevalence odds ratio. There's all the numbers right there.
Okay? Did I get them right? Yeah, I got them right. Okay.
That's that.
Okay. Perchlorate. Now I'm going to talk about my
research. Everybody likes talking about their own research.
Guess what, you get to sit through 15 minutes of my own
research. I study arsenic but I also study perchlorate. It's
used to make rocket fuel. The Department of Defense uses it a
lot. You can get exposed through it through industrial
contamination and also it's naturally occurring. Here's me in
northern Chile carrying a box of urine samples for testing.
We're going to test them for perchlorate. That's perchlorate.
An interesting thing happened with perchlorate. Probably the
biggest manufacturing plant in the United States outside of
Henderson, Nevada. And they were providing perchlorate to the
Department of Defense. They took their waste products and
didn't dispose of them properly. They dumped them in these
ponds next to the factory. It's very stable and sticks around.
What happened was it got into the local ground water and
eventually made its way into the Nevada wash. It doesn't have
a lot of water in it unless it rains. But when it rains water
gets into the Nevada Wash and goes into the Colorado River.
All this perchlorate that was dumped by this factory made its
way into the Colorado River. Guess what the Colorado River
supplies about 50 percent of the drinking water to southern
California. Who is from southern California? Did you drink
the water? No, it wasn't super high level so you don't need to
worry about it. Okay. But it was contaminated. Here's the
number of pounds. You can think almost a thousand pounds per
day got into the Colorado River from this one single source,
this one factory. Eventually it became a superfund site and
they started cleaning it up. The levels have dropped so the
water is much better. Before the water supply to half of
southern craft has these relatively high levels of perchlorate.
You can see some of the counties in southern California that
had perchlorate. There's some northern California counties as
well that were contaminated through other sites. Perchlorate
is an interesting compound because it can affect thyroid
hormone levels. We all need iodine. It's taken up by the
thyroid gland. We need iodine to make thyroid hormones. It
turns out perchlorate can block the thyroid from taking up
iodine. If you have too much perchlorate it blocks the iodine
uptake and your thyroid levels drop.
If you don't have enough iodine or you have too much
perchlorate you can get a goiter. Most importantly we need
thyroid hormone for brain development. Any fetuses or young
children in here? If you are a fetus or young child you need
it for your brain to develop normally. People have shown
there's some evidence even if you have these small drops in
thyroid hormone. If your mom has a small drop in thyroid
hormone during pregnancy that can cause a 5 or 10 point loss in
IQ. Thyroid hormone is important. We want to keep our iodine
levels high and we don't want anything blocking our iodine
uptake. Collaborator of mine, Ben Blount, actually he was a
student at UC Berkeley. He took data from NHANES. It's a big
huge survey that happens in the United States every two years.
They get like 10000 people. It's essentially a random sample.
It's not totally random. They like actually randomly select
clusters. But essentially it's a random sample of the United
States. Basically what they are getting in these 10000 people
in NHANES represents the entire United States population for
the most part.
Okay. So basically what they do is pick these areas to
do and come in with a trailer and they invite you into their
trailer and they'll ask you a whole bunch of questions, a
thousand questions. They'll do a medical examination where
they test all these different things on you. You blood
pressure, your Haight, your weight. All this different stuff
and they'll take a blood and urine sample. You're coming in
usually one day and they check a thousand different things on
you. All this stuff. So NHANES has a whole bunch of data on
these 10000 people every two years and guess what it's all
publicly available. If you are a young researcher around
looking to do a publication and don't want to spend five years
doing a research study go to NHANES there's a ton of data on
them. But a lot of that data is cross sectional. You are
going into that trailer and they are testing your blood sample
now. And a lot of exposure variables they are asking you about
now.
So, I'm not going to go through all this stuff. If you
get a chance look at all this stuff that NHANES has. It has a
ton of data. Thousands and thousands of variables they collect
on all these people. I won't go through all this. It's too
much. Basically they actually collected data on perchlorate
and so they took a urine level. Urine is the best way to
measure perchlorate. They took a urine sample that day and the
same day they took a blood sample of thyroid hormone. We could
look at perchlorate and its relationship to thyroid hormone.
They looked at perchlorate levels in your urine that day and
thyroid hormone levels that day in your blood. They didn't
follow people over time. Guess what, when they looked at that,
here's the results. Each dot is an individual person. For
example, this is a person, this person that's a perchlorate
level. T4 is the major form of thyroid hormone. Let's
consider that thyroid hormone. Each individual person. Have
you talked about regression coefficients?
Regression coefficients basically you can take all this
data and you can use SAS or STATA to draw the best fitting
line. The line that fits this data best. This data is all
over the place. It doesn't look like there's any association.
We expect perchlorate blocks iodine uptake. Our hypothesis we
would see a negative slope. It's a flat line. These people
with low perchlorate they have about this average thyroid
hormone levels as these people with high perchlorate. Ben was
smart. Why don't we look at susceptible groups. Women with
more thyroid disease than men. Their thyroids seem to be more
sensitive for whatever reason. I don't think anybody knows,
but they do. Maybe they are more susceptible to environmental
chemicals that affect the thyroid. He also said perchlorate
works by blocking iodine uptake. Then you throw perchlorate
into the mix, then you are probably really hurting. He looked
at these two susceptible groups. He looked at women who had
low levels of iodine. He plotted these data and told SAS or
STATA, SAS I think. There is kind of a negative slope there.
Results are all over the place. You expect that. There's a
lot of things that determine thyroid hormone. It's not just
perchlorate. There's a lot of things, age gender, race a whole
bunch of things. What you ate. So we expect a lot of scatter.
Overall average there was sort of a negative slope. So
increasing perchlorate here did seem to decrease thyroid
hormone level. There did seem to be an association.
Again, that is a cross evidence -- an example of a
cross sectional study. Didn't look at prevalence odds ratios,
looked at regression coefficients. What's your thyroid status
and what's your perchlorate level now. There's my study. I'll
skip that for now. We're running a little late.
All right. One big issue, perhaps the biggest
criticism of cross sectional studies is people say you can't
assess temporality. Temporality meaning did the exposure come
before the disease. As I think you have a lecture coming up on
causal inference if you haven't had it already. To think
whether this causes this. You want to know this came before
this.
Right? Because if this came after this it can't cause
this. There's no time machine. So your exposure if it's going
to cause your disease it has to come before your disease. And
since we're looking at exposure and disease at the same time,
we can't tell that exposure came before the disease.
So in this particular example, you know, we think that
increasing perchlorate decreased thyroid hormone levels, but
maybe people that have low thyroid hormone levels for whatever
reason maybe they happen to have more perchlorate exposure. We
can't tell which came first. Because of this issue of
temporality some people will say you can't use cross-sectional
studies to assess causal inference. This is just an
exploratory study. Guess who says that? Perchlorate industry
people. This study you did, you can't tell this happened or
this happened because it was cross sectional. And you'll read
that in a lot of your textbooks, they are just exploratory
studies, they don't give any evidence for causal inference.
Don't believe it. If you see that in a textbook, take your
permanent ink marker around blacken that out. Okay?
Because it is well known perchlorate used to be used as
a medication. All right. At super high doses. Thousands of
times higher than what we see in drinking water. It used to be
as a medication for people that had thyroid hormone levels that
were too high. They used to give you perchlorate to lower your
thyroid hormone levels. It's well documented this mechanism of
perchlorate I showed you here is very well documented. So we
know that things go this way.
We know that things go this way. Right? We have
pretty good evidence that it does. Those studies were done at
high exposures, not drinking water levels. There is no
evidence for this. There is no evidence if you are hypothyroid
somehow you are going to a water source and drink more
perchlorate contaminated water. That doesn't make any sense.
This makes sense. This makes absolutely no sense. So if you
use your intuition I think you could say, okay, temporality is
probably not a major issue here. Question.
>>>: So for this example that makes sense because
we have evidence that you are telling us exists. But for other
cross sectional studies we might not have that evidence we
still, is it still anything more than just exploratory?
Craig Steinmaus: Okay. So let's talk about that.
This is another cross sectional study where they looked at do
you exercise? Are you an exerciser? Right now are you an
exerciser? And right now do you have heart disease. And
here's the results of the study. You can see people that
didn't exercise had a greater prevalence of heart disease. But
the problem is, is that we can't, there's an issue of
temporality. In other words, you have higher heart disease, is
that because you didn't exercise or did it happen that you got
heart disease and so you couldn't exercise? You know, you were
too sick to exercise. So which happened? Was it you had heart
disease and therefore you couldn't exercise or was it that you
didn't exercise and then you got heart disease? So yes,
sometimes there's this issue of temporality.
And I guess what I'm saying is that you have to look at
the particular study. You can't just make these, well you can.
You shouldn't, my opinion is you shouldn't make blank
statements like cross sectional studies can't be used to assess
causality because you never know whether the exposure came
before the disease. Sometimes you have a very good indication
that the exposure most, most, most likely came before the
disease. But sometimes you just don't know. So it's the
individual study. So I would try to avoid these blanket
statements. And let me give you another example. Has anybody
heard of popcorn lung? Yeah.
So popcorn lung is there's this chemical used in the
butter flavoring that actually causes a very severe lung
disease. We don't have to worry about it if you eat butter
popcorn. The levels are pretty low. Apparently this one guy
he would take like 10 or 20 of those microwave popcorns every
day and what he would do is he'd put them in the microwave, get
them and open them up and put his mouth, he loved the butter
smell so much and just breathed in and out. He got popcorn
lung. So don't do that.
Usually it's just the people that work in the popcorn
factories. They are exposed to high levels. The chemical is
called diacetyl. They did a cross sectional study. They went
into the factory. Who worked in the diacetyl part of the
factory and who didn't and they checked lung disease or lung
function. They found people that worked in the diacetyl part
of the factory they had more lung symptoms and poorer lung
function than people that didn't work in that part of the
factory.
So you have cross sectional, temporality. Does
diacetyl lead to the popcorn lung or did somehow you having
popcorn lung cause you to then go work in the diacetyl part of
the factory? That second part is ridiculous. I think in that
case this cross sectional study was pretty good evidence of
causality. Again, you have to look at each individual study.
And assess each individual study. And again, try to avoid
these. I would recommend avoiding these sort of blanket
statements. All right.
Quick review. Some advantages and disadvantages of
cross sectional studies compared to the other study designs.
They are usually cheaper. Cohort study you have to start here
and hire all these researchers to follow your subjects over
time. That could take a lot of time and a lot of money and a
lot of effort. The nurses health study that's millions and
millions of dollars. You are following a hundred thousand
nurses over 10, 20, 30 years. That's tens of millions of
dollars to do that. Cross sectional studies boom, you are
doing stuff right now. They usually don't take years and years
so they are cheaper. Okay?
All right. Couple of problems is one, I don't know.
People like to get incidence rates. I'm not exactly sure why
rates are so important. I stole that from Art's slide so I
guess you should probably memorize that. Cross sectional
studies you can't get incidence rates. They are not good for
transitory outcomes. Outcomes that last very short time. I
showed you back injuries. Back injuries you can get them and a
lot of people don't get better within a minute or two. It
lasts a while. Compare that to an asthma attack. People get
an asthma attack and usually it's over in 30 minutes or so.
That's transitory. Think about this example I showed you with
the back injuries up here, the males. Think about if the back
injuries didn't last so long. Think about if they only lasted
30 minutes. Instead of being this long, this guy, instead of
being this long it was very short. It only lasted very short.
Look what would happen to your prevalence. You would miss a
lot of what was going on. You actually had these cases of
asthma. But you would miss them. That's a potential problem
of cross sectional studies.
>>>: Couldn't that be solved by asking people to
recall?
Craig Steinmaus: It would probably be best to do
a different sort of study design in that case. Then again I'm
talking about the classical definition of a cross sectional
study. And again, that is true, you could ask people how many
asthma attacks have you had over the last week? But the
classical cross sectional study it is a very short period of
time that you are assessing your outcome.
Okay. All right. So, that's it. This is my diacetyl
exposure. Again, I want to get back to sometimes you can use
them for assess causal interests. Us epidemiologists we have
our own opinions. That's my opinion. You may talk to other
people in the epidemiology department and they may have their
opinion. Any questions about cross sectional studies. Two
major characteristics, assessing things now and looking at
outcome prevalence. Those are the two major characteristics.
Any questions?
>>>: I have a question about the last statement.
Couldn't you phrase it differently like what's the likelihood
of poor lung function would increase your likelihood of
intaking diacetyl or something along those lines? Maybe you
are more susceptible to exposure.
Craig Steinmaus: Yeah, but there would still be a
diacetyl in, you are more susceptible because you are working
in the diacetyl? Again, the question is, which way are we
going? Is the diacetyl causing this? You don't like the word
cause, I'm going to use it anyways. Is the diacetyl causing
this or somehow is this causing this. Even if you say you are
more susceptible to poor lung function. Why would being more
susceptible to lung disease make you more susceptible to
diacetyl. What's the likelihood that something like this is
going to cause you to, okay, I'm working at a desk. I decided
I'm not going to work at a desk. I'm going to work in the
diacetyl. It's all about likelihood. I guess there's always a
remote possibility but I think it's pretty improbable.
Remember in epidemiology we never prove anything. But we prove
that smoking causes lung cancer. We have proven that. But
other than that, other than ***. Other than those we haven't
proved anything. It's all about likelihood. How likely does
this evidence suggest this causes this?
Any other questions? Yeah, we're done. No, we're not
done. You get meta analysis from me. Okay. That's right. Never mind. (Applause)