Tip:
Highlight text to annotate it
X
>>IAN WILKINSON: Okay, my name is Ian Wilkinson.
I'm with the Department of Teaching and Learning, and it's great
coming last and trying to think of something new to say.
[audience laughing]
As I've been sitting here wondering what you haven't heard.
So, I'll keep this very brief.
I don't have much to add beyond what people have already said.
I do research in reading, and in particular, I do research on
classroom discussions in elementary school and classroom
discussions about text, high-level thinking and comprehension.
And, I've had a couple of grads from the institute of education
sciences, and last night in preparation for this, I looked through the
OFA that I applied to, and there was no mention of a logic model,
but they all mentioned that they wanted to see a theory of change -
a theory of change, which is another code word for logic model.
And, I've always struggled with logic models because, as you've
picked up already, they come out of program evaluation, program management.
And, I'm not into program management.
I'm not particularly into program evaluation, so I've always struggled with this,
and you'll see that in a couple of examples I'm going to show you.
So, the logic models I've used in my proposals embody the theory of change.
And, because of that, the models I use are theoretically informed.
I know you've heard the word theory once today,
have used the word theoretical.
So, I like to make my models inform my theory as well as
the pragmatics of the inputs and outputs.
So, I'm going to show you a couple models, very simple ones.
The first one is a model on a grant where we're concerned with
promoting high-level reading comprehension with a particular
intervention I was developing called quality talk.
So, this was a reading and writing grant.
It was a goal to development, an innovation grant.
Our goal was to develop the intervention from scratch basically.
The other thing I want to mention is, because what I do is informed
by theory, the theory I use tells me the context is important -
the context in which the intervention occurs is important.
And, so, as soon as you realize the context is important, the culture-
How do you put that on a graph? Where would it go?
How would you put that in a figure? Where would you put the context?
It's sort of like, it's everywhere. So, we'll see who responded to that.
So, here's one logic model we developed for this quality talk intervention.
So, you can see that in this particular quality talk model,
there were 1, 2, 3, 4 main parts. Let me come to the output.
The outcome that I was interested in was high-level comprehension of text.
So, my intervention was built around promoting high-level comprehension of text.
And, the components were
:Instructional framework for what this should look like in the classroom.
When it should occur... (indistinguishable)
Given this instructional frame or context, there were certain talk
tools and signs, certain discourse tools that we were interested in looking at.
I knew that the teacher had to model and scaffold those discourse
tools and signs, so rather than put this in a linear fashion,
I put teacher modeling and scaffolding as a mediated factor between context,
frame, and outcome - to promote the discourse tools and signs. So, that's 1, 2, 3.
But, here's my - I knew that all of this intervention had to take place
within a culture of dialogic inquiry, so I sort of put that as a
pervasive bar on which everything else is captured because that
culture of dialogic inquiry was important, not just to boost up the
whole intervention, but to work in context with my instructional
framework for everything else to work.
So this was our attempt at reflecting how to better the intervention
within a particular context for the particular culture of dialogic inquiry.
So, I actually like that quality of the logic model, and it seems to
serve the purpose for our proposal.
It seems to work pretty well - not particularly complicated, but the
inputs and outputs were easily represented, as well as these more
amorphous things like culture and teacher modeling and scaffolding.
The project I'm working on right now is a little bit different.
It's a professional development project where we're developing a
program of professional development in something that we call
dialogic teaching, and our goal is to help teachers conduct classroom
discussions about text to improve students total ability.
So, this is quite new to me. I've never worked in professional development before.
It was interesting. And, this was in the Effective Teaching program
within the Institute of Educational Sciences.
We have a very simple logic model much like the ones we've seen earlier today.
So, this is our logic model.
We're teaching professional development dialogic teaching for discussion.
We have a set of inputs, and we characterize these as 5 modules for
teachers in our professional development program,
5 different components to professionally develop
:Discourse coaching, assessment, interactive discussions,
analyzing discourse, dialogic teaching.
We have a set of outcomes premised on the Common Core State Standards.
Our outcomes are literacy, generating arguments with good reasons
and evidence, comprehending arguments - So, those are our outcomes.
Here's our inputs over here.
Given that professional development provides conceptual support
for teachers to engage in dialogic teaching, and the features of
dialogic teaching, they in turn helped inform teachers how to
conduct discussions about text with their students.
And, then we had a theory about which the discussions about text
would enhance students individual literacy outcomes.
Now, again, context played a role here.
We weren't quite sure how to represent it, but we knew that this
whole process, for it to work - We had a theory about teachers
epistemology, teachers imbued with a natural knowledge.
We thought that if teachers would be successful in engaging with
this professional development program and really taking hold of it,
they would have to take up what we call an evaluative epistemology,
a view of knowledge that suggests that we can reason reasonably
effectively about certain issues using talk.
So, supporting this whole process is a progressive shift from an
absolutist epistemology to this evaluative epistemology.
And we have measures in our project to assess shifts in
epistemology over time to see if our [indistinguishable] is caught in
that shift and see if the shift is attainable.
So that was the challenge for us to represent that in a pervasive
context, somewhat similar to what I showed you previously.
Interestingly enough, when we submitted this proposal, which was
funded, we got back several queries and one of the queries was,
"Please tell us more about your theory by which [indistinguishable] by discussion.
There's a little bit of a black box here that we missed, we didn't really
fill in that theory by which discussions foster individual learning outcomes.
I think it's refreshing for IES to ask us for a theory.
[AUDIENCE LAUGHTER]
And so he really went to town on that theory.
So that black box, we probably should've had that in there.
We should've somehow represented that better,
but in the end we got the grant so I guess none of this matters.
[AUDIENCE LAUGHTER]
Just to close, I did some homework last night on logic models and a
Google search turned up this talk by Mark Lipsey.
Some of you might know him, he's a very well-known statistician and
methodologist who works a lot with IES.
Here's a talk which he gave for IES on evaluating stages for better
interventions and in this powerpoint he had a logic model.
I thought, "Oh, if I really want to know what a logic model is,
maybe I should look at this." This is his logic model for pre-K literacy intervention.
Right there, with his target population of four year olds.
Here's the set of proximal outcomes, attitudes,
literacy scores, school behavior, and distal outcomes.
Increased school readiness and greater learning gains in K.
So I actually thought that's not much more fancy than what I had so
maybe I was okay but then he does get more specific.
He starts naming the various variables in this attachment to his model.
So maybe that's a sort of thing to do rather like the models we saw
from Jerry and Emily; more fully fleshed out like that.
Are there problems in doing a logic model for developing grants
through IES's by definition of developing,
You are going though an iterative process of developing a model and it's changing over time.
So to pre-specify a logic model seems kind of weird.
And so you know, for instance a grant we're working on now under a
professional development project. We're changing as we go,
for instance, in modules. They're no longer there as if that was a stupid idea.
And so in the second year into the project, there were no more modules.
So, being iterative and developmental but having a logic model.
It's not... It doesn't quite. fit well. It fits better for program evaluation.
[Applause]