Tip:
Highlight text to annotate it
X
Introduction: Good afternoon, my name is Tatiana Pashkova-Balkenhol, I am librarian for instructional
design and I am honored to introduce my Longwood colleagues, Emily ***, librarian for instruction
and research services, Dr. Heather Lettner-Rust, Professor of Rhetoric and Professional Writing,
and Dr. Naomi Johnson, Professor of Communication Studies. Today we are going to discuss our
inter-disciplinary collaborative project, which was focused around the assessment of
student blogging, the center of which was a blog rubric, which we developed to be flexible
for customization and adaptation across various disciplines and levels, we will talk about
the methodology of our study, discuss the assessment results of the blog rubric and
we will conclude with pedagogical strategies for implementation of our blog rubric.
Over the last 10 years, educational blogs have been used across disciplines for various
pedagogical purposes in the higher education. Recent research documents many benefits of
using blogging as an instructional technology in teaching & learning. Academic blogs provide
a medium for reflection and engagement with the course content. Blogging also can serve
as a communication tool between students, allowing discussion of course content outside
of the classroom. Ultimately, research demonstrates the benefits of blogging as means for developing
a learning community that extends beyond the classroom. In addition to the pedagogical
values of course blogs promoting deeper levels of reflection, engaged dialogue, constructive
peer feedback and collaborative learning, our institutional course blogging experiences
suggest that this medium is particularly beneficial in helping students learn how to write in
public and for a public audience. Longwood Blogs is a web-based publishing platform,
developed and sponsored by the Janet D. Greenwood Library, Information & Instructional Technology
Services, and Public Relations. Officially launched in the spring of 2011, Longwood Blogs
originated as a brainchild of the University of Mary Washington blogging platform in 2007.
As of today, Longwood Blogs hosts 1,702 blog sites and 2,339 users and supports course
blogging across disciplines and showcases academic research projects, scholarships,
professional activities and talents. Since the official launch of 2011 of Longwood blogging
platform, librarians and teaching faculty have collected plenty of anecdotal evidence
about the impact of course blogging on student learning. However, we have not implemented
any formal assessment of Longwood Blogs in regards to student learning in the areas of
writing in public and for the public. So, we have launched over the fall of 2012 an
interdisciplinary collaborative project with the following goals:
1) To develop and test a blog rubric, flexible enough to be adapted across disciplines and
levels 2) To assess to what extent the blog rubric
got integrated into student learning 3) To evaluate how well students performed
on different levels by using the rubric (which I believe is our next step for our study and
today we are going to focus on the first two goals).
Now, I am going to turn over to Emily, who is going to provide us with the literature
review on assessment methods of student blogs, particularly grading rubrics.
End Tatiana Begin Emily
In the past several years, many in the academic community have started to address how to create
rubrics and evaluation guidelines for student blogs. I’m going to cover a couple concepts
today from the scholarly literature that discuss implementing rubrics in an institution and
creating a rubric for blogs. Please see the Powerpoint presentation for the complete citations.
In regards to creating a rubric to evaluate writing or speaking, Anson, Daniels, Flash,
and Housely Gaffney believe that rubrics should be constructed while considering the institution’s
‘local conditions’ – which are the linguistic, rhetorical, relational, and contextual characteristics
of writing and communication found in higher education. When evaluation tools reflect local
conditions, they incorporate disciplinary nuances that help instructors shape performance
criteria tailored to their own assignments which generic, campus-wide rubrics fail to
reflect. Similar to the point the authors argue in
this article, Dr. Lettner Rust and Dr. Johnson created a flexible, cross curricular blogging
rubric for both first year English composition students and upper level Communication Studies
students. This rubric was developed with each course goal in mind, with the finished product
a reflection of specific assignments throughout the semester.
A fundamental question many faculty have about assessing blogs is how to create evaluation
standards that reflect the dynamic nature of a blog. Olofsson, Lindberg, and Hague,
used Fostaty Young and Wilson’s ICE (Ideas, Connections, and Extension) model to create
a blogging assessment in an online higher education course. The authors applied this
model to their specific class needs in regards to blog posts, commenting, and peer-to-peer
evaluation. The blogging rubric that our group created
also incorporate the differences between the traditional ‘paper,’ and blogging, where
students can update posts and publish them in real time, add various multimedia elements
to posts, and moderate comments made by students, faculty, and members of the community outside
of Longwood. Now Naomi is going to tell about our methodology.
So, in order to find out how students in different disciplines at different levels would be able
to use this rubric, we decided to look at two different courses. one of the courses
is an English composition course that was a 100 level, and the other course was a 300
level communication theory course. So, we have very different course levels and content
since we're really interested in finding out can the same rubric
be adapted to go across different types of disciplines the methodology that we used,
we surveyed these students and we had about fifty percent response rate the students were
surveyed on three different areas first they were surveyed on to what
extent meaning how thoroughly did they actually read the rubric and then also along with that
we looked at when did they read that rubric. So, did they do it at the beginning, before
they started writing, during the writing process or at the conclusion or
a combination of all three? The second area that we looked at in the survey was assessing
what the students thought of the rubric where the requirements clear and did they actually
help them write And the third area that we evaluated was looking
at how blogging overall influenced student learning in the course, so we wanted to look
at how did it help them in that particular class but also did they extend beyond that
to learning outside the classroom to apply this knowledge in other areas of their life
from classrooms to personal life So, those were the three areas that we surveyed in our
work and now i'm going to turn it over to dr Heather Lettner-Rust and she's going to
talk about these results of the survey
Our questions fell into three areas of inquiry—student use of the rubric, student perception of the
rubric, and students perception of the LEE method of analysis. What you will see is a
favorable response by student in all three areas. The more incremental division of ratings
can be seen in a closer viewing of the slides. Today, I’ll cover the areas of larger favorable
responses. Our first question was whether the students
had viewed the rubric at all. An overwhelming percentage had viewed the slide at some point
during the course. Only 2 percent said no. and 4% didn’t answer the question.
The next sections parse when and how the students viewed the blog. And I’m going to combine
some of the bigger response rates.
We asked what way best described how they viewed the rubric and the largest percentage,
a total of 75, viewed it before and during the creation of their blog. This suggests
the rubric was a guide to their writing and design of the blog rather than something extra
or inconsequential to their learning.
We asked them how they viewed the rubric before the creation of their blog, and 54% scanned
it for general ideas, read some of it with care, or read it all carefully
During their writing, 50% of them scanned it for general ideas, read some of it with
care, or read it all carefully
After they finished the blog, the percentage increased from the last question and 62% of
them scanned it for general ideas, read some of it with care or read it all carefully.
The next series asks about students’ perception of the rubric. Black and light blue in the
bar graph are favorable responses of agreeing and strongly agreeing. As you can see here
by looking at the black part of the bar graph and above, a majority found the rubric fair,
with clear language, and having criteria related to important features.
The responses here are not as favorable when compared to the last categories, but still
a majority agreed or strongly agreed with each category on this graph. And I’ll read
them in order: They agreed that the rubric influenced their work in that they wrote clearly
and concisely, helped them with proofreading and editing, organizing their work, and finally
helped them to meet the assignment requirements.
This question asked them about the blog specifically as we wondered about their understanding of
the blog’s broader purposes. Of the four questions we asked, 2 showed the most favorable
response: students agreed that the blog helped them build a digital identity and that the
blog was important for their professional growth.
The final two slides ask students about the LEE method which was a way to name the rhetorical
moves in their analysis. Label, explain, example. We asked if they used it in other classes.
42% said they hadn’t which is understandable because a full third of the participants were
first year students. The rest were upper classmen. A little under 60% said they had used it in
other classes. This suggests the method translates to other subjects or subsets of their discipline.
And finally, we asked the students if the LEE method helped them in their current class.
A majority found the LEE method helpful in each facet of their work. It helped them to
write clearly and concisely, to understand some of the course material, to organize the
content of their writing, and to present analysis in other assignments.
And now to Dr. Naomi Johnson to present our conclusions from this study.
now that you've heard a little bit about what we found out with the results of our study
we want to talk about how you can actually adapt this rubric for use in your own class
because we did create this so that it could be an interdisciplinary tool that regardless
of what field you're teaching in you would be able to use this rubric if you
want a electronic copy of this rubric we would be happy to
provide that for you just contact us and we'll be happy to mail it to you. You will notice
that some of the writing in the rubric is in blue what you're seeing here is actually
the very first category of the rubric there are several categories that correspond with
different elements of the blog so we're going to start out looking at this piece-by-piece
and i want to point out that wherever you see blue writing is something that we believed
you might want to customize this to your particular course goals and needs so this is a very nice
template for you so that you can easily adapt it to your own class at the beginning, i also
want to point out that the goal in writing a blog is different than writing a paper and
that you have to consider a broader audience than just your professor this is a real a
real advantage for you as a professor because students have an intrinsic desire to go perhaps
a little bit further than they would in a paper in a paper they know that you as a professor
will see it and maybe some other classmates and for some students that may not be as much
of a motivation as thinking about this is a public document where i can and demonstrate
my ability to analyze using material from our discipline to future employers to potential
graduate schools and so by gearing this toward an audience that students can see goes beyond
just the immediate understanding of the class material on to something that will help them
in the future as they graduate has the ability to create stronger work which of course, makes
it much easier for you as a grader and much more gratifying as well so as we go through
each of these categories just bear in mind that the audience is a broad public audience
beyond just the classroom so this should been written for someone who is educated but not
necessarily an expert in this field, so the very beginning the rubric which were looking
at here is the introduction this is of course if you're thinking about a broader audience
just like you might not read every single news article that you see or every single
internet post you see you want to train the students to be thinking about trying to draw
a future graduate school evaluator your trying to draw in your interviewer where he's going
to hire you what are you going to do to make them and keep reading this why should they
care developing the introduction... you'll evaluate that's the first element the second
element of the rubric is to put it in context so we have that labeled building curiosity
in context and this is where you can choose as an instructor how you want them to develop
this...basically how is this phenomenon that they're writing about... how was it occurring
in the world why is it important in some cases for my classes i will have them draw from
news sources and from other popular press sources in order to contextualize this and
explain why it's important next element of the rubric, which would be a corresponding
part of their blog this and they considered the heart of the blog this is where they will
be able to demonstrate their ability to take whatever class material that you have decided
to emphasize as the focus of this blog assignment but they will be able to take these theoretical
constructs and break it down into a form of analysis that again, an outside audience can
understand but what that does for you as an instructor is it gives you the ability to
know that yes they really do truly understand this because if you can explain this theoretical
construct and give examples in a way that someone who's not expert in the field can
actually understand it then i, as the instructor know that you truly understand it but for
the student again, it gives them that ability to think about how can i use my ability to
analyze using discipline specific knowledge to show that i am someone that you might want
to hire or allow into your graduate school program this LEE, the label, explain, example
method that that is definitely something that you can adapt into your own classes the final
element of writing of the final component that would be in the blog would be the inclusion
and this is again keeping in mind there's an outside audience.
now that i have told you about how the situation is occurring, i contextualized it. i have
analyzed it using discipline specific knowledge i'm going to conclude with something that
tells you again why does this matter why should we care what can we draw from this there are
three other elements of the rubric beyond the actual sections of the blog so what we've
done so far is we have provided a template a model through this rubric for the students
so that they know what different elements of the blog should be in there beyond that,
we're looking at some of the ways the ways that you make it more appropriate for the
blogging platform itself so part of that is digitally have they represented this you can
decide again this is customizable so do you want to have particular graphs or charts do
you want pictures. How is it that digitally, are we are making this appealing to the audience
and then also things as well as the blogging format has to have smaller paragraphs it tends
to have more headings and sub headings it should be easily linkable people should be
able to scan it and be able to quickly tell what the main points are and not have giant
blocks of text everywhere? A second key area that you can evaluate on, of course, is the
sources, on the way they are cited again this is customizable for you, you can decide what
type of sources and how many and then finally overall how do you evaluate this blog, looking
at it as a whole? There is an area that you might choose to
create deductions if there's something that you think well, this is so standard you shouldn't
earn points for it, for example if you think everything should be spelled correctly written
grammatically correct sentences and then maybe you think well that's not something necessary
that you should earn points for but if you're not performing at that basic college-level
there'll be points deducted. So, as you can tell this rubric is very customizable so that
you can adapt it to your own specific discipline's needs for the level of students that you're
looking at and it's something that I hope you're able to use so that the students can
use it as a tool to create better blogs which is certainly what our own students that we
surveyed demonstrated that they felt that it was doing it also help you as an instructor
to ensure that you are fairly and consistently evaluating your work and making clear that
your expectations were for the students. In conclusion, the purpose of this study leads
to create an interdisciplinary rubric for students that would be beneficial to both
the students and the instructor . And what we found was that this gives the students
the tool to better understand class material and better understand how to write in a blogging
platform. I hope that you will join us in developing a more interdisciplinary work so
that we can cross those silos and gain from each other's knowledge. And once again, I
would like to thank the collaborating team who put the rubric together and did the research:
Dr Heather Lettner-Rust, Emily ***, Tatiana Pashcova-Balkenhol, and myself Dr. Naomi Johnson,
thank you very much!