Tip:
Highlight text to annotate it
X
It is a large undertaking and it’s a significant one.
There are a number of different players coming from the
institutions, from the governments, from the quality
agencies, from the disciplines having the experts involved.
It is a large scale operation but all of the players, all
of the stakeholders are committed to the ideas. There are a
number of challenges that are occurring and we're coming
upon as we go through the feasibility study trying to
understand how best to focus our attentions and trying to
understand how best to implement some of the ideas that we
have about how can we ascertain international understanding
of learning outcomes but the dedication of the people is
there and the desire for the information is there and it’s
not just a political activity, the focus or the unit of
analysis of AHELO is actually at the institutional level.
The purpose really is to help the institutions understand
what their colleagues are doing in different parts of the
world and help them to understand where some of their
strengths and perhaps where some of their weaknesses lie.
It’s not a ranking activity and because of that the buy in
and the uptake of the ideas is quite strongly resonating
throughout the institutions; so certainly there are some
challenges with it both in the direction in which it’s
taking, the specific unit of analysis, how the data is
presented, those are some of the issues that we’re still
going through similarly when it comes to the
implementation. You know implementing the same test in the
same general way around the world is not always an easy
activity. There are certain things that we want to do to
ensure that students are given the same opportunities and
that the data is collected in a similar way that are
hindered by very basic things. Sometimes it’s a matter of
whether or not the institution has broadband or suitable
broadband. Sometimes it’s whether or not an institution has
enough computers for students to actually write the tests.
So in one country for example their institutions didn’t
have enough computers so there was a van with computer
laptops that went to the various institutions and these
were the sorts of implementation things that you don’t
necessarily think about or understand until you’re actually
in that field work level. So those are the sorts of things
that are coming about now and that we’re starting to
realise need to be dealt with and need to be considered
when thinking about the feasibility of actually
implementing something like this on a regular basis.
Sure, well, I think that in an undergraduate education, if
we speak specifically about the universities, there are
certain expectations that all students graduating will
have. Those are very commonly understood around the world.
Many different countries and institutions have different
verbiage for what they actually use to describe it but
there are basically about six categories of expectations of
those generic skills. They tend to be in; basic knowledge
base; communication; social responsibility; teamwork, we
well in Ontario, we have that falling into our
communication, those sorts of activities; critical and
creative thinking certainly; and then there’s practice and
methods. So those are the sort of generic skills that we
think that students, regardless of their knowledge base,
really should be coming out with. And the agreement across
sectors and in around the world is impressive. It really is
impressive how nicely they map to each other. So those are
the things that we impart and we provide to our students
but we don’t always do so in a very directed or very
purposeful way. We assume that by having students work in
groups, they’re going to gain teamwork experience and
become good leaders or that by offering the opportunity to
give a presentation they’re going to actually become good presenters
and have that as a skill when they complete. So we’re very, so
around the world there is a lot of work going into
authentically assessing these skills and that tends to
happen within the classroom. The individual instructor, the
individual course designer, really is the best judge of how
those generic skills are imparted and how well the student
is doing with them. But on the flip side we have the
discipline specific skills which is absolutely within the
domain of the faculty members and the discipline within
which they operate but again there’s a good deal of
similarity around the world about what the expectations for
discipline are. That for example, for the AHELO project
with all of the countries that we're involved in both the
engineering, and in the civil engineering and the economic
strands, the expectations on student's basis of knowledge
were absolutely the same. It was really incredible to see
how quickly the experts came to agreement on what the
expectations were. The assessments of the engineering and
the economics tests were not explicitly on content
knowledge, it was above content knowledge. So it was asking
the students to apply what they already know, to use their
critical thinking skills, their written language skills, to
use their numeracies skills, to actually apply a practical
problem so looking for that above content and because of
that it’s possible then to really assess both their
generically abilities as well as their ability to apply
that to a content specific area. So, in as much as it’s
difficult over all to assess generic skills in perhaps a
standardised way, within a discipline specific assessment
you can approach those more generic skills and I think that
was one of the findings of some of the work that I’ve been
involved in. Nonetheless, there is an absolute interest in
harnessing those generic skills and really seeing if we can
figure out very straight-forward, appropriate ways to test
those generic skills across the different disciplines.
When it comes to the actual assessment part, we’re getting
pretty good at asking the right questions and garnering the
right information from students to make some pretty sound,
reliable and valid judgments about the student capacities.
They are common around the world, they provide suitable
information that is able to be linked with other existing
information that we already have.
You can link some of the generic skills assessments to
information that you have from course grades, you can look
at their entering grades, you can look at some of the
characteristics and information that you would find in
something like an Auzie or a Nessie. The information from
the generic skills assessments tend to link quite closely,
for example, students perception of their own abilities. If
they say they do well they tend to actually be some of the
top performers in either their classes or in assignments
and the information that we gathering from these large
scale assessments corroborates that so if they say they're
good students well they tend to actually be the top
performers. Similarly student engagement if they say they
are engaged in their institution, if they’re engaged
learners they tend to do better on these tests which
corroborates the other information that we have suggesting
that the more engaged the students, the better their
results are. So in that sort of way we can sort of validate
the results that we are getting from these large scale
assessments. But it is an assessment and it’s only one
aspect and there’s so many elements of student learning
that aren’t able to be captured in that type of assessment
so things like teamwork, things like presentation skills,
things like global awareness and social responsibility and
that’s where that demonstration of achievement and
demonstration of learning outcomes becomes really, really
important and it is through elements -learning artefacts
almost is what you can call them. So if a student performs
an activity, be it a presentation that’s videoed or a
report that they can share and demonstrate that they have
met a learning outcome through that activity, that is a way
of not necessarily giving a mark for their achievement but
it’s a way of demonstrating that they have exercised that
learning outcome, that they’ve worked with that learning
outcome and so, these are important elements for things
like E- portfolios or badges, the idea of the badges, or
learning passports. There are ways in which students can
demonstrate to themselves and demonstrate to employers or
demonstrate to other institutions for example if they’re
interested in going onto graduate school or a different
program, there are ways of providing information on what
they have accomplished and I think that’s really an
important element. It’s not just about the measurement of
activities it’s also about the demonstration of success.
I would certainly like to thank you very, very much for
having me here, it’s been very interesting to learn from
the other experts that have been here this week and also to
engage with the faculty members coming from Deakin as well as
institutions from around the country. It’s been a learning
opportunity for me to understand what’s happening within
your institutions and know that you’re struggling with many
of the issues that we’re struggling with. It really is a
global problem, I wouldn’t say a problem, but it's a global
issue and something that we can absolutely learn from each
other and support each other in. So I thank you very much
for the opportunity.