Transcript A Catalyst for Collaboration: Supporting Teaching with Technology

The Catalyst Initiative:
Using Assessment to Drive
Innovation in Teaching & learning
with Technology
Tom Lewis, Director
Ed-Tech Development Group
Educational Partnerships & Learning Technologies
Scott Macklin, Director
Program for Educational Transformation
through Technology
How do you scale beyond
the early adopters to
reach wary adopters and
encourage innovation?
Scaling Innovation with the
4-Tiered Catalyst Initiative
Lesson Learned:
Scale support through
partnerships, making
disparate resources and
innovative practices
visible and available to all.
http://catalyst.washington.edu
The Catalyst Initiative
On an average day in Autumn 2002,
instructors viewed 5000 pages of
Catalyst content every day.
8557 instructors have created 18,592
implementations of Catalyst Web Tools.
Catalyst Web Tools were used for 66,000
learning activities during Autumn 2002.
Catalyst Web
Site
Catalyst
Profiles . . .
Stories
of Innovation
Teaching
Guides start
with your
teaching goals
Action Plans
help you use
technology
The How-To
section has
detailed
instructions for
specific tasks
The learning
section provides
information
about
workshops and
clinics
Participatory Design &
Catalyst Web Tools
Lesson Learned:
Create tools that meet pedagogical
needs expressed by faculty and
constantly engage with faculty to
make sure the tools remain current
and easy to use.
Collaborative Development
Assessing what the educator wants is
central:
• Get feedback at every step of design.
• Process is more important than
product.
• Direct communication between
developers and educator.
• Make feedback and interaction easy.
• Value educator’s ideas and time.
SimpleSite
• Create Course
Web pages
without HTML or
an editor
• Publish Web
pages without
FTP
• Keeps usage
statistics for you
WebQ
• Create online
surveys,
quizzes &
questionnaires
• Download
data, or look at
it on the Web
• Many security
options for
password
protecting your
data
• Approved by
UW Human
Subjects
Division
Peer Review
•
•
•
•
•
Create a forum for to review
each others’ documents
Encourage group projects with
the new group features
Supports any kind of Web
content: (text, graphics, sounds,
movies)
Comment on sentences,
paragraphs or entire document
Complete participant reports
accessible to instructor
UMail
• An anonymous
email feedback
tool
• Messages
come to your
regular email
inbox
E-Submit
• Students can
turn in
homework
online
• Saves time and
frustration with
email
attachments
• Students receive
a digital
“receipt”
EPost
• Create an
online
discussion
board
• Many sort
options: by
thread,
subject,
author, and
date
• Generates
“participation
reports”
QuickPoll
•
•
•
•
•
Create 1-question
surveys for your Web
site
Results tabulated
automatically
Numeric and graphic
displays
Many submission
controls available
Can be linked to EPost
Partnerships as a support strategy
to bring easy to use, flexible &
campus-wide support.
Lesson Learned:
Make sure that all the units that
should be shaping technology
innovation are shaping innovation.
Portfolio NEW!
Resources
• Catalyst group (under Vice-Provost of
Educational Partnerships & Learning
Technologies) supplied main funding
• Student Technology Fee committee gave
financial support.
• Program for Educational Transformation
through Technology provided financial and
assessment support.
• Computing & Communications furnished
server space and maintenance
• Office of Undergraduate Education offered
the Freshman Interest Group program as a
test-bed
Project Assumptions
• Tool cannot be built without input from
students, instructors, and academic and
career advisers
• Tool should be flexible—students should be
able to use it for classes, extra-curricular
activities, job searches, or personal
reflection
• Tool should be useful for students in all
colleges and schools
How did we build it?
•
•
•
•
Asked everyone on campus who has ever
uttered, or is likely to utter the word
“Portfolio”: 200 +
Looked at other universities’ tools and
commercial products
Read scholarly articles on the uses of
portfolios in education
Conducted focus groups and usability tests
with UW instructors and students
Basic Decisions:
• Students should be able to collect anything
they want to and have exclusive access to
those collections
• Students should be able to publish many
different portfolios of their work on the Web
• Instructors and advisors should be able to
easily lead students through the process of
building certain types of portfolios, but
students still own these portfolios
What is a portfolio tool?
• There is a distinction between collecting
artifacts, reflecting upon them, and
thoughtfully presenting them in a portfolio
• Students will be collecting artifacts
throughout their academic careers and
should have exclusive access to their
collections
• Using portfolios for reflection is a process
that is important in all phases of an
academic career, not just at the end of
senior year
What is a portfolio tool (cont.)?
• Instructors, advisers, and peers should
collaborate with students to help them
present artifacts in a reflective way
• Students should be able to present a variety
of portfolios for different objectives:
completing coursework or demonstrating
academic, personal, or career development
Instructors need help in
effectively integrating
technologies into
teaching.
Lesson learned:
Need research and assessment to
inform the development of learning
technologies and teching strategies.
The process
1. which in turn inform the
design of appropriate tools.
2. These new tools are
integrated into educational
settings,
3. in which new studies are
conducted to understand how
these new tools have
affected learning and
learners.
4. Perform studies in real
settings to investigate what
is learned, who learns, and
how it happens,
The FIG Program and Portfolios:
• Pilot project in Autumn 2001 with three
Freshman Interest Group (FIG) sections
(~60 students).
• Development of the Catalyst Portfolio
Tool in 2001-2002.
• Introduction of the tool into the FIG
program in Autumn 2002, with a study by
PETTT researchers.
Our Goals for Autumn 2002:
• Introduce the concept of portfolios to
3,200 freshman students.
• Within the goals of the existing University
Community class, encourage reflective
learning.
• Learn lessons that can guide future
portfolio projects and inform future
revisions of the tool.
What was our experiment?
Collect data on the use of the tool by FIG
leaders and students through:
• Surveys
• Focus groups
• Classroom observation
• Examination of student portfolios
Study design:
• 162 Freshman Interest Groups
• 162 FIG leaders, each with a class of 1025 students
• Special group: 20 FIG leaders, paid to
give regular, individualized feedback to
students
Study questions:
1) What method(s) of instruction help
students feel most competent using
Portfolio?
2) How can Portfolio be best integrated
into the GENST 199 curriculum?
3) What instructional supports (in-class
discussion, feedback, prompts within
tool) might best support reflective
thinking?
Data collection:
1)
2)
3)
4)
First day classroom observations
Student surveys
FIG leader surveys
Focus groups with students and FIG
leaders
5) Classroom observations & student
interviews
6) Review of student portfolios
Three themes from preliminary analysis:
• Students found Portfolio easy and convenient to
use. However, they largely perceived answering
the questions as busywork.
• FIG leaders' attitude toward the tool and the work
affected students' response to the tool and to
class.
• Feedback was important--Students wanted to
know that someone was reading what they wrote.
These are issues to consider as we head towards a
redesign and make recommendation for how to
use Portfolio. NOT reasons to abandon the
project.
Quotes…
“I feel it’s really fun to me, I mean maybe
it’s weird but I think it’s almost like a
penpal letter kind of to our FIG leader
and we can answer the questions and
sometimes they’re really in-depth and we
actually have to think about what we
think about life and on these certain
subjects, we have to think about what
our beliefs are, and I think that’s helped
me a lot to really figure out what I
believe and why I believe it, and apply
that to the discussion that we’ve had in
class and stuff.”
-FIG Student
“It’s nice when they return it and they
add comments so you know that they’re
actually looking at it.” (student)
-FIG Student
“If the FIG leaders did give feedback… it
does kind of make it more interesting and
you do feel like your FIG leader is paying
a little bit of attention to you. It might
even create discussions that you could
have in your FIG class”.
-FIG Student
“[Portfolio and the feedback function]
helps you know more about your
students, share and get ideas”.
-FIG leader
“Because I comment so much, they
[students] see me as a friend and come
talk to me. I learned a lot from them.”
-FIG leader
Building Catalyst through
Co-Branding
Lesson Learned:
Renew and refresh anytime-anywhere
technology support resources by cobranding and co-developing them
with campus teaching practitioners.
Students learn through portfolios
when they…
• assemble a collection of relevant artifacts
• select items from that collection for a
specific purpose
• reflect on the selected items and/or
process of selection
• design the presentation of their portfolio
for a particular audience/purpose
• receive feedback on the quality of their
selection, reflection, and/or design
Considerations for teaching with
portfolios…
•
•
•
•
specific purpose/audience
assignments to produce relevant artifacts
guidelines for item selection
guidelines for and examples of “good”
reflection
• guidelines for and examples of “good”
design
• time for feedback on quality of selection,
reflection, and/or design
More considerations for teaching…
Evaluation
• individual items?
• whole?
• reflection?
• establish clear criteria & SHARE with
students ahead of time.
• will the evaluation count toward the grade?
• if students don’t have opportunity to learn
from feedback, a grade isn’t a measure of
what they learned—only innate ability.
Extending Innovation
through Assessment
Lesson Learned:
Reach out to instructors
and students in as many
ways and as often as
possible.
PETTT Educational Technology Surveys
Faculty, Spring 2001
2000 respondents
Students, Fall 2002
2000 respondents
Technology Adoption: Student Driven
•20% of college students began using a
computer between 5 and 8 years old.
•By the time they are 16-18, all had used
a computer.
•80% of UW undergraduates use a
computer daily.
Students…
•72% of students think all courses
should have a Web site
•Students particularly want online:
course syllabi (81%)
grading criteria (79%)
problem sets or exercises (78.6%)
links to electronic course reserves
(72%)
•7% of students stated that educational
technology is used consistently through
UW
“I’m graduating now and I’m pretty upset
with how little I know about technology, I
really wish that I was more confident and
knew more after four years of my education at
a university. A lot of it is probably
independent work that you can do to learn it
but we’re busy enough with jobs and school
that I just really haven’t felt like I had the
time to actually independently learn it…it
would have been really helpful to have not
been expected to know it but to have actually
been taught it.”
-Undergraduate Student
Which of the following concern faculty when
using information technology to do university
work?
60
40
30
20
10
of
La
ck
dw
ar
e
Ha
r
ni
ng
ea
r
St
ud
en
tl
nt
ive
In
ce
Sk
ill
of
La
ck
of
Ti
m
s
e
0
La
ck
Percent
50
Top 5 desired faculty uses
for the Web
90
80
70
60
50
Percent
40
30
20
10
0
35
41
33
sm
es
Ac
c
3
21
33
50
ea
at
rc
er
h
ial
sa
Su
rc
hi
bm
ve
it f
s
in
al
gr
ad
es
ct
du
Co
n
es
Ac
c
re
s
er
ve
s
24
sr
es
at
er
ial
s
m
rs
e
ou
tc
un
Mo
21
37
Already use
Very much
Somewhat
Faculty preferred methods for learning
80
60
73.4
67.1
% selecting 40
51
46.5
38.6
20
0
Colleagues, friends
Local tech support
Books, journals, tutorials
Exploring, experimenting
Workshops
www.catalyst.washington.edu
[email protected]
Tom Lewis
Scott Macklin
<<[email protected]>>
<<[email protected]>>
depts.washington.edu/pettt