Transcript No Slide Title

Evaluating electronic
textbooks: a methodology
Ruth Wilson and Monica Landoni
ECDL 2001
5th September 2001
Outline
• The Visual Book
• The WEB Book
• EBONI’s methodology for evaluating
electronic textbooks
The Visual Book
• Studied the application of the paper
book metaphor to the design and
production of electronic books
• Found that users were able to rely on
their experience with paper books to
interact with the electronic book.
The WEB Book
• Focused on the impact of appearance on the
•
usability of textbooks on the Web.
Comparison of two versions of a textbook
chapter:
– Original version: very plain, one “page” per
chapter, basic use of hyperlinks.
– Revised, “scannable” version: chapter divided
between 10 “pages”, heavier use of hyperlinks,
extra headings, indented lists, coloured keywords.
• Revised version was 92% more usable.
EBONI: aims
EBONI aims to:
• Evaluate the different approaches to the design of
•
•
electronic textbooks
Identify and report on the individual requirements
of students and academics in learning and
teaching using electronic textbooks
Compile and promote a set of best practice
guidelines for the publication of electronic
textbooks
Experiments:
psychology Web books
• Peter Kaiser. The Joy of Visual Perception.
•
•
http://www.yorku.ca/eye/thejoy.htm
Eric Chudler. Neuroscience for Kids.
http://faculty.washington.edu/chudler/neur
ok.html
C. Robin Timmons and Leonard W. Hamilton.
Drugs, Brains and Behaviour.
http://www.rci.rutgers.edu/~lwh/drugs/
Experiments:
Hypertext in Context
• Cliff McKnight, Andrew Dillon and John
Richardson. Hypertext in Context.
– Print
– Original electronic version
– Revised electronic version
Experiments:
commercial formats
• Rosa Ainley (ed.). New Frontiers of
Space…
– MobiPocket Reader
– Adobe Acrobat eBook Reader
– Microsoft Reader
– Paperback
Experiments:
e-encyclopaedias
• Britannica
• Columbia
• Encarta
Experiments:
portable ebooks
•
•
•
•
•
HP Jornada
Franklin eBookMan
PalmPilot
Rocket eBook
Softbook
Ebook evaluation model
• Selection of material
• Selection of participants
• Selection of tasks
– Scavenger hunt, memory tasks, high
cognitive skill tasks
• Selection of evaluation techniques
– Questionnaires, observation, think-aloud,
interviews
Selection of material (1)
Electronic books offer a diverse array of
material for evaluation:
• Hardware devices, e.g. REBs,
eBookMan, goReader
• Ebook reader software, e.g. Microsoft
Reader, Adobe Acrobat Ebook Reader
• “Web books”
Selection of material (2)
• EBONI aims to identify design techniques
•
which maximise information intake by users,
in order to provide guidance to creators of
digitised educational material.
Texts can be compared according to three
parameters:
– Format
– Content
– Medium
Selection of actors
Four main roles can be distinguished:
•
•
•
•
Participant
Evaluator
Task developer
Task assessor
Selection of tasks
• Tasks are a way of bringing together
participants and the test material in a
structured manner. Three types are
suggested:
– “Scavenger Hunts”, or retrieval tasks
– Memory tasks
– High cognitive skill tasks
Selection of evaluation techniques
Qualitative feedback can be obtained via:
•
•
•
•
Subjective satisfaction questionnaires
Behaviour observation
Think-alouds
Interviews
Technique
complexity
Study X
Interviews
Think-aloud
Covert
observation
Questionnaires
WEB
Book
Scavenger
hunt
Memory
tasks
High
cognitive
skill tasks
Task
complexity
Guidelines
• Sophisticated experiments can be
“mapped” to simple experiments.
• Results of all evaluations will be
comparable at some level.
• Results will feed into a set of best
practice guidelines for producing
electronic textbooks.
Contact details
• Ruth Wilson: [email protected]
• EBONI Web site:
http://eboni.cdlr.strath.ac.uk/
• To join the EBONI mailing list, follow the
instructions at:
http://www.jiscmail.ac.uk/lists/openeboni.html