Transcript Abstract Presentation

Usability Testing of a
Medical Library Portal
Anamarija Rožić-Hristovski, M.D., M.Sc.
Central Medical Library
Iztok Humar, B.Sc., M.Sc.
Faculty of Electrical Engineering
1
Presentation Overview
Introduction
 CMK portal
 Case study goals
 Usability studies
 Methods
 Results
 Conclusions

2
Introduction
A Web portal or gateway aggregates library
resources and services through a single
access point.
 Medical library Web interfaces have
become increasingly complex.
 There is a fundamental need for the
systematic evaluation of the usability of
these Web based systems.

3
CMK Portal
 CMK
decided to extend its Web site
with information portal
 CMK portal based on MyLibrary
system at NC State University
 Customisation to the needs of CMK
users was needed
 MyLibrary
functionality is enabled
4
MyLibrary CMK functionality
Consideration of the CMK organisation,
role and function
 Slovene-English communication with
MyLibrary
 Customisation of text and content
 Personalization of users’ profiles
 Slight modification of the portal design to
suit the rest of the CMK Web site

5
6
Case Study Goals
Formal usability case study for the
evaluation of the pilot portal
 Review of applications of usability testing to
library Web sites
 Selection of the most appropriate usability
methods
 Application of selected methods to evaluate
CMK portal

7
Usability & usability testing



ISO defines usability as “the extent to which a
product can be used by specified users to
achieve specific goals with effectiveness,
efficiency, and satisfaction in a specified context
of use”
In the world of WWW this means that the
interface of a portal must be easy to learn,
remember and use and must lead to as few
errors as possible
In usability testing, users are observed while
using the portal, to perform given task or achieve
a set of predefined goals
8
Most common methods for
evaluating the web site usability
Objective performance
 Subjective user preferences
 Experimental evaluation
 Direct observation

9
“Think aloud” method





The most systematic and valid direct observation
method
The users are asked to perform specific task
A relatively small sample size (5-6 users) is
needed for identifying the majority of the usability
problems
During their work, they are required to verbalize
their problems
User verbalizations and keystrokes are recorded
on audio tapes, observations are written to
notebooks
10
“Think aloud” method: properties

This method provides answers:
– How easy a user can learn and use the system
– How flexible the system is
– Why users have made mistakes

“Critiques of this method goes to:
– Presence of the experimenter:

disturbance for the user
– The need for verbalization:

depends on the user’s character
11
Methods- I
“Think aloud” method was selected as the
most appropriate
 Designing the test:

– 12 test exercises
– Instructions to observer
– Tape-recording of sessions
– Observations written to notebooks
12
Methods –II

Test participants
– Before instruction
3 students
 3 faculty

– After instruction
3 students
 3 faculty

13
Usability Testing of MyLibrary@CMK
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
Login to MyLibrary@CMK portal!
On your personal page, select the library links that you intend to use at your work.
Return to MyLibrary!
On your personal page, select the university links that you intend to use at your
work. Return to MyLibrary!
On your personal page, select the quick searchers that you intend to use at your
work. Return to MyLibrary!
On your personal page, select the databases that you intend to use at your work.
Return to MyLibrary!
On your personal page, select the e-texts that you intend to use at your work.
Return to MyLibrary!
Identify all the resources about AIDS available in MyLibrary! Return to MyLibrary!
Find all books from the field of dermatology, available at the Central Medical Library!
Return to MyLibrary!
Find journal Neurology! Return to MyLibrary!
Find a journal article about headache! Return to MyLibrary!
Find new books from the field of public health of the last three months! Return to
MyLibrary!
Switch the language for communication with MyLibrary!
14
Success rate
Results - I:
Success in Performing Tasks
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
100%
95%
80%
78%
Students before
instruction
Faculty before
instruction
Students after
instruction
Faculty after
instruction
Participants
15
Results - II:
Time Needed to Performing Tasks
30
30
25
25
22
20
15
10
5
Time (minutes)
19
Students before
instruction
Faculty before
instruction
Students after
instruction
Faculty after
instruction
0
Participants
16
Results – III:
Summary of Usability Problems









Note on length of password
Inconsistent performance of buttons
Complaints about registration
Unclear terminology
The role and content of MyLibrary
The role and content of OPAC
The role and content of databases
No idea how to find an article
No distinction between the title of an article and
the title of an journal
17
Conclusions
 Students
performed tasks much better
than faculty
– More familiar with internet and concept of digital
library

Role of education
– No influence on success rate
– Less time needed for finishing the tasks

Identification of main usability problems
18
Further Work
Redesign of library portal considering
results of usability study
 New usability study to verify
appropriateness of changes
 Updating the content of portal
 Public use of portal

19