Transcript IRB Web Site Usability Test Final Report

IRB Web Site Usability Test
Final Report
English 3367 Web Usability
Testing Team
Prepared for:
Donna Peters, Project Sponsor
Human Subjects Research
Coordinator
Texas Tech University
Institutional Review Board
Introduction
Background
Purpose
Project Overview
Methodology
Qualitative
• Heuristic evaluation
• Site map
Quantitative
• Lab tests
• Post test
Heuristic Evaluation
Visibility of System Status

Does the web site tell you where you are?
Match of system and the real world

Does the web site use realistic language and conventions?
User control and freedom

Does the web site allow the user to control activities?
Consistency with standards

Does the web site present information and terms consistently?
Error prevention

Does the web site make it hard or easy to make mistakes?
Recognition Rather Than Recall

Does the web site make you remember things or make things obvious?
Flexibility and Efficiency of Use

Does the web site make it easy to do things in more than one way?
Minimalist Design

Does the web site encourage interaction?
Error Recovery

Are error messages plainly stated with clear solutions?
Help and Documentation

Does the web site present help effectively?
Heuristic Evaluation Results
Characteristic
Observation
Visibility of system status
Yes, there is a title page for each main subject. Title tag is not being effectively used:
currently, using URL address, could repeat title or be more descriptive over the site
Match of system and the real
world
Technical wording mixed with common understandable language
User control and freedom
No, feels like there are too many options and if mouse moves just a bit, there are too
many options with other sub-categories
Consistency with standards
The Tech pages are consistent with tables and text, but if it is linked out, the pages
change or documentation is too wordy to understand
Error prevention
Very easy to make mistakes, no clarity to where user is on site.
Recognition rather than
recall
No, overload of information in technical terms, needed information is hidden quite
well.
Flexibility and efficiency of
use
No, there is no return to main page/back button; there is really no good
navigation. There is also no site map.
Minimalist design
No, site seems to be very cold and technical, thus making the user shy away from
interaction.
Error recovery
There is an email to contact the webmaster for any questions to the site, but there is not
a Q&A section available for common questions.
Help and documentation
There is a lot of documentation but it is not organized well. It seems like there is too
much documentation, which makes the user shy away from trying to find the page
he/she needs. Wording and organization is poor.
Test Objectives
1.
2.
3.
4.
5.
6.
7.
8.
9.
Terminology. Do users understand the terminology on the IRB site?
Resource types. Do users understand the differences between resource types as described on the
site and how that affects the availability of items?
Organization. Is the site organization effective and usable for students trying to locate institutional
review process information? Are users able to identify parts of the site relating to IRB and
differentiate it from other parts of the site (ORS)?
Navigation. Is the navigation of the site efficient for an IRB approval task? Are users aware of their
current location in the site and how to return to a prior point in the process? Can site be accessed
from different locations by different users?
Page layout. Are the page layouts confusing or distracting? Are they too similar or too different—
too dense, or too sparse?
Form usage. Are the users made aware of the function of the various forms they need to fill
out? Can users understand and be able to properly complete the checklist?
User control. Do users feel engaged and “in control” when using the web site?
Quality of writing. Is the site well-written overall and clear enough for users to understand the
information they see?
Links and controls. Are hyperlinks and controls always spotted and recognized as such on each
page?
User Personas
Faculty, “Jeffery”
Graduate Students, “Emily”
Age
34, Married
27, Single
Education
PhD in Psychology
Work
Experience
Full-time professor at
Texas Tech University
Residence
Off-campus town home
Computer
Experience
Moderate, computer use
on daily basis mostly for
research and e-mails
IRB
Experience
Uses IRB website
regularly, has trouble
finding coversheet form.
Use IRB website to find
forms needed for human
research and locate
funding for research
projects
Working towards PhD in
Sociology
Worked as a Grant Writer for
Planned Parenthood from
2001-2004
Off-campus apartment,
Lubbock, Texas
Moderate, proficient with
Microsoft Office and Access,
but otherwise does not like to
use technology
Has vast writing experience,
but has never used the IRB
before
Needs to learn how to have her
study approved in a timely
manner; avoid driving to
campus; retrieve forms from
the internet; save money
Priorities
Tasks
Task 1: Find the “Getting Started page” Link
Task 2: (F) Using the “Exemptions” page for IRB approval
Task 2: (G) “What are some requirements for Principle Investigators?”
Task 3: “Is TTU approval needed if TTU is not funding your research?”
Task 4: What are the elements of a consent form?
Task 5: Who would you contact to get additional information and how
Tasks Linked to Test Objectives
Graduate Scenario/Task List:
Scenario 1: Allocated time, 10 minutes
You are thinking about doing research involving human subjects and are required to
obtain IRB permission before proceeding. Locate the “How to Get Started” link on
the site. http://www.ors.ttu.edu/Newors/newhome/home/trymain.html
Relevant test objectives
X 1. Terminology
2. Resource types
X 3. Organization
X 4. Navigation
7. User control
X 5. Page layout
8. Quality of
writing
6. Form usage
X 9. Links and
controls
User Test Results
Task 1: Find the “Getting Started page” Link
• 4 out of 5 were unable to find the “Getting Started” page
Task 2: (F) Using the “Exemptions” page for IRB approval
• 2 out of 3 had difficulty finding and using the form
Task 2: (G) “What are some requirements for Principle Investigators?”
• 2 out of 2 had difficulty answering the question
Task 3: “Is TTU approval needed if TTU is not funding your research?”
• 4 out of 5 answered the question incorrectly
Task 4: What are the elements of a consent form? Link
• 3 out of 5 were not able to answer correctly
Task 5: Who would you contact to get additional information and how
• 5 out of 5 were able to answer correctly
Post Test Results
Key Findings
• Language unclear in pages and in forms
• Navigation of site difficult and navigation bar
confusing
• Introductions to forms unclear and skimpy
• Users did not know where to start
• Overall site poorly organized
Language Not Clear
http://media.english.ttu.edu/faculty/barker/3367irb/language.wmv
Introduction Not Useful
Introduction to forms need more information
http://media.english.ttu.edu/faculty/barker/3367irb/introductio
n.wmv
Navigation Unclear
Improve navigation by reducing redundant links.
http://media.english.ttu.edu/faculty/barker/3367irb/navigation.wmv
“How To Get Started” Not Useful
Make “How to Get Started” a series of steps with links.
http://media.english.ttu.edu/faculty/barker/3367irb/howto.wmv
Site Map Evaluation
“The site is mostly a collection of documents…”
Organize the site around recognized user persons
Site Poorly Organized
Organize the site around recognized users
• Tester -using the site to find information on testing
procedures and policies
• Testee -interested in participating in the testing process
• Eraider member (password needed)-links that can only
be used by those with an eraider account
• Administrator (password possibility) staff who will be
conducting or overseeing testing and research, and may
need administrative access to use the link
• External link -leads to links that are not directly controlled
by texas tech (or ORS & IRB website)
Recommendations
• Clarify definitions of terms Link
• Make “How to Get Started” useful
• Improve navigation by reducing redundant
links
• Create informative overviews for forms
• Reorganize the pages according to user types
Conclusion
Project Mission: make educated recommendations concerning the effectiveness of the ORS
Human Subjects website
Project Methods: heuristic evaluation, qualitative and quantitative measurements, usability
testing
Subjects Studied: terminology, resource types, organization, navigation page, layout, form
usage, user control, quality of writing, links and controls
Recommendations:
• Clarify definitions of terms
• Make “Getting Started” useful
• Improve navigation by reducing redundant links
• Create informative overviews for forms
• Organize the site around user needs
Final Report
Please view our entire report at:
http://irbusability.pbwiki.com/
Questions?