Transcript Powerpoint version

QA For Web Sites
Brian Kelly
UKOLN
University of Bath
Bath
Email
[email protected]
URL
http://www.ukoln.ac.uk/
Contents
• What Is Quality
• Quality And Web
Sites
• QA Case Study
• Approaches To
Testing
• Benchmarking
• Conclusions
UKOLN is supported by:
A centre of expertise in digital information management
www.ukoln.ac.uk
Background To Web & QA Focus
Web Focus:
• A JISC-funded post to advise the UK’s HE and
FE community
• Provided by Brian Kelly
QA Focus:
• An advisory service for JISC’s 5/99 projects
• Provides advice and support for projects in order
to ensure that projects comply with standards and
best practices
• Provided by UKOLN (Marieke Guy & Brian Kelly)
and ILRT
This talk is informed by the work of Web and QA Focus
A centre of expertise in digital information management
2
www.ukoln.ac.uk
Quality Control and Quality Assurance
Quality Control (QC):
• Use of regular testing procedures against your
definitions of quality and more specifically the
refinement of these procedures
• Relates to outputs
Quality Assurance (QA):
• examines the processes that shape your Web
site in the first place and so encompasses Quality
Control
• Relates to inputs
• Should be involved in development stage of a
Web site and throughout its life cycle
A centre of expertise in digital information management
3
www.ukoln.ac.uk
What Can Fail on Your Site?
•
•
•
•
•
•
•
•
•
•
•
•
Broken links, broken emails
Server load – too many hits on the site
Client side performance –down load time
Security isn’t working
Content is out of date
Browser incompatibility,HTML doesn’t validate
Interface – navigation, link colour
Graphics missing or too large
Scripts don’t work - forms, databases
Isn’t accessible to those with disabilities
Browser dependencies
…
A centre of expertise in digital information management
4
www.ukoln.ac.uk
What Can You Test?
•
•
•
•
•
•
•
•
•
•
•
•
Functional testing
Compatibility testing
Load/performance testing
Stress testing
Usability testing
Security testing
Integration of unit testing
Link testing
HTML Validation
Reliability testing
Regression testing
…
A centre of expertise in digital information management
5
www.ukoln.ac.uk
What Procedures?
• Good documentation
• Requirements
• Specifications
• Mission statements or statement of direction
• Define your audience
• Testing suite and tools
• Usability testing
• Use the right authoring tools etc.
• Track problems
A centre of expertise in digital information management
6
www.ukoln.ac.uk
Issues
Things to be aware of:
• Testing goals should relate to kind of site you have
• Testing time is limited  use automated tools
• Automated tools can be inadequate  use manual
tests
• Documentation is critical to ensure practices are
repeatable
• Motivating/persuading people to implement a QA
culture can be difficult
• Implement your QA procedures over time to avoid
stifling productivity
• QA planning at the start of development means
less time fixing things at the end
A centre of expertise in digital information management
7
www.ukoln.ac.uk
Case Study
Case Study: IWMW Web Site
http://www.ukoln.ac.uk/web-focus/events/workshops/webmaster-2002/
A centre of expertise in digital information management
8
www.ukoln.ac.uk
Case Study
Aims Of Web Site
The aims of the Web site were:
• To provide information about the workshop for
participants, speakers, etc.
• To provide an online booking facility
• To act as a demonstrator of standards, best
practices and innovative solutions
Subject to the following constraints:
• No Content Management System
• Limited time
• Limited software development / design expertise
A centre of expertise in digital information management
9
www.ukoln.ac.uk
Case Study
Approach
The approach taken was to use:
• The design and structure used for previous two
workshops
• The HTML-Kit authoring tool to edit HTML files
• The Xenu link checker
Innovations this year include:
•
•
•
•
•
•
Use of HTML and CSS validation icons
Provide an XHTML Web site
Make greater use of CSS
Provide a dynamically-generated RSS news feed
Provide live usage statistics
Access to Web site on PDAs using AvantGo
Is this approach typical of yours – incremental development of an existing
Web site,
and ofintroducing
new
features
and more up-to-date standards?
A centre
expertise in digital
information
management
www.ukoln.ac.uk
10
Tools
HTML-kit is my
preferred editor
(and recent
version is even
better, providing
batch validation)
A centre of expertise in digital information management
11
The XHTML and
CSS icons act as
live links
The SiteMeter
icon gives
realtime info on
usage www.ukoln.ac.uk
Case Study
Problems
Some problems:
• When updating pages, links, HTML and spelling
were not checking systematically
• When using the HTML and CSS checker the
browser could check an old, cached version
• At one stage the SiteMeter usage statistics
service was not available
It can be a useful (and therapeutic!) process
to talk about problems with your Web site.
A centre of expertise in digital information management
12
www.ukoln.ac.uk
Case Study
Validating In Batch
Checking (e.g. HTML and CSS validation, links,
accessibility, etc.) initial files is time-consuming and, for
large or continually updated Web sites is not a scalable
solution
Validation in batch:
• Familiar for link-checking
• Limited set of tools available for HTML validation
(e.g. HTML Validator Pro>
• Batch HTML validation validates files generated
from PowerPoint, over which I have no control
• Bobby batch accessibility check is now a licensed
application – so used old copy
Can now use HTML-Kit for validating the Web site.
A centre of expertise in digital information management
13
www.ukoln.ac.uk
Case Study
Netscape (sigh)
User feedback revealed that:
• £ does not display in Netscape :-(
• The Suggestions text box and the Submit button
(!) were not displayed in versions of Netscape,
due to failure to support XHTML correctly
Suggestions
Submit
A centre of expertise in digital information management
14
www.ukoln.ac.uk
Case Study
RSS News Feed
The RSS News Feed:
• Provides a mechanism for syndication of content
• News feed can be viewed on other Web sites or
in other applications
The News Feed was created:
• By transforming (lightly structured) HTML from
the news page using a remote service
• A static version was also created in case of
problems with the live transformation
Problems:
• The remote service was not always available
• The static version was not always in sync with the
live version
Subsequently the dynamic
transformation service
A centre of expertise in digital information management
www.ukoln.ac.uk
became
unavailable
15
Case Study
Online Bookings
The online booking forms:
• Were used for booking for the workshop and choosing the
parallel sessions
The backend processing:
• Is not yet fully automated to store the information in a
database
• Confirmation page did not echo the user’s data
• When users followed link to check session, input data may
have been lost
• Crashed on surnames with spaces and accents
• No validation was performed
Despite messages on acknowledgements page, some
users still expected a separate email confirmation
A centre of expertise in digital information management
16
www.ukoln.ac.uk
Case Study
What Should Be Done
Some simple things can be done for next time:
• Use of SSIs or a CMS (or equivalent ) to manage
resource fragments
• Improve the online booking system to provide:
• Validation (which will save time in administration)
• Display of user input
• Separate confirmation of entry into database
But:
• I recognise that minor typos, etc. will happen and,
as long as they are not critical, can be tolerated
• The goal is quality – not perfection
A centre of expertise in digital information management
17
www.ukoln.ac.uk
Case Study
What About The Processes?
What QA processes were used?
• Automated tools
• Peer review from members of the organising
committee
• Alternating between Opera, IE and Mozilla when
working on Web site
• Viewing the Web site on my Palm to:
• Allow me to work on the bus, train, etc.
• See how the Web site works on a low-spec device
See <http://www.ukoln.ac.uk/web-focus/
events/workshops/webmaster-2002/qa/>
A centre of expertise in digital information management
18
www.ukoln.ac.uk
Case Study
19
What Else Should I Do?
Additional processes I should employ:
•
•
•
•
•
•
•
•
•
Reviewing by users
Peer-reviewing (others in similar situation)
Analysis of logs (esp. failure logs)
Usability testing
Multiple browser testing
Test data with unusual entries for online forms
Post-workshop review of Web site
Systematic documentation of successes and failures
Make case for extra resources, software, etc. to
implement improvements
• Documenting the architecture of the Web site
• Documenting the limitations of the Web site
• Preparing these slides!
A centre• of …
expertise in digital information management
www.ukoln.ac.uk
Automated Testing
Automated Testing:
•
•
•
•
Should be scalable
Only suitable approach for large Web sites
Can push (email) information about problems
Many free automated testing tools available
but:
• Will not spot all usability / accessibility problems
• Can provide too much information
• Comprehensive and configurable testing tools
can be expensive
A centre of expertise in digital information management
20
www.ukoln.ac.uk
Manual Testing
Manual Testing:
• Can spot usability / accessibility problems
• Can make use of one’s community / user base /
peers
• Can use of key pages which will identify
problems which occur elsewhere
• Particularly useful for testing new Web sites
• “5 users can spot most of your problems”
but:
• Not suitable for testing large numbers of pages
• Quality of feedback may be variable
• Often not useful for testing established Web
sites
A centre of expertise in digital information management
21
www.ukoln.ac.uk
Approaches To Automated
Testing
Automated testing can make use of:
• Desktop tools
• Web-based tools
UKOLN has made use of Web-based testing tools:
• Monitor accessibility, HTML, CSS, compliance,
page size, links quality, nos. of links, etc.
• Benchmarking approach provides comparison
with one’s peers
• See, for example,
<http://www.ukoln.ac.uk/web-focus/
events/conferences/ucisa-tlig-2002/
benchmarking/>
A centre of expertise in digital information management
22
www.ukoln.ac.uk
Interfaces to Testing Tools (1)
Bookmarklets
• Embed a testing service into your browser
• Available for IE, Netscape, Opera, …browsers
• Article at <http://www.ariadne.ac.uk/
issue19/web-focus/>
• See <http://www.bookmarklets.ac.uk/> or
Google search for “bookmarklets”, “accessibility
bookmarklets”, etc.
A centre of expertise in digital information management
23
www.ukoln.ac.uk
Bookmarklets Example
Bookmarklets can
be accessed:
• From the
browser’s normal
Bookmarks /
Favourites Menu
• From a side bar
• By right-clicking
the mouse button
A centre of expertise in digital information management
24
www.ukoln.ac.uk
Interfaces to Testing Tools (2)
URL Interface
• UKOLN’s Web site has an interface to various
tools provided at the server
• Easier to maintain
• See <http://www.ukoln.ac.uk/,tools>
for overview
<http://www.ukoln.ac.uk/,validate>
A centre of expertise in digital information management
25
www.ukoln.ac.uk
Benchmarking (1)
Benchmarking:
• We have discussed various tools for checking
one’s own site
• But how do we compare with our peers?
• Can we learn from others best practices?
• Can we avoid making mistakes that others have
made?
Can the methodology to be described be used across your
community – e.g.
• On a regional basis
• On candidates for Best Public Library Web Site award
• ByAgovernment
auditors for checking e-GIF compliance
centre of expertise in digital information management
www.ukoln.ac.uk
26
Benchmarking (2)
A centre of expertise in digital information management
27
www.ukoln.ac.uk
QA Focus Web Site Benchmarking
QA Focus:
•
•
•
•
Carried out survey of 43 project Web sites
Findings freely available
Methodology can be applied to other sectors
Where best practices found, projects asked to
provide a case study
• Where problems found, appropriate advice
provided
A centre of expertise in digital information management
28
www.ukoln.ac.uk
HTML Compliance
http://www.ukoln.ac.uk/qa-focus/surveys/
A centre of expertise in digital information management
29
W3C’s HTML Validation
service is used to
record the HTML
compliance of project
home pages
www.ukoln.ac.uk
Accessibility
Bobby is used to record compliance
with W3C WAI guidelines:
24 comply with A
5 comply with AA
21 are not compliant
Note that only compliance with guidelines which
be monitored
bymanagement
automated tools is recorded
A centrecan
of expertise
in digital information
www.ukoln.ac.uk
30
Repurposing Resources
Are pages from project Web sites available in the
Internet Archive?
Can the project Web site be accessed on a PDA?
A centre of expertise in digital information management
31
www.ukoln.ac.uk
404 Pages
404 error pages can provide an important
navigational feature for Web sites
Tailored 404 Page
14 33%
* Rating
3 21%
Default 404 Page
28 67%
** Rating
3 21%
*** Rating
8 57%
**** Rating
0 0%
*
**
***
****
A centre of expertise in digital information management
32
Basic
Simple branding
Additional functionality
Full functional
www.ukoln.ac.uk
Critique
Limitations of this approach include:
• Project Web sites may be:
 The project deliverable  About the project
 For intra-project communications
• The project deliverable may be, say,
middleware
• Limitations of the analysis tools
• Limited number of pages surveyed
• …
A centre of expertise in digital information management
33
www.ukoln.ac.uk
Supporting Best Practices
QA Focus is
supporting JISC
5/99 projects by
providing advice
on compliance
with standards
and best
practices, based
on the survey
findings
http://www.ukoln.ac.uk/
qa-focus/documents/
A centre of expertise in digital information management
34
www.ukoln.ac.uk
Case Studies
QA Focus is
commissioning
case studies:
• Document
approaches
projects have
taken in
particular areas
• Describe the
problem area,
the approach
taken, problems
experienced and
lessons learnt
A centre of expertise in digital information management
35
www.ukoln.ac.uk
Conclusions
To conclude:
• Quality assurance is important
• It will grow in importance: e.g. compliance with eGIF guidelines, accessibility legislation, etc.
• Automated tools can help
• Manual testing is needed to complement
automated approaches
• Benchmarking can provide information on
approaches across communities
A centre of expertise in digital information management
36
www.ukoln.ac.uk