Transcript MS PowerPoint 97/2000 format

The QA Focus Perspective
Marieke Guy
UKOLN
[email protected]
http://www.ukoln.ac.uk/qa-focus/
QA Focus – Supporting JISC's 5/99 Programme
The QA Perspective
If you want to sort out the problems on your
Web site you can do so by developing a
Quality Assurance Framework
This involves:
• Testing
• Problem tracking and corrective action
• Policies
• Procedures
• Documentation
QA Focus – Supporting JISC's 5/99 Programme
What Can You Test?
•
•
•
•
•
•
•
•
•
•
•
Functional testing
Compatibility testing
Load/performance testing
Stress testing
Usability testing
Security testing
Integration of unit testing
Link testing
HTML and CSS validation
Reliability testing
Regression testing
• …
QA Focus – Supporting JISC's 5/99 Programme
Automated Testing
• Should be scalable
• Only suitable approach for large sites
• Can push (email) information about
problems
• Many free testing tools available
but:
• Will not spot all usability/accessibility
problems
• Can provide too much information
• Comprehensive and configurable
testing tools can be expensive
QA Focus – Supporting JISC's 5/99 Programme
Manual Testing
• Can spot usability/accessibility problems
• Can make use of one’s community/user
base/peers
• Can use of key pages which will identify
problems which occur elsewhere
• Useful for testing new Web sites
• “5 users can spot most of your problems”
but:
• Not suitable for testing large numbers of
pages
• Quality of feedback may be variable
• Often not useful for testing
established
Web sites
QA Focus – Supporting JISC's 5/99 Programme
Approaches to Automated Testing
Automated testing can make use of:
• Web-based tools
• Desktop tools
UKOLN has made use of Web-based testing
tools:
• Monitor accessibility, HTML, CSS,
compliance, page size, links quality, nos.
of links, etc.
• Benchmarking approach provides
comparison with one’s peers
http://www.ukoln.ac.uk/qa-focus/surveys/
QA Focus – Supporting JISC's 5/99 Programme
Peer Review
• “Give me comments on my Web site and
I’ll also test yours”
• Can help in community-building
• Particularly useful for programmes in
which project Web sites being developed
at same time
But:
• Can be difficult in processing
unstructured responses
• Need to manage process
QA Focus – Supporting JISC's 5/99 Programme
Problem Tracking
There are three stages in a corrective action
system:
• The short term fix
• The cure
• Prevention of errors – identify potential
causes of error by analysing data to setect
trends which if allowed to continue would
result in errors
QA Focus – Supporting JISC's 5/99 Programme
Documentation
3 Levels of documentation:
• Policy e.g. Quality Manual
• System e.g. Processes and procedures
• Work practices e.g. instructions
More detailed, more specific, applies to
fewer people
QA Focus – Supporting JISC's 5/99 Programme
Definitions
Policy
A plan or course of action intended to
influence and determine decisions
Procedure
A way of performing or effecting something
Process
A series of actions, changes, or functions
bringing about a result
QA Focus – Supporting JISC's 5/99 Programme
Documentation
• Good documentation
•
•
•
•
•
•
•
•
•
•
Policies, procedures and audits
Define procedures for all processes
Requirements
Specifications
Mission statements or statement of direction
Define your audience
Document the testing suite and tools
Document authoring tools etc.
Document problems tracking
A Quality Management System
(QMS)??
QA Focus – Supporting JISC's 5/99 Programme
Issues I
• Goals for testing will be related to the kind
of site you have
• Testing time is limited – use automated
tools
• Documentation is critical – to make sure
practices are repeatable
• Implement your QA procedures slowly
over time to avoid stifling productivity
• QA planning at the start of development
means less time fixing things at the end
QA Focus – Supporting JISC's 5/99 Programme
Issues II
• Motivating/persuading people to
implement a QA Culture can be difficult –
run workshops to show people what you
are aiming for
• Make sure you have a feedback
mechanism for the procedures you are
implementing
• Having a great team/management is
crucial
QA Focus – Supporting JISC's 5/99 Programme
The QA Focus Perspective
QA Focus:
• Funded by JISC
• Currently provided by UKOLN and AHDS
• Currently supports JISC's 5/99 programme
• Staff:
•
•
•
•
Brian Kelly, Project manager
Marieke Guy, QA Focus officer, UKOLN
Hamish James, QA Focus officer, AHDS
Gareth Knight, QA Focus officer, AHDS
QA Focus – Supporting JISC's 5/99 Programme
Approach Taken
Two possible approaches to ensuring
compliance with standards and best practices:
Enforce
• Inspect all project’s work
• Strict auditing, with penalties for nocompliance
The latter approach is preferable,
Encourage
especially in a HE context
• Training for project staff
• Developmental, explaining reasons for
compliance, documenting examples of best
practices and providing advice on
implementation and monitoring
QA Focus – Supporting JISC's 5/99 Programme
QA Focus Work
• Documentation and Advice
•
•
•
•
Briefing Papers
Case Studies
FAQs
Descriptions of tools and architectures for
projects to implement best practices
• In-house QA
• Surveys
• Providing Motivation
• Workshops
• Liasing
• The QA Focus Toolkit
QA Focus – Supporting JISC's 5/99 Programme
Documentation: Briefing Papers
Advisory briefing
documents are being
produced
These are:
• Brief, focussed
documents
• Informed by
findings of the
surveys
QA Focus – Supporting JISC's 5/99 Programme
Documentation: Case Studies
Case Studies are being
commissioned
These are:
• Written by projects
themselves
• Describe the solution
adopted to a
particular problem
• Include details of
lessons learnt – not
just a press release!
QA Focus – Supporting JISC's 5/99 Programme
Web Site Testing Tools
• Full list of Web site
testing tools
available from
database
• Surveys run with a
number of the tools
• In the future there
will be reviews of
tools
http://www.ukoln.ac.uk/qa-focus/qa-projects/
QA Focus – Supporting JISC's 5/99 Programme
Describing Tools and Approaches
As an example of
approach which provides
easier access to testing
tools, see the ,tools
approach deployed on
UKOLN Web site.
This approach:
• Applies to all resources on Web site
• Covers HTML and CSS validation and
various other tests
• Some recursive apps (,rvalidate)
• Easily implemented
withJISC's
single
line redirect
QA Focus – Supporting
5/99 Programme
In-house QA
• Policies
• Procedures
• Audits
To date:
• Web Standards - XHTML, CSS
• Linking - links to and from
• Accessibility
• Usage Statistic
• Metadata
QA Focus – Supporting JISC's 5/99 Programme
Documentation and Advice I
We have:
• Surveyed project Web sites and identified
areas of lack of compliance with standards
and best practices
• Demonstrated examples of the potential
importance of compliance for repurposing
resources
QA Focus – Supporting JISC's 5/99 Programme
Documentation and Advice II
In addition we need to provide:
• Brief focussed advice on the standards
• Information on how to monitor compliance
• More Case studies on solutions deployed
by projects themselves
• Guidance on dealing with implementation
difficulties and what to do when strict
compliance is difficult to achieve
• Reviews of testing tools
QA Focus – Supporting JISC's 5/99 Programme
Surveying The Community
Surveys of project Web sites have been carried
out in order to:
• Obtain a profile for the community
• Identify examples of best practices
• Identify areas in which further advice is
needed
Surveys included:
• Accessibility
• 404 error pages
• HTML & CSS compliance
QA Focus – Supporting JISC's 5/99 Programme
Survey Philosophy I
The surveys made use of freely-available Webbased tools:
• Methodology is open
• No software needs to be installed locally
(apart from Web browser)
• Findings can be reproduced
• Latest results can be obtained by clicking
on link to testing service
QA Focus – Supporting JISC's 5/99 Programme
Survey Philosophy II
The surveys typically examined project entry
points and not entire Web site as:
• This page has the highest profile
• The aim is to validate a methodology
which can be deployed by projects
themselves, not to test every page on
behalf of the projects
QA Focus – Supporting JISC's 5/99 Programme
Survey Findings
Findings available from
http://www.ukoln.ac.uk/qa-focus/surveys/
QA Focus – Supporting JISC's 5/99 Programme
Providing Motivation
There is a need to explain why compliance with
standards is important and to provide
motivation for projects to update their tools,
authoring procedures, etc.This may involve
• More workshops
• Evaluating project's QA plans
• Liaising with projects over the requirements of the
key QA procedures applicable to their work
• Liaising with the Advisory Services over their
provision of advice and address maintenance and
feedback mechanisms
• Overseeing QA work
QA Focus – Supporting JISC's 5/99 Programme
Self Assessment Toolkit
We will be developing a self-assessment toolkit
for projects to use, by individual projects or
across project clusters
The toolkit will consist of:
• Examples of QA procedures
• Documented examples of use of testing
tools
• Self-assessment questionnaires
• Advice on standards and best practices
• Case studies, FAQs etc.
http://www.ukoln.ac.uk/qa-focus/toolkit/
QA Focus – Supporting JISC's 5/99 Programme
QA for Digitisation
Do it once…..do it right:
• Project is fundamentally dependent upon
the quality of original product
• Quality is the pre-requisite to preservation
• Quality expectations will only grow
• Delivery problems can be fixed, but
capture problems normally can’t
QA Focus – Supporting JISC's 5/99 Programme
QA For Web Sites
• The Web is the main delivery mechanism for
projects and services
• An increasing awareness of the importance
of accessibility, use of new devices (PDAs,
WAP, e-books, …), repurposing of Web
content (e.g. archiving)
• But invalid HTML is still the norm
• Look to standards – XHTML, XML, CSS, WAI
• If proprietary formats need to be used, flag
them and use in most open way
• QA Focus is trying to explain why and how
QA Focus – Supporting JISC's 5/99 Programme
Next Steps
Next Steps
Extended Coverage
We will be moving on from Web and
digitisation to include other areas including:
• Metadata
• Multimedia
• Software development
• Deployment into service
•…
QA Focus – Supporting JISC's 5/99 Programme
Next Steps – Metadata
Thoughts on QA for metadata:
Syntax
Check syntax on embedded DC
Cataloguing Rules Ensure projects have appropriate
cataloguing rules
Interoperability
Ensure metadata can
interoperate with third parties
Fitness for purpose Ensure metadata is appropriate
for its purpose
Change control
Ensure architecture for managing
metadata can cope with change
control (cf Exploit Case Study)
QA Focus – Supporting JISC's 5/99 Programme
Next Steps – Software
Thoughts on QA for software:
Specification
Methodology
Testing
Deployment
Process for developing systems
spec, agreeing spec with
stakeholders, etc.
Documentation of systems
development methodology (cf UML
case study)
Documentation of testing
methodology
Awareness of deployment
challenges
QA Focus – Supporting JISC's 5/99 Programme
Limitations
There are a number of limitations to the work we
have carried out so far:
• Project Web sites have different purposes
(information about the project;
communications with project partners;
project deliverables themselves; etc.)
• Projects have different levels of funding,
resources, expertise, etc.
• Projects are at different stages of
development (and some have finished)
QA Focus – Supporting JISC's 5/99 Programme
QA Focus And You!
What relevance has QA Focus for you?
• You can deploy QA Focus practices for
your projects, etc
• You can write case studies and advisory
documents:
• Seen to be working with JISC
• We can promote your approaches and work
• This will help instigate best practices within
your group
• You can develop QA procedures in
innovative areas (e.g. OAI, thesaurii,
ePrints, e-Books, etc.) which
QA Focus can use
QA Focus – Supporting JISC's 5/99 Programme
Questions
Any questions?
QA Focus – Supporting JISC's 5/99 Programme