Transcript QA Focus

Quality Assurance (QA) For
Digital Library Projects
Brian Kelly
UKOLN
[email protected]
http://www.ukoln.ac.uk/qa-focus/
QA Focus – Supporting JISC's Digital Library Programmes
1
This talk will describe
the work of the JISCfunded QA Focus
team and outlines
approaches to QA
which can be
deployed in Digital
Library projects
Contents
• Introduction
• Background:
• The need for QA
• Work activities:
• Web
• Providing Advice
• Future Work
• Metadata
• Service Deployment
• A QA Methodology
• Relevance To You
• Conclusions
 QA Focus
 Accessibility
 Software Development
 Toolkit
QA Focus – Supporting JISC's Digital Library Programmes
2
About UK Web Focus / UKOLN
UK Web Focus:
• Funded by JISC since Nov 1996 to advice UK HE
(and the FE) on standards and best practices for the
Web
• Provided by Brian Kelly
• Based at UKOLN, a national centre of expertise in
digital information management
• Located at the University of Bath
• Project manager for JISC's QA Focus project
Note: Since 1 August 2003 UK Web Focus has been jointly
funded by JISC and Resource and now also advises the
cultural
heritage sector in England and Wales
QA Focus – Supporting JISC's Digital Library Programmes
3
Introduction
Aims Of Today’s Talk
• To describe the role of quality assurance in
digital library programmes
• To describe the approach taken by the QA
Focus project
• To summarise QA Focus activities to date
• To describe future activities
• To explore potential for collaboration with
projects & services represented here
Note that although this talk is based on a JISC-funded
project the content is applicable to everyone involved in
Web
activities
– including Webmasters
QA
Focusdevelopment
– Supporting JISC's Digital
Library Programmes
4
Introduction
An Introduction to QA
What is Quality?
“Quality is the ability of your product to be able to
satisfy your users”
What is Quality Assurance?
“Quality assurance is the process that demonstrates
your product is able to satisfy your users”
QA Focus – Supporting JISC's Digital Library Programmes
5
Introduction
An Introduction to QA
Why Is QA Needed Now?
In the past JISC had standards-based philosophy but:
• Approach of "let a 1,000 flowers bloom"
• Diversification as no killer application then
(it could have been Gopher!)
Things are different now:
• The Web is the killer application
• There is now a need for interoperability (not just
self-contained independent Web sites)
• There is a need for publicly-funded services to be
seen to provide value for money
QA Focus – Supporting JISC's Digital Library Programmes
6
Introduction
Approach Taken
Two possible approaches to ensuring
compliance with standards and best practices:
Enforce
• Inspect all project’s work
• Strict auditing, with penalties for no-compliance
Encourage
• Training for project staff
• Developmental, explaining reasons for compliance,
documenting examples of best practices and
providing advice on implementation and monitoring
The latter approach is preferable,
especially in a HE context
QA Focus – Supporting JISC's Digital Library Programmes
7
QA Focus
QA Focus
QA Focus:
• Funded by JISC
• Currently provided by UKOLN and AHDS
• Supports JISC's 5/99, FAIR & X4L programmes
• Staff:




Brian Kelly, Project manager
(Marieke Guy, QA Focus officer, UKOLN)
Hamish James, AHDS
Gareth Knight, QA Focus officer, AHDS
• Areas covered:
 Web
 Metadata
 Service deployment
 Digitisation
 Software development
QA Focus – Supporting JISC's Digital Library Programmes
8
QA For Web
9
QA For Web Sites
The issues:
• The Web is the main delivery mechanism for projects
and services
• An increasing awareness of the importance of:
 Accessibility
 Use of new devices (PDAs, WAP, e-books, …)
 Repurposing of Web content (e.g. archiving)
• Technologies such as XSLT will support repurposing
of valid XML resources
But:
• Invalid HTML is the norm
• Many authoring tools produce poor HTML
QA• Focus
– Supporting
JISC's aware
Digital Library
Authors
aren’t
ofProgrammes
the problems
QA For Web
Guidelines
We often say:
• Open standards are important
• HTML, XML, XHTML, CSS, … are important
but fail to explain why and how
JISC’s QA Focus is addressing such concerns by:
• Documenting example of best practices in which
projects can share their implementation successes
(and difficulties they experienced)
• Provide brief advice in specific aspects of the
standards and best practices
• Surveying its communities to highlight best
practices and areas in which improvements can be
made
• Demonstrating use of testing tools and procedures
QA Focus – Supporting JISC's Digital Library Programmes
10
QA For Web
Standards & Best Practices
Standards For Web:
• Use compliant HTML / XHTML
• Use CSS
Best Practices For Web:
• Implement WAI accessibility guidelines
• Ensure Web resources are suitable for reuse and
repurposing
• Where proprietary formats need to be used, flag
them and use in most open way
QA Focus – Supporting JISC's Digital Library Programmes
11
QA For Web
Surveying The Community
Surveys of JISC 5/99 (and FAIR & X4L) project Web
sites have been carried out in order to:
• Obtain a profile for the community
• Identify examples of best practices
• Identify areas in which further advice is needed
Surveys included:
• HTML & CSS compliance
 Accessibility
• 404 error pages
 HTTP headers
• Repurposing resources
QA Focus – Supporting JISC's Digital Library Programmes
12
QA For Web
Survey Philosophy
The surveys made use of freely-available Web-based
tools:
• Methodology is open
• No software needs to be installed locally (apart from Web
browser)
• Findings can be reproduced
• Latest results can be obtained by clicking on link to testing
service
The surveys typically examined project entry points and
not entire Web site as:
• This page has the highest profile
• The aim is to validate a methodology which can be deployed
by projects themselves, not to test every page on behalf of
the projects
QA Focus – Supporting JISC's Digital Library Programmes
13
Advice
Documentation: Case Studies
Case Studies are
being commissioned
These are:
• Written by
projects
themselves
• Describe the
solution adopted
to a particular
problem
• Include details of
lessons learnt –
not just a press
release!
QA Focus – Supporting JISC's Digital Library Programmes
14
QA For Web
Survey Findings
Initial set of findings available from
<http://www.ukoln.ac.uk/qa-focus/surveys/web-10-2002/>
QA Focus – Supporting JISC's Digital Library Programmes
15
QA For Web
Providing Motivation
We have found evidence of failure to comply with HTML
standards
There is a need to explain why compliance is important
(and avoid the “it’s OK in my browser” argument) and to
provide motivation for projects to update their tools,
authoring procedures, etc.
A further set of surveys look at repurposing of the
project Web sites:
• Availability of Web sites in the Internet Archive
• Ease of making Web sites available on a PDA
• Transformation of embedded metadata
QA Focus – Supporting JISC's Digital Library Programmes
16
QA For Web
Repurposing Resources
We examined the Web sites to see if they were
available in the Internet Archive and could be
transformed into a format for viewing on a PDA
small number
AAsmall
numberof
sites
could not
ofWeb
Web
sites
be transformed.
were not in the
Analysis of HTTP
Internet
headers Archive
indicated
due
that to
thisthe
was due to
robots.txt
incorrect HTTP file.
headers.
We
will need to
We will need
to in
provide
advice
provide
advice in
this
area.
this area.
QA Focus – Supporting JISC's Digital Library Programmes
17
QA For Web
Surveys For FAIR And X4L Programmes
QA Focus now
supports JISC's
FAIR and X4L
programmes
The Web site
surveys now
provides a selfassessment
interface
http://www.ukoln.ac.uk/qa-focus/
surveys/fair/web-self-assessment/
We hope that projects will (a) check and, if necessary, fix
their home page; (b) use such tools to check/fix other key
pages
(c) JISC's
address
underlying
QA Focus –and
Supporting
Digital Library
Programmescauses of problems
18
QA For Web
A Caveat
There are a number of limitations to the work we have
carried out so far:
• Project Web sites have different purposes
(information about the project; communications with
project partners; project deliverables themselves;
etc.)
• Projects have different levels of funding, resources,
expertise, etc.
• Projects are at different stages of development (and
some have finished)
The surveys are intended to demonstrate a methodology
which projects can use for themselves
QA Focus – Supporting JISC's Digital Library Programmes
19
Advice
Providing Advice
We have:
• Surveyed project Web sites and identified areas of
lack of compliance with standards & best practices
• Shown examples of the potential importance of
compliance for repurposing resources
In addition we need to provide:
• Brief focussed advice on the standards
• Information on how to monitor compliance
• Case studies on solutions deployed by projects
themselves
• Guidance on dealing with implementation difficulties
and what to do when strict compliance
is difficult to achieve
QA Focus – Supporting JISC's Digital Library Programmes
20
Advice
Documentation: Advice
Advisory briefing
documents are being
produced
These are:
• Brief, focussed
documents
• Informed by
findings of the
surveys
QA Focus – Supporting JISC's Digital Library Programmes
21
Tools
Putting it Together
Based on surveys we find project Web sites which have
invalid HTML and CSS or other examples of failure to
implement best practices
So we:
• Explain why projects should follow best practices
• Provide case studies on approaches by projects
But how do we ensure projects :
• Check Web sites for themselves
• Have systematic checking procedures
There is a need to describe:
• Tools & architectures for projects for check
compliance with standards & best practices
• Systematic QA procedures
QA Focus – Supporting JISC's Digital Library Programmes
22
Tools
Example - Link Checking Tools
We should all be familiar
with link checking tools.
But:
• Do such tools check all
types of links (e.g. What
about links to external
style sheet files,
JavaScript files, etc.)?
• Are such tools
integrated with normal
working practices?
QA Focus – Supporting JISC's Digital Library Programmes
23
Describing Tools and Approaches
Approach is documented at
<http://www.ukoln.ac.uk/qa-focus/
documents/papers/iadis-2003/>
As an example of approach
which provides easier access
to testing tools, see the
,tools approach deployed
on UKOLN Web site.
This approach:
• Applies to all resources on
Web site
• Covers HTML and CSS
validation and various
other tests
• Some recursive apps
(,rvalidate)
• Easily implemented with
single line redirect
QA Focus – Supporting JISC's Digital Library Programmes
24
Next Steps
Next Steps
Extended Coverage
We are addressing other areas besides the Web (and
digitisation) including:
 Metadata
 Deployment into service
 Software development
Moving On From Automated Testing
The initial work made use of automated testing tools:
 Can be used remotely
 Objective
 Applicable across all projects
We have started work on QA procedures in areas
which are not suitable for automated checking
QA Focus – Supporting JISC's Digital Library Programmes
25
Next Steps
Self Assessment Toolkit
Further Deliverables
We will be developing a self-assessment toolkit for
projects to use, by individual projects or across project
clusters
The toolkit will consist of:
• Examples of QA procedures
• Documented examples of use of testing tools
• Self-assessment questionnaires
• Advice on standards and best practices
• Case studies
• FAQs
• …
QA Focus – Supporting JISC's Digital Library Programmes
26
Next Steps
Self Assessment Toolkit
QA Focus – Supporting JISC's Digital Library Programmes
27
Next Steps
Next Steps – Metadata
Thoughts on QA for metadata:
Syntax
Check syntax on embedded DC
Cataloguing Rules Ensure projects have appropriate
cataloguing rules
Interoperability
Ensure metadata can
interoperate with third parties
Fitness for purpose Ensure metadata is appropriate
for its purpose
Change control
Ensure architecture for managing
metadata can cope with change
control (cf Exploit Case Study)
QA Focus – Supporting JISC's Digital Library Programmes
28
Next Steps
Next Steps – Software
Thoughts on QA for software:
Specification
Process for developing systems spec,
agreeing spec with stakeholders, etc.
Methodology
Documentation of systems
development methodology (cf UML
case study)
Testing
Documentation of testing
methodology
Deployment
Awareness of deployment challenges
See for example <http://www.ukoln.ac.uk/qafocus/documents/briefings/#software>
QA Focus – Supporting JISC's Digital Library Programmes
29
Next Steps
Next Steps – Service Deployment
Thoughts on QA for service deployment:
Development Architecture
Early public documentation of
development architecture
Service Architecture
Awareness of likely service architecture
and liaison with service
Security
Awareness of security concerns
Licensing
Licensing issues
Scenarios
30
Software developed using cool open software environment – but
JISC Service has no knowledge of environment.
Resources digitised, but copyright clearance not documented.
Focus – Supporting
Digital Library
ServiceQArefuses
to takeJISC's
delivery
due Programmes
to liability concerns.
Next Steps
QA Focus And You
What relevance has QA Focus for you?
• You can write case studies and advisory
documents:
 Seen to be working with JISC
 We can promote your approaches (and
therefore your work)
 This will help instigate best practices within
your group
QA Focus – Supporting JISC's Digital Library Programmes
31
Next Steps
QA Focus And You
What relevance has QA Focus for you?
• You can deploy QA practices for your projects, etc:
 You should benefit
 You can provide feedback to QA Focus
• You can develop QA procedures in innovative areas
(e.g. OAI, thesaurii, ePrints, etc.) which QA Focus
can use:
 You gain kudos, publications, etc.
 We can help embed your practices across the
community
e.g. see joint UKOLN/CDLR paper on "Deployment Of Quality Assurance
Procedures For Digital Library Programmes" at
<http://www.ukoln.ac.uk/qa-focus/documents/papers/iadis-2003/>
QA Focus – Supporting JISC's Digital Library Programmes
32
Next Steps
Beyond Advice – Implementing QA
QA Focus is providing a wide range of advice
for digital library programmes
However there is also a need for projects
themselves to be proactive in implementing
QA procedures
What approaches can projects take, bearing in
mind issues such as resources, existing
procedures, etc.?
QA Focus – Supporting JISC's Digital Library Programmes
33
QA Focus Methodology
The QA Focus Methodology
The QA Focus methodology which we are
advising projects (and JISC) to deploy:
• Based on well-established QA principles
• Seeks to be lightweight and achievable
• Seeks to provide benefits to the projects
themselves, as well as (downstream)
interoperability benefits to JISC programmes
Consists of:
•
•
•
•
Documented policies
Procedures for monitoring compliance with policies
Audit trails
Sharing and collaboration
QA Focus – Supporting JISC's Digital Library Programmes
34
QA Focus Methodology
Applying QA – Standards
Policy:
Open standards used where possible.
Proprietary formats with published spec and
cross-platforms viewers and authoring tools
may be acceptable
Monitoring: Project manager responsible
Documentation:
Document on standards (cf QA Focus doc)
Exceptions: If open standards not available, immature
or costly to implement proprietary standards
may be used.
The project manager should document
such decisions: why proprietary format was
used, migration strategies to open formats
in future, architecture for
migration and indication of costs
QA Focus – Supporting JISC's Digital Library Programmes
35
QA Focus Methodology
Applying QA – Web Standards
Policy:
The Web site will use XHTML 1.0 and
CSS 2.0 standards
Architecture: The Web site will be based on XHTML
templates and SSI
Monitoring:
New and updated pages validated using
,validate and ,cssvalidate. Every
month ,rvalidate will be used
Exceptions:
HTML derived automatically (e.g. Save As
HTML in PowerPoint) need not comply
with standards. The files will be stored in
a standard directory to enable such files
to be excluded from checks
QA Focus – Supporting JISC's Digital Library Programmes
36
QA Focus Methodology
Applying QA – Proprietary Formats
Policy:
Monitoring:
Proprietary formats may be deployed on
Web site in certain documented
circumstances including MS Office files,
PDF and Flash
MS Office files should also be stored in
most open HTML format.
PDF files should include link to online PDF
converter
Flash can be used for specific applications
in which it is needed.
Project manager
QA Focus – Supporting JISC's Digital Library Programmes
37
QA Focus Methodology
Applying QA – Accessibility (1)
Policy:
The Web site will strive to attain WAI A
guidelines.
Consistent accessibility shortcuts will be
used.
An accessibility policy will be published.
Architecture: The Web site will be based on XHTML
templates which comply with WAI A.
Monitoring: New and updated pages will be validated
using ,bobby. A monthly batch checker
will be used and audit reports published (to
enable any trends to be spotted).
Exceptions: A list of permitted exceptions will be
provided.
Note that University of Edinburgh Library
have
a useful Web accessibility policy
QA Focus – Supporting JISC's Digital Library Programmes
38
Applying QA – Accessibility (2)
Policy:
The organisation has no
accessibility policy
Authors are free to implement their
own accessibility shortcuts (if at all)
Architecture: No centralised policy covering
authoring tools or architecture will
be provided
Monitoring: No monitoring will be carried out
QA Focus – Supporting JISC's Digital Library Programmes
39
QA Focus Methodology
Applying QA – Usability (1)
Policy:
A structured usability test will be applied
using at least 5 users for new Web sites
Tailored 404 error pages will be provided
Use of <link rel="home" …> to
support navigation across Web site
Use of <link rel="next" …> to
support navigation in structured areas of
Web site
Architecture: The <link rel="next" …> elements
should be implemented by automated
scripts
Monitoring:
Need to check the next, previous goes to
correct page.
Exceptions: …
QA Focus – Supporting JISC's Digital Library Programmes
40
QA Focus Methodology
Applying QA – Usability (2)
Policy:
Monitoring:
Exceptions:
Broken links to be kept to a minimum
The ,checklink command applied to
new and updated resources and
,checklink run monthly
The Xenu link checker will be run monthly
and a report published, in order for any
trends to be monitored
Don't fix broken links in published
documents or derived files.
Need to formulate policy on large
numbers of broken links
(cf JISC Web site)
QA Focus – Supporting JISC's Digital Library Programmes
41
Recommendations
QA Focus recommendations are likely to include:
• Project proposals should describe approaches to
selection of standards, technical architecture & QA
• Once approved initial project report should document:
 Standards chosen (e.g. see "ESDS Web
Standards Policy" case study and "Matrix for
Selection of Standards" briefing document)
 Justification for any deviations from open
standards (e.g. see NOF-TAS FAQ)
 Technical architecture
 Intended service deployment
environment
QA Focus – Supporting JISC's Digital Library Programmes
42
Reality Check
Is this really needed? The pages are fine in my
browser.
But:
• What about repurposing resources (XHTML  RSS)?
• What about new devices (using wireless PDA in
student's union)?
• What about accessibility?
• What about tomorrow's maintenance nightmare?
• What about the exploitation of XML resources
(quicker, richer, …)?
• What about the vision of universal access, and
removal of platform and application dependencies?
QA Focus – Supporting JISC's Digital Library Programmes
43
Getting There
I want to improve the quality of my Web services. But
I've got a large Web site, investment in current systems,
inertia, … So:
• Deploy QA procedures in new developments
• Use tools such as W3C's Log Validator to prioritise
key areas - <http://www.w3.org/QA/Tools/>
•
•
•
•
•
A CMS can help
Consider use of Tidy on server (cf. LSE approach)
Ensure output from backend scripts is compliance
Revisit MS Office  Web tools
Plan migration strategy to XHTML 2 world
(compliant XHTML 1.0  XSLT  XHTML 2.0 = )
Remember
you
already
have QA procedures
QA•Focus
– Supporting JISC's
Digital
Library Programmes
44
Conclusions
We have described:
• The approaches to QA which have been taken by
the QA Focus project
• Findings of a number of surveys which have
highlighted common problems
• An example of an approach to providing access
to testing tools using a URI-interface
• A systematic methodology for QA
• Recommendations for future programmes
QA Focus – Supporting JISC's Digital Library Programmes
45
Questions
Any questions?
QA Focus – Supporting JISC's Digital Library Programmes
46