Transcript MS PowerPoint 97/2000 format

QA For Digital Library
Projects
Brian Kelly
UKOLN
[email protected]
This talk will describe
the work of the JISCfunded QA Focus
team and outlines
approaches to QA
which can be
deployed in Digital
Library projects
http://www.ukoln.ac.uk/qa-focus/
QA Focus – Supporting JISC's 5/99 Programme
Contents
• Background:
• The need for QA
• QA Focus
• Work activities:
• Web
• Accessibility
• Providing Advice
• Future Work
•
•
•
•
Metadata
Software Development
Service Deployment
Toolkit
• Synergies With Other Groups
• Conclusions
QA Focus – Supporting JISC's 5/99 Programme
Introduction
Aims Of Today’s Talk
• To describe the background to the QA
Focus project
• To summarise QA Focus activities to
date
• To describe future activities
• To explore potential for links with CDLR
QA Focus – Supporting JISC's 5/99 Programme
EIB
Introduction
An Introduction to QA
What is Quality?
“Quality is the ability of your product to be able to
satisfy your users”
What is Quality Assurance?
“Quality assurance is the process that demonstrates
your product is able to satisfy your users”
QA Focus – Supporting JISC's 5/99 Programme
Introduction
An Introduction to QA
What does Quality Assurance give?
• ‘Quality’ means your project is ‘useful’ without ‘quality’ you may have little to offer
• ‘Quality’ can help to future-proof projects
• But ‘quality assurance’ needs documented
standards and best practices to be meaningful
• ‘Quality’ & ‘Best Practice’ can only be considered
in terms of being ‘Fit for Purpose’
QA Focus – Supporting JISC's 5/99 Programme
Introduction
An Introduction to QA
Why Is QA Needed Now?
In the past JISC had standards-based philosophy but:
• Approach of "let a 1,000 flowers bloom"
• Diversification as no killer application (it could have
been Gopher!)
Things are different now:
• The Web is the killer application
• There is now a need for interoperability (now selfcontained independent Web sites)
• There is a need for publicly-funded services to be
seen to provide value for money
QA Focus – Supporting JISC's 5/99 Programme
Introduction
Approach Taken
Two possible approaches to ensuring
compliance with standards and best practices:
Enforce
• Inspect all project’s work
• Strict auditing, with penalties for no-compliance
Encourage
• Training for project staff
• Developmental, explaining reasons for compliance,
documenting examples of best practices and
providing advice on implementation and monitoring
The latter approach is preferable,
especially in a HE context
QA Focus – Supporting JISC's 5/99 Programme
QA Focus
QA Focus:
•
•
•
•
Funded by JISC
Currently provided by UKOLN and AHDS
Currently supports JISC's 5/99 programme
Staff:
 Brian Kelly, Project manager
 Marieke Guy, QA Focus officer, UKOLN
 Hamish James, QA Focus officer, AHDS
QA Focus – Supporting JISC's 5/99 Programme
QA For Digitisation
QA for Digitisation
Do it once…..do it right:
• Project is fundamentally dependent upon
the quality of original product
• Quality is the pre-requisite to preservation
• Quality expectations will only grow
• Delivery problems can be fixed, but
capture problems normally can’t
QA Focus – Supporting JISC's 5/99 Programme
QA For Digitisation
QA Philosophy
A multi-level approach may be taken to
QA:
• Strategic QA Carried out before
digitisation starts
Research and
establishing best practice
& standards
QA Focus – Supporting JISC's 5/99 Programme
QA For Digitisation
QA Philosophy
A multi-level approach may be taken to
QA:
• Strategic QA
• Workflow QA
Formative assessment,
before & during
development
Establishing &
documenting workflow
& processes
QA Focus – Supporting JISC's 5/99 Programme
QA For Digitisation
QA Philosophy
A multi-level approach may be taken to
QA:
• Strategic QA Quality Control :
• Workflow QA Summative assurance
at end of each process,
• Sign-off QA providing an audit
history for all QA work
undertaken
QA Focus – Supporting JISC's 5/99 Programme
QA For Digitisation
QA Philosophy
A multi-level approach may be taken to
QA:
• Strategic QA
Summative assurance
• Workflow QA as part of long term QA
to establish a system to
• Sign-off QA
report, check & fix any
• On-going QA faults found in future
QA Focus – Supporting JISC's 5/99 Programme
QA For Digitisation
QA Philosophy
QA Focus promotes a multi-level
approach to digitisation:
• Strategic QA
• Workflow QA
• Sign-off QA
High Quality Product
• On-going QA
QA Focus – Supporting JISC's 5/99 Programme
QA For Web
QA For Web Sites
The issues:
• The Web is the main delivery mechanism for projects
and services
• An increasing awareness of the importance of:
 Accessibility
 Use of new devices (PDAs, WAP, e-books, …)
 Repurposing of Web content (e.g. archiving)
• Technologies such as XSLT will support repurposing
of valid XML resources
But:
• Invalid HTML is the norm
• Many authoring tools produce poor HTML
• Authors QA
aren’t
the5/99
problems
Focus –aware
Supportingof
JISC's
Programme
QA For Web
Guidelines
We often say:
• Open standards are important
• HTML, XML, XHTML, CSS, … are important
but fail to explain why and how
JISC’s QA Focus is addressing such concerns by:
• Documenting example of best practices in which
projects can share their implementation successes
(and difficulties they experienced)
• Provide brief advice in specific aspects of the
standards and best practices
• Surveying its communities to highlight best
practices and areas in which improvements can be
made
• Demonstrating use of testing tools and procedures
QA Focus – Supporting JISC's 5/99 Programme
QA For Web
Standards & Best Practices
Standards For Web:
• Use compliant HTML / XHTML
• Use CSS
• Support WAI accessibility guidelines
Best Practices For Web:
• Ensure Web resources are suitable for reuse and
repurposing
• Where proprietary formats need to be used, flag
them and use in most open way
QA Focus – Supporting JISC's 5/99 Programme
QA For Web
Surveying The Community
Surveys of project Web sites have been carried
out in order to:
• Obtain a profile for the community
• Identify examples of best practices
• Identify areas in which further advice is needed
Surveys included:
• HTML & CSS compliance
• 404 error pages
• Repurposing resources
 Accessibility
 HTTP headers
QA Focus – Supporting JISC's 5/99 Programme
QA For Web
Survey Philosophy
The surveys made use of freely-available Web-based
tools:
• Methodology is open
• No software needs to be installed locally (apart from Web
browser)
• Findings can be reproduced
• Latest results can be obtained by clicking on link to testing
service
The surveys typically examined project entry points and
not entire Web site as:
• This page has the highest profile
• The aim is to validate a methodology which can be deployed
by projects themselves, not to test every page on behalf of
the projects
QA Focus – Supporting JISC's 5/99 Programme
QA For Web
Survey Findings
Initial set of findings available from
<http://www.ukoln.ac.uk/qa-focus/surveys/web-10-2002/>
QA Focus – Supporting JISC's 5/99 Programme
QA For Web
Providing Motivation
We have found evidence of failure to comply with HTML
standards
There is a need to explain why compliance is important
(and avoid the “it’s OK in my browser” argument) and to
provide motivation for projects to update their tools,
authoring procedures, etc.
A further set of surveys look at repurposing of the
project Web sites:
• Availability of Web sites in the Internet Archive
• Ease of making Web sites available on a PDA
• Transformation of embedded metadata
QA Focus – Supporting JISC's 5/99 Programme
QA For Web
Repurposing Resources
We examined the Web sites to see if they were
available in the Internet Archive and could be
transformed into a format for viewing on a PDA
small number
AAsmall
numberof
sites
could not
ofWeb
Web
sites
be transformed.
were not in the
Analysis of HTTP
Internet
headers Archive
indicated
due
that to
thisthe
was due to
robots.txt
incorrect HTTP file.
headers.
We
will need to
We will need
to in
provide
advice
provide
advice in
this
area.
this area.
QA Focus – Supporting JISC's 5/99 Programme
QA For Web
Transforming Resources
Project entry points were processed by several online
transformation services in order to validate and
visualise embedded Dublin Core metadata
Tidy
(online)
Virtual
XHTML
resource
Visualisation
& validation
of DC
RDF
Validator
HTML
resource
Original page,
containing
embedded DC
metadata
QA Focus – Supporting JISC's 5/99 Programme
XSLT
extraction
of DC
DC
in RDF
format
Advice
Providing Advice
We have:
• Survey project Web sites and identified areas of lack of
compliance with standards and best practices
• Demonstrated examples of the potential importance of
compliance for repurposing resources
In addition we need to provide:
•
•
•
•
Brief focussed advice on the standards
Information on how to monitor compliance
Case studies on solutions deployed by projects themselves
Guidance on dealing with implementation difficulties and
what to do when strict compliance is difficult to achieve
QA Focus – Supporting JISC's 5/99 Programme
Advice
Documentation: Advice
Advisory briefing
documents are being
produced
These are:
• Brief, focussed
documents
• Informed by findings
of the surveys
QA Focus – Supporting JISC's 5/99 Programme
Advice
Documentation: Case Studies
Case Studies are
being commissioned
These are:
• Written by projects
themselves
• Describe the
solution adopted to
a particular problem
• Include details of
lessons learnt – not
just a press release!
QA Focus – Supporting JISC's 5/99 Programme
Putting it Together
Based on surveys we find Project Web sites which:
• Have invalid HTML and CSS
• Don't have helpful 404 pages
• …
So we:
• Explain why projects should follow best practices
(e.g. repurposing XHTML pages)
• Provide case studies on approaches by projects
• Describe tools and architectures for projects to
implement best practices
• Describe tools and architectures for
projects to check compliance with best
practices
QA Focus – Supporting JISC's 5/99 Programme
Describing Tools and Approaches
As an example of approach
which provides easier access
to testing tools, see the
,tools approach deployed
on UKOLN Web site.
This approach:
• Applies to all resources on
Web site
• Covers HTML and CSS
validation and various
other tests
• Some recursive apps
(,rvalidate)
• Easily implemented with
single line redirect
QA Focus – Supporting JISC's 5/99 Programme
Limitations
There are a number of limitations to the work we have
carried out so far:
• Project Web sites have different purposes
(information about the project; communications with
project partners; project deliverables themselves;
etc.)
• Projects have different levels of funding, resources,
expertise, etc.
• Projects are at different stages of development (and
some have finished)
The surveys are intended to demonstrate a methodology
which projects can use for themselves
QA Focus – Supporting JISC's 5/99 Programme
Next Steps
Next Steps
Extended Coverage
We will be moving on from Web and digitisation to
include other areas including:
Metadata
Software development
…
 Multimedia
 Deployment into service
Moving On From Automated Testing
The initial work made use of automated testing tools:
Can be used remotely
 Objective
Applicable across all projects
We have started work on QA procedures in areas
which are not suitable for automated
checking
QA Focus – Supporting JISC's 5/99 Programme
Next Steps
Self Assessment Toolkit
Further Deliverables
We will be developing a self-assessment toolkit for
projects to use, by individual projects or across project
clusters
The toolkit will consist of:
• Examples of QA procedures
• Documented examples of use of testing tools
• Self-assessment questionnaires
• Advice on standards and best practices
• Case studies
• FAQs
• …
QA Focus – Supporting JISC's 5/99 Programme
Next Steps
Next Steps – Metadata
Thoughts on QA for metadata:
Syntax
Check syntax on embedded DC
Cataloguing Rules Ensure projects have appropriate
cataloguing rules
Interoperability
Ensure metadata can
interoperate with third parties
Fitness for purpose Ensure metadata is appropriate
for its purpose
Change control
Ensure architecture for managing
metadata can cope with change
control (cf Exploit Case Study)
QA Focus – Supporting JISC's 5/99 Programme
Next Steps
Next Steps – Software
Thoughts on QA for software:
Specification
Process for developing systems
spec, agreeing spec with
stakeholders, etc.
Methodology
Documentation of systems
development methodology (cf UML
case study)
Testing
Documentation of testing
methodology
Deployment
Awareness of deployment
challenges
QA Focus – Supporting JISC's 5/99 Programme
Next Steps
Next Steps – Service Deployment
Thoughts on QA for service deployment:
Development Architecture
Early public documentation of
development architecture
Service Architecture
Awareness of likely service architecture
and liaison with service
Security
Awareness of security concerns
Licensing
Licencing issues
Scenarios
Software developed using cool open software environment – but
JISC Service has no knowledge of environment.
Resources digitised, but copyright clearance not documented.
QA Focus
– Supporting
5/99 Programme
Service refuses to take
delivery
dueJISC's
to liability
concerns.
QA Focus And CDLR
What relevance has QA Focus for CDLR?
• You can write case studies and advisory
documents:
 Seen to be working with JISC
 We can promote your approaches (and
therefore your work)
 This will help instigate best practices within
your group
QA Focus – Supporting JISC's 5/99 Programme
QA Focus And CDLR
What relevance has QA Focus for CDLR?
• You can deploy QA Focus practices for your
projects, etc:
 You should benefit
 You can provide feedback to QA Focus
• You can develop QA procedures in innovative
areas (e.g. OAI, thesaurii, ePrints, e-Books, etc.)
which QA Focus can use:
 You gain kudos, publications, etc.
 We can help embed your practices
across the community
QA Focus – Supporting JISC's 5/99 Programme
QA Examples
Applying QA – Standards
Policy:
Open standards used where possible.
Proprietary formats with published spec and
cross-platforms viewers and authoring tools
may be acceptable
Monitoring: Project manager responsible
Documentation:
Document on standards (cf QA Focus doc)
Exceptions: If open standards not available, immature
or costly to implement proprietary standards
may be used.
The project manager should document
such decisions: why proprietary format was
used, migration strategies to open formats
in future, architecture for
migration and indication of costs
QA Focus – Supporting JISC's 5/99 Programme
QA Examples
Applying QA – Web Standards
Policy:
The Web site will use XHTML 1.0 and
CSS 2.0 standards
Architecture: The Web site will be based on XHTML
templates and SSI
Monitoring:
New and updated pages validated using
,validate and ,cssvalidate. Every
month ,rvalidate will be used
Exceptions:
HTML derived automatically (e.g. Save As
HTML in PowerPoint) need not comply
with standards. The files will be stored in
a standard directory to enable such files
to be excluded from checks
QA Focus – Supporting JISC's 5/99 Programme
QA Examples
Applying QA – Proprietary Formats
Policy:
Monitoring:
Proprietary formats may be deployed on
Web site in certain documented
circumstances including MS Office files,
PDF and Flash
MS Office files should also be stored in
most open HTML format.
PDF files should include link to online PDF
converter
Flash can be used for specific applications
in which it is needed.
Project manager
QA Focus – Supporting JISC's 5/99 Programme
QA Examples
Applying QA – Accessibility
Policy:
The Web site will strive to attain WAI A
guidelines
Consistent accessibility shortcuts will be
used
An accessibility policy will be published
Architecture: The Web site will be based on XHTML
templates which comply with WAI A
Monitoring:
New and updated pages will be validated
using ,bobby. Every month a batch
checker will be used. Monthly audit
reports will be published (to enable any
trends to be spotted)
Exceptions: A list of permitted exceptions
will be provided
QA Focus – Supporting JISC's 5/99 Programme
QA Examples
Applying QA – Usability (1)
Policy:
A structured usability test will be applied
using at least 5 users for new Web sites
Tailored 404 error pages will be provided
Use of <link rel="home" …> to
support navigation across Web site
Use of <link rel="next" …> to
support navigation in structured areas of
Web site
Architecture: The <link rel="next" …> elements
should be implemented by automated
scripts
Monitoring:
Need to check the next, previous goes to
correct page.
Exceptions: …
QA Focus – Supporting JISC's 5/99 Programme
QA Examples
Applying QA – Usability (2)
Policy:
Monitoring:
Exceptions:
Broken links to be kept to a minimum
The ,checklink command applied to
new and updated resources and
,checklink run monthly
The Xenu link checker will be run monthly
and a report published, in order for any
trends to be monitored
Don't fix broken links in published
documents or derived files.
Need to formulate policy on
large numbers of broken links
(cf redesign of JISC Web site)
QA Focus – Supporting JISC's 5/99 Programme
Questions
Any questions?
QA Focus – Supporting JISC's 5/99 Programme