Transcript QA For Metadata: The QA Focus Methodology

QA For Metadata:
The QA Focus Methodology
Brian Kelly, UKOLN
Supported by
Amanda Closier, UKOLN and
Gareth Knight, AHDS
http://www.ukoln.ac.uk/qa-focus/
QA Focus – Supporting JISC's Digital Library Programmes
1
This session
addresses the
importance of
quality assurance
for metadata and
describes the QA
methodology
developed by JISCfunded QA Focus
project.
Background
QA Focus has sought to:
• Help ensure the functionality, wide accessibility
and interoperability of JISC-funded projects
• Provide advice and support materials to projects
• Recognise the limited resources available to
projects
• Develop a light-weight methodology which can
help projects achieve interoperability and
compliance with appropriate standards and best
practices
• Develop a self-assessment methodology which
recognises the difficulties of external checking
QA Focus – Supporting JISC's Digital Library Programmes
2
Why Do Things Go Wrong?
Why do project deliverables fail to be:
• Functional
• Interoperable
 Widely accessible
 Easily deployed into service
Reasons include:
• Failure to implement standards & best practices
(perhaps due to a lack of understanding)
• Inappropriate standard / best practice used
• Use of an inappropriate architecture (tools,
workflow, …)
• Failure to check that standards & best practices are
being used correctly
• Failure to understand limitations of checking tools
• Having over-optimistic intentions
QA Focus – Supporting JISC's Digital Library Programmes
3
QA Focus Methodology
The QA Focus methodology:
• Based on mainstream QA principles
• Projects should develop written policies on
standards, best practices, etc.
• Projects should deploy systematic procedures for
ensuring the policies are implemented
• Projects should share experiences (good and bad)
Note that:
4
• The policies should be achievable
• The procedures may lead to changes in systems,
workflow, etc. if non-compliance spotted
• Audit trails should be kept to help identify trends,
QA Focus
– Supporting
JISC's Digital Library
Programmes of best practices, etc.
provide
documented
evidence
Policies
How do you know what you should do if you don't
have documented polices?
Policy example
Policy: Web Standards
Standard: XHTML 1.0 and CSS 2.0
Architecture: Use of SSIs and text editor
Exceptions: Automatically-derived files
Checking: Use ,validate after update
Audit Trail: Use ,rvalidate monthly and
document findings
Please note that this is a template – you can add
additional headings
QA Focus – Supporting JISC's Digital Library Programmes
5
QA Focus Methodology
Applying QA – Accessibility (1)
Policy:
The Web site will strive to attain WAI A
guidelines.
Consistent accessibility shortcuts will be
used.
An accessibility policy will be published.
Architecture: The Web site will be based on XHTML
templates which comply with WAI A.
Monitoring: New and updated pages will be validated
using ,bobby. A monthly batch checker
will be used and audit reports published (to
enable any trends to be spotted).
Exceptions: A list of permitted exceptions will be
provided.
Note that University of Edinburgh Library
have
a useful Web accessibility policy
QA Focus – Supporting JISC's Digital Library Programmes
6
Applying QA – Accessibility (2)
Policy:
The organisation has no
accessibility policy
Authors are free to implement
their own accessibility shortcuts (if
at all)
Architecture: No centralised policy covering
authoring tools or architecture will
be provided
Monitoring: No monitoring will be carried out
If you don't have a written policy, the unwritten policy
may well be frightening!
QA Focus – Supporting JISC's Digital Library Programmes
7
Who Checks Compliance?
QA Focus recommendations:
• Projects should implement appropriate QA
procedures
• Projects may find QA Focus support materials
helpful (including case studies)
• The QA Focus toolkits may be used to help ensure
appropriate areas are addressed
• Compliance issues should be addressed within
projects with the expectation that significant
discrepancies are brought to attention of funders
• External auditing of compliance is not felt to be
appropriate for the JISC sector / culture
QA Focus – Supporting JISC's Digital Library Programmes
8
Applications To Metadata
For metadata there is a need to:
• Clarify the purpose of your metadata (embedded
metadata in Web pages won't put you at the top of
Google)
• Have an appropriate architecture for creating and
managing your metadata - metadata rots
• Ensure you have appropriate cataloguing rules
• Address interoperability with others and not just your
own functionality – this is hard
• Provide appropriate training, documentation, etc.
• Have appropriate mechanisms for ensuring above work
correctly
• Document the above – for yourselves, for new staff and
potentially for 3rd parties (funders, service providers, …
QA Focus – Supporting JISC's Digital Library Programmes
9
Example 1 – ACRONYM Tag
Simple example which
illustrates several points:
• Why bother (not supported
in IE)?
• Difference between Acronym
and Abbreviation
• Policy on content (full stops,
apostrophes, US vs UK English, …)
• Error checking – when the metadata is not displayed
Solution
• Simple policy produced.
• Used Tom Heath's acronym harvester to create automated
glossary as (a) value-added service and (b) to check for errors.
• Policy and approach documented
QA Focus – Supporting JISC's Digital Library Programmes
10
Example 2 – Web Metadata
QA Focus Web site:
• Uses PHP to process simple metadata (e.g.
$author="Brian Kelly", $area ="documents",
&keywords="xxx"…)
• Used for processing ($area used to pull in
stylesheet) as well as resource discovery
• Architecture isn't ideal (backend database better) –
but QA is about managing what you've got
• How do we detect errors?
11
Solution
• HTML validation detects incorrect PHP syntax errors
• Visual inspection for correct &area values
• Keywords & description copied to hard copy document, which
allowed for visual checking (and there were errors)
QA•Focus
– Supporting
JISC's Digital
Library Programmes
Policy
documented
(shortly)
What Needs To Be Addressed?
You'll need to address:
• Purpose of metadata
• Selection of standards, conventions, …
• Implementation architecture(s)
• Work flow
• Project outputs
• …
Note:
• Some aspects will be already chosen or outside of you
control (e.g. FAIR projects will use OAI, …)
QA Focus – Supporting JISC's Digital Library Programmes
12
Help Is Available
Briefing Documents
• See <http://www.ukoln.ac.uk/qafocus/documents/briefings/#metadata>
Case Studies
• See <http://www.ukoln.ac.uk/qafocus/documents/case-studies/#metadata>
Toolkits
• See <http://www.ukoln.ac.uk/qa-focus/toolkit/>
You can contribute to the case studies: brief documents covering (a) the
project (b) area addressed (c) solution used (d) lessons learnt.
You document experiences for sharing with community – and we promote
you (at events
and on Google-friendly Web site) e.g. Healthier Nation
QA Focus – Supporting JISC's Digital Library Programmes
13
Toolkits
Online and paper based
toolkits are available
14
Note the toolkits won't solve your
problems for you – they are meant as
QA Focus – Supporting JISC's Digital
Programmes
an Library
aid to
help you address key areas
I'm Too Busy!
Projects may feel they are too busy to take on this work. However:
15
• You're probably doing much of the work already!
• QA methodology has been designed to be lightweight
• It is needed in order to provide the interoperability and
wide access for which projects are funded
• Documented QA policies should help ensure the
functionality of your deliverables (e.g. when staff leave)
• There is more likelihood of your project deliverables been
deployed into service by others if QA issues have been
addressed
• QA Focus recommends that future JISC programmes
require QA to be addressed
• Projects themselves may prefer the lightweight selfassessment approach than an external audit – could this
QA Focus
– Supporting
JISC's Digital (such
Library Programmes
happen
if disasters
as UKeU) occur?
Your Feedback
Feedback on our approaches are welcome
Please complete evaluation forms
QA Focus – Supporting JISC's Digital Library Programmes
16
Questions
Any questions?
QA Focus – Supporting JISC's Digital Library Programmes
17