Transcript Quality Assurance For Digital Library Programmes: The QA Focus

Quality Assurance For
Digital Library Programmes:
A Practical Workshop
Workshop Aims
• Learn about QA for
digital library work
Brian Kelly
• Hear about QA
UK Web Focus
Focus's activities
UKOLN
• Identify QA
requirements in
Email: [email protected]
metadata & service
deployment
• Work on solutions
http://www.ukoln.ac.uk/qa-focus/ • Agree on follow-up
actions
QA Focus – Supporting JISC's Digital Library Programmes
1
Timetable
14:00
14:20
14:50
15:15
15:30
15:45
16:05
16:30
16:40
17:00
Why Do We Need QA? / What Is QA Focus? [BK]
What Can Go Wrong In Your Project? [ALL]
Report back [Group Reporters]
Coffee
The QA Focus Approach [BK]
QA For Metadata – An Example [MG]
Embedding QA in your Project [ALL]
Report Back [Group Reporters]
What Next? (Get some answers and win a prize!)
Finish (?)
QA Focus – Supporting JISC's Digital Library Programmes
2
About Us
The workshop organised by:
• QA Focus – a JISC-funded advisory service
provided by UKOLN and AHDS which supports
JISC's digital library programmes (5/99, and now
FAIR and X4L)
• Brian Kelly, QA Focus project manager (and UK
Web Focus)
• Additional support provided by:
 Marieke Guy (UKOLN): formerly QA Focus
officer and now SPP and E-Prints UK project
manager
 Gareth Knight (AHDS): QA Focus officer
Note Amanda Closier (UKOLN) will be
working on QA Focus from January
QA Focus – Supporting JISC's Digital Library Programmes
3
Aims of the Session
The aims of the QA session are:
• To provide participants will an awareness of the
importance of quality assurance (QA) in ensuring
that project deliverables are (a) functional (b)
interoperable (c) widely accessible and
(d) re-purposable
• To find out about QA Focus's resources and gain an
understanding of the QA Focus methodology
• To make use of the QA Focus methodology in an
area of your work
• To have an opportunity to look and evaluate QA
Focus resources
• To agree on areas to be addressed by projects
QA Focus – Supporting JISC's Digital Library Programmes
4
Introduction
5
Why Do We Need QA?
Why Is QA Needed Now?
In the past JISC had standards-based philosophy but:
• Approach of "let a 1,000 flowers bloom"
• Diversification as no killer application then
(it could have been Gopher!)
Things are different now:
• The Web is the killer application
• There is now a need for interoperability (not just selfcontained independent Web sites)
• JISC Programmes increasingly intended as service
development and not 'speculative' research
• There is a need for publicly-funded services to be
seen
to provide
value
money
QA Focus
– Supporting
JISC's Digital
Libraryfor
Programmes
Technical Areas Addressed
The areas addressed by QA Focus are:
•
•
•
•
•
•
Standards
Digitisation
Access (Web)
Metadata
Software
Service Deployment
The areas of standards, digitisation and access/Web
have been addressed and a wide range of advice is
currently available.
In this session we will focus on the areas of metadata,
software and service deployment
QA Focus – Supporting JISC's Digital Library Programmes
6
E
Group Exercise 1
• Split into a number of small groups
• Choose a reporter
• Address the question "What can go wrong with
my project?"
• The focus should be primarily on technical
challenges (i.e. not 'all the staff leave')
• Address primarily the areas of metadata, software
and service deployment
QA Focus – Supporting JISC's Digital Library Programmes
7
Report Back – Exercise 1
What can go wrong with your project?
What have you identified?
QA Focus – Supporting JISC's Digital Library Programmes
8
Coffee Break
We should now break for coffee
QA Focus – Supporting JISC's Digital Library Programmes
9
What Can Go Wrong?
We have previously identified the following:
• A lack of understanding of appropriate standards
and best practices
• A failure to make use of appropriate implementation
architectures
• A lack of awareness of failure to comply with
appropriate standards and best practices
• A lack of awareness of checking & validation tools
and procedures (and their limitations)
• Using quality control rather than implementing
quality assurance procedures
QA Focus is addressing these
high level areas
QA Focus – Supporting JISC's Digital Library Programmes
10
Introduction
Approach Taken By QA Focus
Two possible approaches to avoiding problems:
Enforce
• Inspect all project’s work
• Strict auditing, with penalties for no-compliance
Encourage
• Developmental:
 Working with community
 Explaining reasons for compliance
 Documenting examples of best practices
 Providing advice on implementation & monitoring
 Developing a QA methodology
 Seeking to embed QA in working practices
11
The latter approach is felt to be preferable,
QA Focus – Supporting
Library Programmes
especially
in aJISC's
HE Digital
context
QA Focus Activities
12
QA Focus activities:
• Focus groups identified key concerns (in areas of
standards, implementation and service deployment)
• Surveys profiled approaches, best practices and
common problems
• Provision of focussed advice primarily addressing:
 Standards and best practices
 Implementation issues
 Compliance checking
• Case studies describing examples of best practices
• Development of a QA methodology
• Validation of approach
QA•Focus
– Supporting JISC's
Digital
Library Programmes
Embedding
QA
methodology
within projects
Surveys
Survey Findings
Surveys of compliance of 5/99 project entry points helped
to profile community and identify common problems and
best practices
QA Focus – Supporting JISC's Digital Library Programmes
13
Advice
Documentation: Advice
Advisory briefing
documents are being
produced
These are:
• Brief, focussed
documents
• Informed by
findings of the
surveys
• Over 50 briefing
documents have
been published to
date
QA Focus – Supporting JISC's Digital Library Programmes
14
Advice
Documentation: Case Studies
Case Studies have been
commissioned
These are:
• Written by projects
themselves (helping
to achieve 'buy-in')
• Describe the solution
adopted to a
particular problem
• Include details of
lessons learnt – not
just a press release!
• Over 30 published
QA Focus – Supporting JISC's Digital Library Programmes
15
QA For Web
Surveys For FAIR & X4L Programmes
QA Focus now
supports JISC's
FAIR and X4L
programmes
We have
developed a selfassessment
interface for
projects
http://www.ukoln.ac.uk/qa-focus/
surveys/fair/web-self-assessment/
The aim is that projects will (a) check and, if necessary,
fix their home page; (b) check/fix other key pages and
(c) address underlying causes of problems
16
"I have found the webpage very useful for testing the
technical compliance and accessibility of our website"
QA Focusthanks
– Supporting
Library
Programmes
"Many
forJISC's
this -Digital
what
a really
useful tool!"
Tools
Putting it Together
We have:
• Explained why projects should implement standards
and best practices (they don't always know)
• Provide case studies on approaches by projects
• Described implementation architectures
• Described various checking tools
But how do we ensure projects :
• Check deliverables for themselves
• Have systematic checking procedures
There is a need to describe:
• Tools & architectures for projects for check
compliance with standards & best practices
• Embed systematic QA procedures
QA Focus – Supporting JISC's Digital Library Programmes
17
QA Focus Methodology
The QA Focus Methodology
The QA Focus methodology which we are advising
projects to deploy:
• Based on well-established QA principles
• Seeks to be lightweight and achievable
• Seeks to benefit the projects themselves, as well as
interoperability benefits to JISC programmes
Consists of:
• Documented policies
• Procedures for monitoring compliance with policies
• Audit trails
• Sharing and collaboration
See <http://www.ukoln.ac.uk/qa-focus/documents/briefings/briefing-30/>
QA Focus – Supporting JISC's Digital Library Programmes
18
QA Focus Methodology
Applying QA – Web Standards
Policy:
The Web site will use XHTML 1.0 and
CSS 2.0 standards
Architecture: The Web site will be based on XHTML
templates and SSI
Monitoring:
New and updated pages validated using
,validate and ,cssvalidate. Every
month ,rvalidate will be used
Exceptions:
HTML derived automatically (e.g. Save As
HTML in PowerPoint) need not comply
with standards. The files will be stored in
a standard directory to enable such files
to be excluded from checks
QA Focus – Supporting JISC's Digital Library Programmes
19
QA Focus Methodology
Applying QA – Accessibility (1)
Policy:
The Web site will strive to attain WAI A
guidelines.
Consistent accessibility shortcuts will be
used.
An accessibility policy will be published.
Architecture: The Web site will be based on XHTML
templates which comply with WAI A.
Monitoring: New and updated pages will be validated
using ,bobby. A monthly batch checker
will be used and audit reports published (to
enable any trends to be spotted).
Exceptions: A list of permitted exceptions will be
provided.
Note that University of Edinburgh Library
have
a useful Web accessibility policy
QA Focus – Supporting JISC's Digital Library Programmes
20
Applying QA – Accessibility (2)
Policy:
The organisation has no
accessibility policy
Authors are free to implement their
own accessibility shortcuts (if at all)
Architecture: No centralised policy covering
authoring tools or architecture will
be provided
Monitoring: No monitoring will be carried out
If you don't have a written policy, the unwritten policy
may well be frightening!
QA Focus – Supporting JISC's Digital Library Programmes
21
Next Steps
Self Assessment Toolkit
Further Deliverables
We are developing a self-assessment toolkit for projects
to use, by individual projects or across project clusters
The toolkit consists of:
• Self-assessment questionnaires
• Examples of QA procedures
• Documented examples of use of testing tools
• Advice on standards and best practices
• Case studies
• FAQs
• …
QA Focus – Supporting JISC's Digital Library Programmes
22
Next Steps
Self Assessment Toolkit (1)
QA Focus – Supporting JISC's Digital Library Programmes
23
Self Assessment Toolkit (2)
The toolkit is also
available in paper
format, suitable for use
in workshops,
meetings, etc.
This approach will be
used in today's QA
workshop
Toolkits on Metadata,
Software and Service
Deployment will be
used
QA Focus – Supporting JISC's Digital Library Programmes
24
QA For Metadata – An Example
Presentation By Marieke Guy
QA Focus – Supporting JISC's Digital Library Programmes
25
Group Exercise 2
Embedding Quality Assurance Into Your
Project Work
In your groups use the QA framework to define:
• Appropriate policies
• Appropriate checking procedures
• Audit trails
In the areas of metadata, software and service
deployment
You may use the metadata, software and service
deployment toolkits.
QA Focus – Supporting JISC's Digital Library Programmes
26
Report Back – Exercise 2
Please provide a brief report back on:
• Any significant issues which arose
• How you envisage addressing the issues covered
in the exercise?
• What additional support you may need
QA Focus – Supporting JISC's Digital Library Programmes
27
Comments On Exercises
Your project deliverables may not be interoperable
because:
• You were unclear of the purpose of the metadata / software
or there were differing views across project partners
• You chose inappropriate standards for your purpose or your
level or expertise, funding, timescales, etc.
• Your modelling of your metadata (or database) meant that it
was not interoperable with others
• You chose an inappropriate implementation architecture
• You failed to implement appropriate cataloguing, etc. rules
• You had cataloguing rules, but no processes for spotting
problems, deviations, ambiguities, etc.
• You haven't allocated sufficient resources to implement the
above
You will need to implement QA
procedures to address such issues
QA Focus – Supporting JISC's Digital Library Programmes
28
Thoughts On Timeliness of QA
Hype
Realism &
service deployment
Early
adopters
Maturity
Software
Digitisation
Web
Metadata
Web Services
Semantic Web
Despondency
QA implemented
QA Focus – Supporting JISC's Digital Library Programmes
29
Note that QA for
metadata is at an
early stage. We are
currently still
developing and
evaluating the
potential for
technologies such as
OAI, IMS, …
Next Steps
Next Steps – Metadata
QA for metadata is at early stages (little documented
work found).
Thoughts on QA for metadata:
Syntax
Cataloguing Rules
Interoperability
Fitness for purpose
Change control
Check syntax on embedded DC
Ensure projects have appropriate
cataloguing rules
Ensure metadata can interoperate
with third parties
Ensure metadata is fit for its purpose
Ensure architecture for managing
metadata can cope with change
control (cf. Exploit Case Study)
See <http://www.ukoln.ac.uk/qa-focus/documents/
briefings/#metadata> and <http://www.ukoln.ac.uk/qafocus/documents/case-studies/#metadata>
QA Focus – Supporting JISC's Digital Library Programmes
30
Next Steps
Next Steps – Software
QA for software development is mature, but there may
be a need to engage in culture change within the sector
Thoughts on QA for software:
Specification
Process for developing systems
spec, agreeing spec with
stakeholders, etc.
Methodology
Documentation of systems
development methodology
(cf UML case study)
Testing
Documentation of testing
methodology
Deployment
Awareness of deployment challenges
See for example <http://www.ukoln.ac.uk/qafocus/documents/briefings/#software>
QA Focus – Supporting JISC's Digital Library Programmes
31
Next Steps
Next Steps – Service Deployment
QA for service deployment is important to protect
investment in project funding (but there is a lack of
awareness in some cases).
Development Architecture Early public documentation of
development architecture
Service Architecture
Awareness of likely service
architecture and liaison with
service
Security
Awareness of security concerns
Licensing
Licensing issues
Scenarios
32
Software developed using cool open software environment – but
JISC Service has no knowledge of environment.
Resources digitised, but copyright clearance not documented.
Focus – Supporting
Digital Library
ServiceQArefuses
to takeJISC's
delivery
due Programmes
to liability concerns.
Approaches For Web
Examples of areas
which may be
considered for QA for
Web is illustrated.
We intend to provide
such advice on
solutions for metadata,
software an service
deployment once we
know the answers and
have obtained case
studies.
Web Toolkit
The Web toolkit seeks to address quality assurance
issues for the provision of a Web site.
1. Purpose Of Your Web Site
Have you identified the purpose of your Web site?
There can be a number of roles for your project Web
site: information about the project; for use by project
partners; access to project deliverables, etc. You
may wish to take different approaches depending on
the different roles (e.g. use of standards; access
conditions; preservation policy, etc.)
For further information see:
briefing-15,
2. Standards For Your Web Site
Have you chosen the standards to be used on your
Web site??
Use of open standards such as HTML and CSS are
important in order to maximise access to resources,
ensure resources function correctly and to facilitate
reuse of your Web site.
For further information see:
briefing-01, briefing-35, …
case-study-02, case-study-05, …
QA Focus – Supporting JISC's Digital Library Programmes
33
What Next? (1)
QA Focus materials:
• Available for you to use
• Feedback needed
• Book token will be provided for 2 people providing
best feedback (see note)
QA for metadata, software and service deployment
• Currently under development
• Your input today is valuable
• Why not contribute a case study?
QA Focus – Supporting JISC's Digital Library Programmes
34
What Next? (2)
Embedding QA In Your Work
• There is a need for QA to ensure interoperability
and protect JISC's investment
• QA Focus have recommended a self-assessment
approach, rather than external checkers
• This is reliant on projects implementing
appropriate QA procedures
• You will need to consider how to embed QA into
your project work
Please complete the form in which you'll list
your action plans
This is for your personal use.
QA Focus – Supporting JISC's Digital Library Programmes
35
What Next? (3)
Future Programmes
QA Focus is recommending that for future JISC
programmes:
• Programme calls require projects to indicate the
approaches they will take to QA
• Initial reports to JISC should describe in more detail
the projects:
 QA methodology  Criteria for standards
 Intended service deployment environment
Your current project can provide a valuable
opportunity to gain experiences in these areas
QA Focus – Supporting JISC's Digital Library Programmes
36
Conclusions
We have described:
• The approaches to QA which have been taken by
the QA Focus project
• The resources which have been developed for the
community by QA Focus
• A systematic methodology for QA
You have:
• Addressed the QA requirements for your project
• Started to employ the QA methodology
• Identified areas for you to address
• Identified areas in which you can contribute to the
community
Time for a drink!
QA Focus – Supporting JISC's Digital Library Programmes
37