Transcript Grey Literature

Meet the PRESS,
and other secrets
of systematic
searchers
CADTH SYMPOSIUM BREAKFAST SESSION
TUESDAY, APRIL 12, 2016
Meet the PRESS, and other secrets
of systematic searchers: an update
on literature search essentials for
finding the evidence
Amanda Hodgson, CADTH
Dr. Jessie McGowan, University of Ottawa
Melissa Severn, CADTH
Danielle Rabb, CADTH
1
Disclosure
• Funded by federal, provincial, and territorial ministries of
health.
• Application fees for three programs:
• CADTH Common Drug Review (CDR)
• CADTH pan-Canadian Oncology Drug Review (pCODR)
• CADTH Scientific Advice
• The presenters declare that there are no conflicts of
interest.
2
Agenda
• Introduction (Amanda Hodgson)
• PRESS Update (Dr. McGowan)
• Filters Update (Melissa Severn)
• Grey Literature Update (Danielle Rabb)
3
Literature search standards resources
https://www.cadth.ca/resources/finding-evidence
4
Literature search standards resources
Cochrane Handbook – Chapter 6: searching for studies (2011)
http://handbook.cochrane.org/
Institute of Medicine. Finding what works for healthcare:
standards for systematic reviews. Section 3. (2011)
http://www.nationalacademies.org/hmd/Reports/2011/FindingWhat-Works-in-Health-Care-Standards-for-SystematicReviews.aspx
5
Literature search standards resources
Summarized Research in Information Retrieval for HTA
SuRE Info
http://www.htai.org/vortal/?q=sure-info
6
Literature search standards resources
Listservs of interest:
Canmedlib (CHLA)
Expertsearch (MLA)
Cochrane (Information Retrieval Methods group)
HTAi (Information Resources Group)
7
Literature search standards Groups of Interest
Health Technology Assessment International Information
Resources Group (HTAi - IRG)
http://www.htai.org/index.php?id=555
Cochrane Information Retrieval Methods Group (IRMG)
http://methods.cochrane.org/irmg/
Canadian Health Libraries Association (CHLA)
http://www.chla-absc.ca/
8
What is PRESS?
Peer Review of Electronic Search Strategies
The PRESS 2015 Guideline is an includes two documents:
• PRESS Guideline Explanation & Elaboration (E&E)
• PRESS Guideline Statement
Report is found at:
https://www.cadth.ca/sites/default/files/pdfCP0015_PRESS_Upd
ate_Report_2016.pdf
Why do we need PRESS?
• In preparing systematic review (SR) and health technology
assessment (HTA) reports, researchers aim to provide
comprehensive, unbiased, and high-quality distillations of
relevant evidence.
• One step in assuring the integrity of the searching process
is the peer review of the work of the librarian or information
specialist who is seeking the evidence for the researchers.
Searching and PRESS
• PRESS focuses on the quality of the Boolean search and
is only one aspect of a comprehensive search for SRs and
HTAs.
• Other important aspects include the search plan, search
validation, and search reporting.
Searching and PRESS
• The Boolean search is typically the core element in the
search plan, identifies the majority of relevant evidence and
is the most productive method of identifying relevant
studies.
• Peer review of the search strategy is one quality assurance
step and provides a subjective validation.
• Other validation approaches include reaching data
saturation, testing the search of the main database
against eligible studies and testing using a related article
search and simple search protocol.
Research components
Systematic Review
• The SR search focused on research and reports on
electronic search strategies within a health science context.
• 6,608 records were identified; 39
Survey: a total of 174 surveys (117 were valid and 108
were complete).
Consensus Forum
• The objective was to create recommendations related to the
existing PRESS checklist items based on updated SR evidence
and survey findings.
• The outcome was that, of the seven existing PRESS elements,
six were retained and one (“Search strategy adaptations”) will
no longer be a PRESS element.
PRESS 2015 Implementation
Four components:
1. PRESS 2015 evidence-based checklist

2. PRESS 2015 recommendations for librarian practice
3. PRESS 2015 assessment form
4. PRESS 2015 implementation strategies
PRESS 2015 evidence-based checklist
PRESS 2015 evidence-based checklist
Search
Filters
What is a Search Filter?
• Pre-defined
• Pre-tested
• Pre-saved
• Also referred to as hedges,
clinical queries, optimal search strategies or
methodological search strategies
Why Use Search Filters?
•
•
•
•
•
Reduce number of results
Extract higher level evidence
Extract a specific concept
Remove off-topic results
Save time and money
Typical Search Strategy
INDICATION
TERMS
INTERVENTION
TERMS
SEARCH FILTER(S)
Types of Search Filters
–Specific methodology
• RCTs, systematic reviews
–Clinical queries
• diagnosis, prognosis, adverse events
–Population specific
• humans, pediatrics
• InterTASC Search Filter Resource
http://www.york.ac.uk/inst/crd/intertasc/
• CADTH Filters: Strings Attached
https://www.cadth.ca/resources/finding-evidence/strings-attachedcadths-database-search-filters
Methodology: RCTs
1. Randomized Controlled Trial.pt.
2. Randomized Controlled Trials as Topic/
3. Randomized Controlled Trial/
4. Randomization/
5. Double-Blind Method/
6. Single-Blind Method/
7. Placebos/
8. (random* or sham or placebo*).ti,ab
9. ((singl* or doubl*) adj (blind* or dumm* or
mask*)).ti,ab
10. ((tripl* or trebl*) adj (blind* or dumm* or
mask*)).ti,ab
11. or/1-11
Clinical Query: Prognosis
1.
2.
3.
4.
5.
6.
7.
25
Incidence/
Mortality/
Follow up studies/
Prognos*.ti,ab
Predict*.ti,ab
Course*.ti,ab
or/1-6
Population Specific: Human Filter
((Animals[MESH] OR Animal Experimentation[MESH] OR
"Models, Animal"[ MESH] OR Vertebrates[MESH]) NOT
(Humans[MESH] OR Human experimentation[MESH]))
26
Challenges to Filter Development
• Author related
• Titles
• Abstracts
• Indexer related
• Subject headings
• Publication type tags
• Database related
• Translation
• Testing
27
Gold Standard Testing
• A gold standard is a set of
relevant records against which
a filter’s performance can be
assessed
• Developed through a range of
techniques
• Gold standard sets can range in
size
Harbour J, et al. Reporting methodological search filter performance comparisons: a literature review.
Health Info Libr J. 2014;31(3):176-194
28
Sensitivity/Recall & Precision
• Sensitivity (or Recall): proportion of relevant records
retrieved by the filter
• Precision: proportion of relevant records in the total
retrieved records
29
http://www.creighton.edu/fileadmin/user/HSL/docs/ref/Searching_-_Recall_Precision.pdf
Performance of Search Filter in Ovid
MEDLINE - Example
Development and Testing of Search Filters to Identify Economic Evaluations in MEDLINE and EMBASE, Oct 2009, pg. 15
https://www.cadth.ca/sites/default/files/pdf/H0490_Search_Filters_for_Economic_Evaluations_mg_e.pdf
https://www.cadth.ca/resources/finding-evidence/strings-attached-cadths-database-search-filters
NEW!
31
Diving into the Grey: Searching for
grey literature
32
What is grey literature?
Grey Literature is a field in library and Information science
that deals with the production, distribution, and access to
multiple document types produced on all levels of
government, academics, business, and organization in
electronic and print formats not controlled by commercial
publishing i.e. where publishing is not the primary activity of
the producing body.
Source: GreyNet.org. The Grey Literature Network Service
http://www.greynet.org/
33
Trends in grey literature
• PISA Declaration on Policy Development for Grey Literature
Resources
http://www.greynet.org/greyforumseries/policydevelopment.
html
• Teaching grey literature
• Publishing of Big Data
• Metadata standards
• Access points and access repositories (pan-Canadian HTA
database)
34
Grey Literature - Groups of Interest
35
What is grey literature?
36
Clinical Practice Guidelines
37
Clinical Practice Guidelines
38
Clinical Practice Guidelines
39
Regulatory approvals
40
Regulatory approvals
41
Clinical Trials
42
Clinical Trials
43
Advisories and Warnings
44
Advisories and Warnings
45
Advisories and Warnings
46
What is grey literature?
Clinical Practice Guidelines
Position Statements
Regulatory approvals
Safety information & adverse event reports
Recalls & alerts
Clinical trial data (CSRs, abstracts, posters)
Supplemental information to published literature
47
Canadian Producers
48
International Producers
49
Benefits
Value of grey literature for systematic reviews
• Unpublished trials
• Conference abstracts
• Guidelines
Can reduce publication bias
Current
Usually free
Unique content
Information on non-mainstream topics
50
Challenges
Large volume of material
Inconsistent access points
Time-consuming to find
Not always available online
Quality of evidence varies
Bibliographic information may be missing
Inconsistent language, no subject indexing
51
Before you begin…
How much time do you have?
What type of information is required?
How are you incorporating your grey lit search?
How well do you know the sites you are searching?
Do you have a well defined PICO?
52
The CADTH Experience
Reasons for searching grey literature:
• To see other HTAs
• To find the best evidence available
• Adhere to standards for systematic reviews – the
exhaustive search for trials
• Help our researchers understand the topic area
53
The CADTH Experience
The Practical Stuff
• Grey literature searching is built into overall search time
• Customized checklists (Grey Matters)
• Results are sent separately from database results (word
document)
• We do not differentiate between commercial and noncommercial sources for our clients
54
Grey Matters
Grey Matters checklist:
CADTH tool for finding
grey literature information
55
Grey Matters
Categories:
Databases
National & international HTA organizations
Clinical practice guidelines
Drug class reviews
Health economics
Safety/advisories
Clinical trial listing
Regulatory approvals
Drug formularies
Statistics/prevalence
Other
56
Grey Matters Light
Grey Matters Light
checklist: Top 14 Resources
for Evidenced Based
Medicine Searching
Categories:
• HTA organizations
• Safety/advisory resources
• Clinical practice guidelines
57
Pan-Canadian HTA database
Pan-Canadian HTA database
58
Pan-Canadian HTA database
The HTA Database Canadian search interface lets you:
• Automatically limit search results to HTAs produced in
Canada or internationally
• Search in English or French
• Limit your search by province
• Easily organize your search by:
• Topic (using medical subject headings)
• Title
• Author
• Agency (CADTH, NICE, INESSS, etc.)
Pan-Canadian HTA database
The pan-Canadian Health Technology Collaborative invites
other HTA producers in Canada to contribute to the
database. For information about how to get involved, please
contact [email protected]
Connect With Us
61
@CADTH_ACMTS
linkedin.com/company/cadth
slideshare.net/CADTH-ACMTS
youtube.com/CADTHACMTS
cadth.ca/photoblog
[email protected]
Subscribe
Sign up at www.cadth.ca/subscribe to get updates sent
directly to your inbox.
CADTH E-Alerts
Calls for stakeholder feedback and patient group input, plus other time-sensitive
announcements.
New at CADTH
Monthly newsletter including a summary of new reports plus corporate news,
announcements of upcoming events, and more.
CADTH Symposium and Events
Updates about our flagship annual Symposium, workshops, webinars and other
events.
62
63