Transcript Slide 1

Searching for and Finding Evidence in
Augmentative and Alternative
Communication (AAC):
Navigating a Scattered Literature
Oliver Wendt, MS, Doctoral Candidate
Purdue University
American Speech-Language Hearing Association,
Division 12: Augmentative and Alternative
Communication (DAAC), 7th Annual Conference
Presentation based on
Schlosser, R. W., Wendt, O., Angermeier, K. L.,
& Shetty, M. (2005). Searching for evidence in
augmentative and alternative communication:
Navigating a scattered literature. Augmentative
and alternative communication, 21 (4), 233255.
What is Evidence-based Practice
(EBP)?
• “Evidence-based medicine is the conscientious,
explicit, and judicious use of current best evidence in
making decisions about the care of individual
patients” (Sackett et al., 1996).
• “…the integration of best and current research
evidence with clinical/educational expertise and
relevant stakeholder perspectives to facilitate
decisions for assessment and intervention that are
deemed effective and efficient for a given direct
stakeholder” (Schlosser & Raghavendra, 2003, p. 256)
The EBP Process
1. Developing a well-built question
2. Selecting evidence sources & executing the
search strategy
3. Examining and synthesizing the evidence
4. Applying the evidence
5. Evaluating the application of the evidence
6. Disseminating the findings
(Sackett et al., 1996 - Steps 1-5)
Is this what it is like to look for evidence in
AAC?
The Role of the Search
• AAC literature is scattered
– It takes knowledge and skills (K & S)
• Consequences of an inadequate search
– Ignore pertinent evidence
• (Erroneously) conclude that there is no evidence
available
– Over- or underestimate the support for a particular
intervention
– Misinterpret the applicability of the evidence
Potential Sources of Evidence
•
•
•
•
Textbooks
Journals
Newsletters
Databases with pre-filtered evidence
– "…an individual or group of individuals with expertise in a
particular substantive area has reviewed and presented the
methodologically strongest data in the field" (Melnyk and FineoutOverholt, 2002, p. 263, based on Guyatt and Rennie, 2002).
• Database of Abstracts of Reviews of Effects (DARE)
– http://www.york.ac.uk/inst/crd/welcome.html
• General Purpose Databases
– CINAHL, ERIC, LLBA, MEDLINE, PsycINFO
• Internet
• General Purpose Databases
CINAHL
Cumulative Index of Nursing and Allied
Health Literatures
ERIC
Educational Resources Information
Center http://www.eric.ed.gov/
Medline
MEDLINE
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi
LLBA
PsycInfo
Linguistics and Language Behavior
Abstracts
E-equivalent of Psychological Abstracts
Assumptions for EBP Searches
Characteristics
Goal
Type of search
Search strategy
Effort/Efficiency
Use of quality
filters
Use of content
keywords
Systematic Reviews
To retrieve all evidence and seek to
avoid missing studies outside oneÕs
regular purview
ÒExhaustive searchÓ
Multi-faceted
DonÕt leave a stone unturned
EBP
To retrieve the best and
most current evidence first
Typically no
ÒBest evidence searchÓ
Multi-faceted
Time-limited (high
efficiency preferred)
Yes
Yes
Yes
Assumptions Continued
• Locate best & most current evidence first
– Work down the hierarchy of sources
• Database of Reviews of Effects (DARE)
• Cochrane Database of Systematic Reviews
• Seek out reviews before individual research
studies
• Look for peer-reviewed evidence before nonpeer-reviewed evidence
About Quality Filters
Database
CINAHL
Quality filters
• Topical subheadings (SH):
–diagnosis, evaluation, review, systematicreview, therapy, therapeutic use, utilization
• Document type (DT):
–Clinical trials, research
ERIC
• Publication type (PT):
–Statistical data, journal article,
evaluative/feasibility, research/technical
• Keyword:
– case studies, meta-analysis, review, quasiexperimental design
Database
LLBA
Medline
Quality filters
–Journal articles only, Latest update
• Publication type (PT):
–Meta-analysis, review, academic review, tutorial
• Medical subject heading (MeSH):
–Clinical trials, research, research design,
guidelines, consensus development conference,
data collection, evaluation study, comparative
study
PsycINFO
• Form/Content type (CT):
–Clinical trial, double-blind design, empirical
study, experimental replication, literature review,
longitudinal study, meta-analysis, prospective
study, single-blind design, treatment outcome
study, treatment outcome
Example 1 - Comparative Efficacy
1. Begin with scenario:
–
A team of practitioners and family members, serving a 6year old child with developmental disabilities (i.e., severe
to profound mental retardation), have deemed it
appropriate to introduce manual signing. They are,
however, unsure what instructional strategy is most
effective and efficient to yield successful expressive use
and receptive learning
2. Consider your question:
–
“What instructional strategies are most effective and
efficient in yielding expressive signing and receptive
speech?”
Example 1 Continued
3. Extract general keywords/limiters (quality filters)
from question
–
Direct extractions
•
•
•
•
•
•
•
•
–
Manual signing, signing
Developmental disabilities
Instruction (treatment, intervention, therapy)
Effectiveness
Efficiency
Expressive use
Receptive speech
Young children
Extrapolations
•
•
•
Comparative (treatment) studies
Group studies
Single-subject experimental designs
Example 1 Continued
4. Determine and prioritize appropriate evidence
sources:
•
•
•
(a) DARE
(b) General-purpose databases (prioritize:
PsycINFO)
(c) Internet (if needed)
Studiesa
CINAHL
ER IC
Barrera (1980)
Barrett (1987)
X
Bennett (1986)
X
Berkow itz (1990)
X
MED -
PSYC-
LINE
INFO
Science
X
X
X
X
X
X
X
X
X
X
X
Belfiore (1993)
LLBA
Web of
X
Brady (1996)
X
X
X
Brady (1978)
X
X
X
X
Carr (1984)
X
X
X
X
Clarke (1986)
X
X
X
X
X
X
X
Clarke (1988)
Conaghan (1992)
X
X
X
Studiesa
CINAHL
ER IC
MED -
PSY C-
LINE
INFO
Dalrymple (1992)
X
X
Duker (1994)
X
X
Goodman (1993)
X
Hurlbut (1982)
Ia cono (1995)
X
X
X
Lim (1998)
X
X
X
X
X
X
X
X
X
X
X
X
(1984)
Kozleski (1991)
Science
X
X
Ia cono (1993)
Konstantareas
Web of
X
Ducker (1986)
Horn (1996)
LLBA
X
X
X
X
X
Studiesa
CINAHL
MED -
PSY C-
LINE
INFO
Science
Linton (1984)
X
X
McNaughton (1993)
X
Oliver (1983)
ER IC
Web of
X
X
Parsons (1993)
Reichle (2000)
LLBA
X
X
Remington (1983)
X
X
X
X
X
X
Remington (1993a)
X
X
Remington (1993b)
X
X
Rotholz (1989)
X
X
X
Schlosser (1995)
X
X
X
X
X
Schlosser (1998)
X
X
X
Studiesa
CINAHL
ER IC
MED -
PSY C-
LINE
INFO
LLBA
Web of
Science
Sigaf oos (1995)
Sigaf oos (1992)
X
Sisson (1984)
X
Soto (1993)
X
X
X
Sundberg (1990)
X
Wraikat (1991)
X
X
X
X
X
Wells (1981)
Wherry (1983)
X
X
X
Watters (1981)
Webster (1973)
X
X
Van A cker (1995)
Vaughn (1995)
X
X
X
X
X
X
X
X
X
X
X
X
X
– AAC-relevant journals and their inclusion in
various databases
CINAHL
1. Advances in Speech
Language Pathology
x
2. Augmentative and
Alternative
Communication
x
DARE
x
3. Australasian Journal
of Special Education
6. Analysis of Verbal
Behavior
LLBA
x
x
x
x
MEDLINE
PsycINFO
Web of
Science
x
x
4. American Journal on
Mental Retardation
5. American Journal of
Speech-Language
Pathology Practice
ERIC
x
x
x
x
x
x
x (from
2003)
x
x
x
x
CINAHL
DARE
ERIC
PsycINFO
Web of
Science
x
x
x
x
x
x
9. Assistive
Technology
x
x
10. Australia and New
Zealand Journal of
Developmental
Disabilities
11. Australian Journal
of Human
Communication
Disorders
12. Autism
x
7. Aphasiology
8. Applied
Psycholinguistics
x
x
LLBA
MEDLINE
x
x
x
CINAHL
DARE
ERIC
LLBA
13. Behaviour Change
14. Behavior
Modification
15. Behavioral
Residential
Treatment
16. British Journal of
Developmental
Disabilities
17. Communication
Disorders
Quarterly
18. Education and
Training in Mental
Retardation
19. Education and
Treatment of
Children
20. Exceptional
Children
x
x
MEDLINE
PsycINFO
Web of
Science
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
21. Exceptionality
x
x
x
22. Focus on Autism
and Other
Developmental
Disabilities
x
x
x
x
x
x
x
x
CINAHL
23. Journal of
Developmental and
Physical Disabilities
24. Journal of
Experimental Child
Psychology
25. International Journal of
Disability,
Development and
Education
26. International Journal of
Language and
Communication
Disorders
27. Journal of Applied
Behavior Analysis
28. Journal of Applied
Research in
Intellectual Disabilities
29. Journal of Autism and
Developmental
Disorders
30. Journal of Behavioral
Education
31. Journal of Childhood
Communication
Disorders
32. Journal of
Communication
Disorders
DARE
ERIC
LLBA
MEDLINE
x
Web of
Science
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
PsycINFO
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
CINAHL
DARE
ERIC
LLBA
MEDLINE
PsycINFO
Web of
Science
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
36. Journal of Special
Education
x
x
x
x
x
37. Journal of Special
Education
Technology
38. Journal of Speech,
Language, and
Hearing Research
39. Journal of the
Association for
Persons with Severe
Handicaps
40. Journal of Visual
Impairment and
Blindness
41. Language, Speech,
and Hearing
Services in Schools
42. Mental Handicap
Research
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
33. Journal of
Intellectual Disability
Research
34. Journal of
Psycholinguistic
Research
35. Journal of Mental
Deficiency Research
x
CINAHL
DARE
ERIC
43. Mental Retardation
44. Mental Retardation
and Learning
Disabilities Bulletin
x
45. Remedial and
Special Education
x
46. Research in
Developmental
Disabilities
47. Sign Language
Studies
48. Topics in Early
Childhood Special
Education
x
LLBA
MEDLINE
x
x
Web of
Science
x
x
x
x
x
PsycINFO
x
x
x
x
x
x
x
x
x
x
Example 1 Continued
5. Implement search for reviews - DARE
• Augmentative communication, alternative communication, assistive
technology, sign language, manual sign
6. Examine results (reviews) - DARE
augmentative communication/All fields OR alternative communication/All fields OR
assistive technology/All fields - 2 Hits (1-2)
(Record 1)
Promoting generalization and maintenance in augmentative and alternative communication:
a meta-analysis of 20 years of effectiveness research.
Schlosser R W, Lee D L. AAC: Augmentative and Alternative Communication 2000; 16(4):
208-226.
(Record 2)
Speech and language therapy to improve the communication skills of children with cerebral
palsy.
Pennington L, Goldbart J, Marshall J. Speech and language therapy to improve the
communication skills of children with cerebral palsy (Cochrane Review). In: The Cochrane
Library, Issue 4, 2004. Chichester, UK: John Wiley & Sons, Ltd
• sign language/All fields OR manual sign/All fields - No Hits
5. Implement search for reviews - PsycINFO
Evidence T able
Study
PsycINFO?
Table 1 Comparat ive Studies Involving
Unaided AAC Modes: Simultaneous
Communicat ion, Sign-alone Inst ruct ion,
and Oral Inst ruct ion
Barret t (1987)
McDonald (1977)
Remington (1983)
Sisson (1984)
Wells (1981)
Yes
No
Yes
Yes
Yes
Clarke (1986)
Conaghan (1992)
Ducker (1986)
Duker (1994)
Goodman (1993)
Linton (1994)
Remington (1993a)
Remington (1993b)
Bennet t (1986)
Dalrymple (1992)
Iacono (1986)
Wolery (1994)
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
Table 2. Comparat ive studies involving
unaided approaches: Determining the
necessary condit ions for simultaneous
communicat ion
Table 3. Comparison studies involving
unaided approaches: Other instruct ional
st rategies
Schlosser, R. W., & Sigafoos, J. (2006). Augmentative and alternative communication
interventions for persons with developmental disabilities: Narrative review of
comparative single-subject experimental studies. Research in Developmental
Disabilities, 27,1-29.
• How to get the other ones not in database?
– McDonald, L. (1977). A comparison of three methods of
word imitation training with Down’s Syndrome children
under six years of age. Unpublished master’s thesis.
University of Manitoba, Winnipeg. Canada.
• ---> AAC Thesis and Dissertations
(Lloyd, Koul, & Arvidson, 1993, 1994; Lloyd, Arvidson, &
Koul, 1996)
– Iacono, T., & Parsons, C. (1986). A comparison of
techniques in teaching signs to the intellectually disabled
using an alternating treatments design. Australian Journal
of Human Communication Disorders, 14, 23-34.
• ----> Handsearch
Prepare search - PsycINFO
Study
Keywords (subject s)
“compar
*”
Form/Content
type
Classificat ion
Barret t
(1987)
Communicat ion skills training,
moderate m ental retardation,
speech t herapy
Yes
Empirical study
Speech &
language therapy
Bennet t
(1986)
Manual Communication; Mental
Retardation; Mult iple
Disabilit ies; Special Educat ion
Studen ts; T eaching Methods
Severe Mental Retardat ion; Sign
Language; Vocabulary;
Language Art s Educa t ion;
Special Education Studen ts
Communicat ion Skills Training;
Phenylketonuria; Severe Mental
Retardation; S ign Language;
Verbal Communicat ion
Hearing disorders, m ental
retardation, posit ive
reinforcement, pract ice, sign
language, communicat ion skills
t raining, generalizat ion
(learning)
Yes
Empirical study
Special &
Remedial
Educa t ion
Alternat i Empirical study
ng
Special &
Remedial
Educa t ion
Yes
Empirical study
Speech &
Language
Therapy
Yes
Empirical study
Interpersonal &
Client -centered
humanist ic
therapy
Clarke
(1986)
Clarke
(1988)
Conaghan
(1992)
St udy
Keywords (subject s)
“compar*”
Form/Con tent
typ e
Classificat ion
Dalrymple
(19 92 )
Mo derat e Mental
Retardation; P ract ice;
Reinforcement ; Severe
Mental Retardation; S ign
Languag e; Non verbal
Learning
Behavior Mo dificat ion;
Downs Syn drome;
Inst itut ion alized Mentally
Retarded; Reinforcement;
S ign Languag e
Communicat ion Skills;
Profound Mental
Retardation; Severe Mental
Retardat ion; St imulus
Cont rol
Reinforcement ; Severe
Mental Retardation; S ign
Languag e; Special
Educa t ion St uden ts
Mental retardation,
ov ercorrect ion , po sit ive
reinforcement, pract ice,
sign lan guag e,
Comprehension , learning
disabilit ies, severe m ental
retardation, sign lan guag e;
special educa t ion studen ts;
communicat ion skills
t raining, select ive At tent ion
Yes
Empirical study
Rehabilitat ion
Yes
Empirical study
Hospit al &
Impat ient
Services
Different ial
effect iveness,
Vs.
Empirical study
Behavior
Therapy &
Behavior
Mo dificat ion
Yes
Empirical study
Special &
Remedial
Educa t ion
Alternat ing
Empirical study
Rehabilitat ion
Yes
Empirical study
Special &
Remedial
Educa t ion
Duck er
(19 86 )
Duker et al.
(19 94 )
Goodman
(19 93 )
Linton
(19 94 )
Remington
(19 93 a)
St udy
Remington
(1993b)
Keywords (subject s)
Communicat ion Skills
Training; Comprehension;
Severe Mental
Retardation; S ign
Language; Special
Educa t ion Studen ts;
Select ive At tent ion
Si sson
Communicat ion skills
(1984)
t raining, mild m ental
retardation, moderate
mental retardat ion, oral
communicat ion, sign
language
W atkins et
Mental retardation, verbal
al. (1990)
learning, videotape
inst ruct ion, oral
communicat ion, sign
language,
W ells (1981) Art iculat ion (Speech);
Communicat ion Skills
Training; Severe Mental
Retardation; S ign
Language; Speech
Therapy
W olery
Delayed Development;
(1993)
Naming; P reschool
St uden ts; Reading;
Teaching Methods
“compar*”
Form/Content
type
Classificat ion
Yes
Empirical study
Special &
Remedial
Educa t ion
Yes
Empirical study
Behavior therapy &
behavior
modification
Yes
Empirical study
Special &
Remedial
Educa t ion
Yes
N/A
Speech &
Language T herapy
Yes
Empirical study
Special &
Remedial
Educa t ion
• What can we glean from this backward process?
– Sign language
• Most widely and consistently used KW
• If not used, other alternatives are
– Manual communication
– Communication skills training
– ( ) Mental retardation
• is best to capture the population
– Quality filter:
• Empirical study (only one without it)
• “compar*” as a free-text word works for most
5. Implement search for reviews & 6. Examine
Results - PsycINFO
-12 references, but none focused on topic (autism, not treatment
focused) ---> refine search
-3 references, but none focused on topic
(autism, not treatment focused)
----> Need to look for individual studies
5. Implement search for studies - PsycINFO
Add limiters (quality filters)
6. Examine the results (studies) - PsycINFO
6. Examine results (& revise & search again)
revise search (and implement again)
6. Examine the results (studies) - PsycINFO
Example 1 Lessons
• Use the thesaurus to build your search
• When you found an appropriate reference, check it
out in terms of indexing
• Use synonyms or terms that describe similar concepts
• Be mindful - terminology changes, but the indexing
may not be retro-active
• Don’t forget the “TX” option
• Consider to use truncation
• Trust no one! Indexers are not perfect.
• “Pearl Growing” can be a beneficial EBP search
strategy (if you have a relevant article to start with)
Pearl Growing
• Pearl Growing involves the following process
– (1) Find a relevant article;
– (2) find the terms under which the article is
indexed in database-1;
– (3) find other relevant articles in database-1 by
using the index terms in a Building Block query;
– (4) repeat 2-3 in other databases;
– (5) repeat 1-4 for other relevant articles; and
– (6) end when articles retrieved provide diminishing
relevance.
An Illustration of Pearl Growing
• Question
– What strategies are most effective and efficient for
introducing manual signs to children with
developmental disabilities in terms of expressive
signing, and/or expressive natural speech, and/or
receptive speech?
• (1) Our Pearl
– Clarke, S., Remington, B., and Light, P., 1988, The
role of referential speech in sign learning by
mentally retarded children: A comparison of total
communication and sign-alone training. Journal of
Applied Behavior Analysis, 21, 419-426.
• (2) Under which terms is the pearl indexed in database 1 Medline?
– Medical Subject Heading terms (italic = most relevant):
• Child; Child, Preschool; Communication Methods, Total;
Comparative Study; Female; Humans; Imitative
Behavior; Male; Manual Communication; Mental
Retardation/rehabilitation; Rehabilitation; Research
Support, Non-U.S. Gov't; Sign Language; and Speech.
– the “pearl” was not indexed with a particular publication
type
• The MeSH term “comparative study” can serve to filter
out studies that involve only one treatment.
– Checked the MeSH database
http://www.nlm.nih.gov/mesh/MBrowser.html to identify
other relevant terms to capture this population
• found “developmental disabilities” and “autistic
disorder.”
– (3) find other relevant articles using terms in
Medline
• First goal is to identify reviews:
– [(communication methods, total OR sign language OR
manual communication) AND (mental retardation OR
developmental disabilities OR autistic disorder)] using
“child,” “human,” and sequentially “meta-analysis,”
“review,” & “practice guideline” as limiters.
» Using “review”- Yield: 9 entries - three were
applicable (Bondy & Frost, 1998; Howlin, 1988;
Wilken, 1996).
• Second goal is to identify individual studies:
– [(communication methods, total OR sign language OR
manual communication) AND comparative study AND
(mental retardation OR developmental disabilities OR
autistic disorder) AND speech)] using “child” and
“human” as limiters
» Yield: 3 additional relevant references (Brady &
Smouse 1978; Layton 1988; Wells 1981).
» PT use varied from case report to clinical trial to RCT
– Repeat search without “speech”
» Yield: 9 references - 4 were applicable (Barrera &
Sulzer-Azaroff 1983; Kahn 1981; Sisson & Barrett
1984; Wherry & Edwards, 1983)
» PT use varied from none to case report to clinical
trials to RCT
» This variation in quality filters along with the absence
of any additional keywords made it unnecessary to
conduct more MEDLINE searches.
– (4) repeat steps 2-3 in PsycINFO
• This pearl is indexed under the following Descriptors:
– communication skills training, phenylketonuria, sign
language, severe mental retardation, and verbal
communication
– Form/Content Type of “empirical study” as a quality filter
– This provided some valuable keywords (in italics) and
one quality filter to start with
– We supplemented other relevant keywords to better
describe the population of developmental disabilities
using the thesaurus
• First goal is to identify reviews:
– [(DE "Sign Language" or DE "Verbal Communication")
and (DE "Communication Skills Training") and (DE
"Mental Retardation" or DE "Developmental Disabilities"
or DE "Pervasive Developmental Disorders")] using
literature review and meta-analysis as quality filters.
» Yield: No hits
• Second goal is to identify individual studies:
– As above with limiters: preschool child, schoolage child, and adolescent
» Yield: 7 hits with 6 meeting our relevancy
criteria (Bonta & Watters 1983; Braam &
Poling 1983; Bucher 1983; Gaines et al.,
1988; Light et al., 1990; Remington et
al.,1990).
» All of these hits were indexed under the
form/content type of “empirical study”
(i.e., a quality filter), just like the “pearl.”
» An analysis of keywords revealed no viable
additional ones and so we abandoned the
search for more entries
Pearl Growing Summary
Features
Purpose
Information
Source Needed
Functions
Representativeness
Effort
Pearl Growing
1. To aid a current EBP search
2. One or a few relevant article/s –
the “pearl/s” (a very small
sample of the population of
relevant studies)
 Identifies relevant keywords
 Identifies relevant quality filters
 Suggests indexing problems
 May not get typical keywords and
quality filters (i.e., a “pearl” in terms
of content may not be one in terms of
keywords and quality filters)
Fairly time efficient
Example 2 - Speech Production
• 1. Start with scenario
• Sam is a 4-year old child who was recently diagnosed with autism. He
is unable to meet his daily communication needs in his preschool
through natural speech. He has recently learned to imitate words such as
“mama,” “dada,” “quack-quack,” and “bye-bye.” To date, Sam does not
use these words to communicate. His family and the staff at his
preschool anticipate many of his needs and consistently respond to his
prelinguistic communication behaviors such as touching objects or
leading people to objects he wants or activities that he would like to do.
Sam’s parents hope that he will eventually speak and be included in a
classroom with typically-developing children. Prior to Sam’s diagnosis,
his parents were not interested in exploring other forms of
communication because they had serious concerns about the impact on
Sam’s potential for developing speech. Their current goals for Sam are
that he improve his speech and communicate more effectively through
whatever means are appropriate and supportive of speech development.
• 2. Consider your question:
• His parents want to know, “Which AAC approach(es) best support
natural speech production?”
Example 2 Continued
3. Extract potential keywords/limiters from question
– Direct extractions
•
•
•
•
•
•
•
•
Autism
Presymbolic (Prelinguistic)
Instruction (treatment, intervention, therapy)
Augmentative communication, alternative communication
Effectiveness/efficiency
Preschool children
Settings: special education, toward inclusion
Speech production
– Extrapolations
•
•
•
•
•
Prognosis
Comparative (treatment) studies
Group studies
Single-subject experimental designs
Social validation
Example 2 Continued
4. Prioritize your search
– DARE
– (Conference Proceedings)
– General-purpose databases
5. Implement your search - DARE
Broaden your search
6. Examine Results
5. Implement search for reviews - PsycINFO
6. Examine the results (reviews) -PsycINFO
5. Implement search for reviews - ERIC
6. Examine the results (reviews) - ERIC
5. Implement search for reviews - Medline
– First go for meta-analyses
- Broaden to other reviews
5. Implement search for reviews - CINAHL
6. Examine the results (reviews) - CINAHL
5. Implement search for reviews -CINAHL
6. Examine the results (reviews) - CINAHL
5. Implement search for reviews - CINAHL
6.Examine the results (reviews) - CINAHL
Searching conference proceedings latest reviews
 Special issue on speech output:
• Yielded 1 relevant review
• Blischak, D. M., Dyson, A. T., & Lombardino, L. J. (2003). Use of
speech-generating devices: In support of natural speech. Augmentative
and Alternative Communication, 19, 29-35.
 ISAAC proceedings 2000, 2002, 2004:
 Yielded 1 systematic review
 Millar, D., Light, J., & Schlosser, R. (2000). The impact of AAC on
natural speech development: A meta-analysis. In Proceedings of the 9th
biennial conference of the International Society for Augmentative and
Alternative Communication (pp. 740-741). Washington, DC: ISAAC.
 ASHA Convention Abstracts 2001, 2002, 2003:
 Yielded 1 systematic review
 Correa, N., & Nye, C. (2001). Sign language and autism: A quantitative
synthesis of single-subject research. Poster presented at the Annual
Convention of the American Speech-Language Hearing Association.
ASHA Leader, 6(15), 189.
Let’s take stock of reviews
• 2 systematic reviews
– 1 on AAC, natural speech production & DD
– 1 on manual signing and autism
• ---> examine for individual studies
• 13 narrative reviews
– Mirenda (2003), Blischak et al. (2003), Goldstein
(2002), Mirenda (2001), Schlosser & Blischak
(2001), Koul et al. (2001), Sigafoos & Drasgow
(2001), Bondy (2001), Bondy (1998), Mirenda &
Schuler (1988), Hedbring (1985), Kiernan (1983),
Bonvilllian (1981)
• ---> examine for sections on autism & natural speech
production & individual studies
5. Implement search for studies - PsycINFO
Add limiters (quality filters)
Drop the peer-reviewed part
Zero in on speech production
6. Examine results (studies) - PsycINFO (and revise
strategy)
5. Implement search for studies again - PsycINFO
Add relevant free-text terms
6. Examine the results (studies) again - PsycINFO
5. Implement search for studies - ERIC
6. Examine the results (studies) - ERIC
5. Implement search for studies - CINAHL
6. Examine the results (studies) - CINAHL
Taking stock of studies
• From the two systematic reviews:
– Barrera, Lobato-Barrera, & Sulzer-Azaroff, 1980,
Benaroya, Wesley, Ogilvie, Klein, & Meaney, 1977; Bondy
& Frost, 1994; Bonta & Watters, 1983; Casey, 1978;
Fulwiler & Fouts, 1976; Kouri, 1988; Yoder & Layton,
1988
• From database searches:
– Ganz (2004), Tincani (2004), Sigafoos et al. (2003),
Charlop-Christy et al. (2002), Anderson (2002), Forsey
(1996), Buday (1995), Kouri (1988), Yoder & Layton
(1988), Shimizu (1988), Carr (1984), Ferrarese (1982),
Creekmore (1982), Konstanteras (1979)
EBP Search Example 3:
AAC and Aphasia
1. Begin with Scenario:
A clinician has a patient with aphasia on the
caseload and needs to know what AAC intervention
strategies have been used in similar cases and what
were the results.
2. Consider your question: What AAC intervention
strategies have been used successfully with patients
with aphasia?
– type of aphasia not further specified at the beginning of
the search to screen a broad range of studies that can be
narrowed down later
EBP Search Example 3:
AAC and Aphasia (cont.)
3. Extract general keywords/quality filters from
question
– Direct extractions
•
•
•
•
•
•
•
•
Aphasia
AAC
Augmentative and alternative communication
Communication
Aided communication
Unaided communication
Sign language
Intervention
EBP Search Example 3:
AAC and Aphasia (cont.)
4. Determine and prioritize appropriate evidence
sources:
•
Start with specialized databases for pre-filtered evidence:
–
•
Cochrane/DARE: 6 hits on “augmentative and alternative
communication”, none related to aphasia; 243 hits on
“aphasia” but AAC not included in any of them
General databases better choice, most appropriate for
aphasia related topics:
–
–
–
(a) PsycINFO
(b) MEDLINE
(c) Cumulative Index of Nursing and Allied Health Literatures
(CINAHL)
EBP Search Example 3:
AAC and Aphasia (cont.)
5.(a) Implement Search - PsycINFO:
•
Select appropriate keywords, chose major PsycINFO
headings:
–
–
–
To locate aphasia relevant entries:
1. Enter “aphasia” (already major heading); perform search
To locate AAC relevant entries:
2. Enter “augmentative communication”, map it to subject
headings, then also chose “sign language”; ignore “nonverbal
communication” and “communication systems” because no
additional results; perform search
3. Enter “symbolism” (major heading) to retrieve symbolrelated entries; perform search
EBP Search Example 3:
AAC and Aphasia (cont.)
5. (a) Implement Search - PsycINFO (cont.):
– Combine results from (2.) and (3.) to summarize
all AAC entries using OR operator
– Combine this summary with (1.) using AND
operator to obtain results relevant to AAC and
aphasia
 89 entries
EBP Search Example 3:
AAC and Aphasia (cont.)
5. (a) Implement Search - PsycINFO (cont.):
– Narrow down results using appropriate quality filters:
• Click on “Limit”, on next screen check general filters
“English Language” and “Peer Reviewed Journals” 
64 entries, peer-review quality
• Several filters available under “Form/Content Types”,
select:
“empirical study”  47 entries,
“quantitative study”  2 entries,
“literature review”  3 entries,
“clinical case report”  15 entries
“qualitative study”  1 entry
EBP Search Example 3:
AAC and Aphasia (cont.)
6. (a) Examine Results
• Final results for PsycINFO search:
 64 entries from peer-reviewed journals
 “empirical study”  47 entries,
“quantitative study”  2 entries,
“literature review”  3 entries,
“clinical case report”  15 entries
“qualitative study”  1 entry
 Need to review these results in terms of content
(relevance to individual case) and research design (quality
of evidence)
 Save search for later use, print out or e-mail search results
EBP Search Example 3:
AAC and Aphasia (cont.)
EBP Search Example:
AAC and Aphasia (cont.)
EBP Search Example 3:
AAC and Aphasia (cont.)
5. (b) Implement Search - MEDLINE
– Select appropriate keywords, consider Medical Subject Headings
(MeSH):
To locate aphasia relevant entries:
– 1. Enter “aphasia” (already a MeSH term); perform search
To locate AAC relevant entries:
– 2. Enter “graphic symbols”, map it to subject headings, then
chose MeSH terms “Communication Aids for Disabled”,
“Symbolism” and also check “graphic symbols” as keyword;
ignore “communication” because it reveals too many unrelated
entries; perform search
– 3. Enter “manual communication” (MeSH term), perform search
– 4. Enter “communication methods, total” (MeSH term), perform
search
EBP Search Example 3:
AAC and Aphasia (cont.)
5. (b) Implement Search - MEDLINE (cont.)
– Combine results from (2.), (3.), and (4.) to summarize all
AAC entries using OR operator
– Combine this summary with (1.) using AND operator to
obtain results relevant to AAC and aphasia
 69 entries
– Narrow down results using appropriate quality filters:
• First, try to locate pre-filtered evidence  checking “Limit
to: ‘Review articles’ and ‘Systematic Reviews’” reveals no
results, field not advanced to that level yet, need to descend
in evidence hierarchy
• Click on “Limit”, on next screen chose filters “English
Language” and under Publication Types chose “Case
Reports”
EBP Search Example:
AAC and Aphasia (cont.)
6. (b) Examine results:
18 “case reports” on AAC and aphasia, 13 are new entries
not listed in the original 62 from PsycINFO
“case reports” not clearly defined, there is still the need to
review these studies in terms of design quality (e.g.,
whether experimental single subject study or narrative
report)
review titles and abstracts (if needed) to identify those most
relevant to your case
Save search for later use, print out or e-mail search results
EBP Search Example 3:
AAC and Aphasia (cont.)
EBP Search Example:
AAC and Aphasia (cont.)
EBP Search Example 3:
AAC and Aphasia (cont.)
5. (c) Implement Search - CINAHL:
– Select appropriate keywords, chose major CINAHL
headings:
To locate aphasia relevant entries:
– 1. Enter “aphasia” (already major heading); perform search
To locate AAC relevant entries:
– 2. Enter “AAC”, map it to subject headings, then chose
“Alternative and Augmentative Communication” and
“Communication Aids for Disabled”; ignore “non-verbal
communication” because it does not lead to additional
results; perform search
– 3. Enter “sign language” (major heading) to retrieve
unaided communication modes; perform search
EBP Search Example 3:
AAC and Aphasia (cont.)
5. (c) Implement Search - CINAHL (cont.):
– Combine results from (2.) and (3.) to summarize all AAC
entries using OR operator
– Combine this summary with (1.) using AND operator to
obtain results relevant to AAC and aphasia
 33 entries
EBP Search Example 3:
AAC and Aphasia (cont.)
5. (c) Implement Search - CINAHL (cont.):
– Narrow down results using appropriate quality filters:
• Click on “Limit”, on next screen check general filters
“English” and “Research”  16 entries, research-based
• Under “Journal Subsets” select “Peer Reviewed Journals”
 13 entries, peer-review quality
• Look at quality filter “Publication types”, several options:
“clinical trial”, “systematic review” and “review” reveal no
hits, “case study” results in 4 entries, if these are applicable
distinguish between single-subject research designs and
narrative case reports
• Interesting, but not yet applicable (no results at this point):
Under “Special Interest Category” filter on “Evidence Based
Practice”
EBP Search Example 3:
AAC and Aphasia (cont.)
6. (c) Examine Results - CINAHL:
 16 research based entries, 13 of which did not appear in MEDLINE,
and 8 not in either MEDLINE or PsycINFO
 13 peer-reviewed
 4 “case studies”
 Again, need to review for quality of research design
 Review titles and abstracts (if needed) to identify those most relevant to
your case
 Save search for later use, print out or e-mail search results
EBP Search Example 3:
AAC and Aphasia (cont.)
EBP Search Example:
AAC and Aphasia (cont.)
– Locating in press-articles through
• Publisher websites (e.g., MetaPress, Science Direct,
etc.)
EBP Search Example 3:
AAC and Aphasia (cont.)
6. (a, b, c) Examine overall results from PsycINFO, MEDLINE,
and CINAHL:
• 83 unique entries covering range of aphasia subtypes,
interventions, research designs, and range in quality (!)
Need to review these results in terms of content (relevance
to individual case) and research design (quality of
evidence)
Start with information embedded in title, then read abstract, and if
the content is still unclear retrieve complete article to review in
further detail
Example 3 Lessons
• No single database covers all relevant literature on a given
topic
• Each database requires slightly different search strategy re:
combination of key words and phrases
• Even if correct keywords are used, irrelevant results may still
show up (accuracy of indexing),
• Databases vary in terms of available filters
– Not all filters are helpful
• Databases use different terms and sometimes ambiguous labels
to classify filtered search results
– Example: “clinical case reports” can cover qualitative case study or
single-subject research design
– Filtered results still need to be reviewed by hand in further detail re:
quality of the research design
EBP Search Example 4:
Efficacy of AAC in Autism
1. Begin with scenario:
A researcher aims to identify the research base for
AAC in autism, wants to investigate effectiveness
of AAC interventions for individuals with autism,
and decides to conduct quantitative synthesis of
intervention literature (meta-analysis)
EBP Search Example 4:
Efficacy of AAC in Autism
2. Consider your question(s):
-
What AAC interventions (e.g., aided, unaided, partner
training, etc.) are effective in terms of behavior change,
generalization, and maintenance in individuals with
autism?
- What AAC interventions are effective in yielding specific
outcomes (e.g., language acquisition, functional
communication training, participation, etc.)?
- Which AAC interventions are more effective than others
in general and in yielding specific outcomes?
- What are research gaps and
methodological gaps in the intervention literature?
Note: more comprehensive approach of systematic review
EBP Search Example 4:
Efficacy of AAC in Autism
3. Extract general keywords/quality filters from
question
–
Direct extractions
•
•
•
•
•
•
•
•
•
–
AAC
Augmentative and alternative communication
Communication
Aided communication
Unaided communication
Manual signing, signing
Aided Language Stimulation
Augmented Input
Autism
Extrapolations
•
•
Group studies
Single-subject experimental designs
EBP Search Example:
Efficacy of AAC in Autism
•
Note different approach of searches for systematic reviews (as
outlined above), particularly:
•
•
•
Multi-faceted, i.e., different searches and search strategies
complement one another
Targeted, but broad and very comprehensive because of need
to locate and finally evaluate all existing research studies out
there
What can be learned from systematic reviews to enhance
searches for EBP?
-
Evidence for AAC scattered widely, often scarce and difficult to
locate through traditional channels (e.g., databases)
Search strategies common for systematic reviews can be of
benefit when evidence is hard to find  improve EBP search
through multiple strategy approach
EBP Search Example 4:
Efficacy of AAC in Autism
4. Determine and prioritize appropriate evidence
sources:
Focus on computerized database searches covering
•
•
•
•
•
Cumulative Index to Nursing and Allied Health Literature
(CINAHL)
Dissertation Abstracts International
Educational Resources Information Center (ERIC)
Language and Linguistics Behavior Abstracts (LLBA)
PsychINFO
EBP Search Example 4:
Efficacy of AAC in Autism
5. Implement search
(a) Computerized databases with selected keywords as
outlined above
Plus additional search strategies (multi-faceted search):
(b) Hand Search - "AAC Theses and Dissertations”
•
•
•
published report "AAC Theses and Dissertations" (Lloyd,
Koul, & Arvidson, 1993, 1994) in Journal of AAC
includes abstracts and bibliographic information of 73 theses
and dissertations
facilitates the further retrieval of unpublished theses
EBP Search Example 4:
Efficacy of AAC in Autism
5. Implement search (cont.)
(c) Hand Search - Journals
• 39 professional journals, identified as possible publication
outlets for AAC interventions
• Covers journals that may not be indexed in databases
• Hand search does not rely on key word indices provided by
the journals; the accuracy of these indices always depends
(e.g., judgment of authors, indexers/editor)
• How to hand search: systematic search of table of contents
for relevant titles; if titles appear relevant, examine the
abstract for further relevance
EBP Search Example 4:
Efficacy of AAC in Autism
5. Implement search (cont.)
•
Selection criteria rather than quality filters; applied by
researcher when reviewing obtained literature:
• Intervention must pertain to AAC (according to ASHA
1989 definition; Facilitated Communication literature
excluded)
• Intervention target must include individuals with
autism spectrum disorders (ASD)
• Experiment employs single-subject or group design
• Experiment needs to be written as an article in a
refereed journal, documents available through ERIC,
or unpublished MS/doctoral theses
• Dated between 1976 and 2004
EBP Search Example 4:
Efficacy of AAC in Autism
5. Implement search (cont.)
•
Selection criteria rather than quality filters; applied by
researcher when reviewing obtained literature:
• Intervention must pertain to AAC (according to ASHA
1989 definition; Facilitated Communication literature
excluded)
• Intervention target must include individuals with
autism spectrum disorders (ASD)
• Experiment employs single-subject or group design
• Experiment needs to be written as an article in a
refereed journal, documents available through ERIC,
or unpublished MS/doctoral theses
• Dated between 1976 and 2004
EBP Search Example 4:
Efficacy of AAC in Autism
6. Examine Results
•
•
•
Literature search revealed a total of 270 publications related
to AAC & autism (excluding FC)
Empirical studies: 134
– Descriptive clinical case studies: 7
– group designs: 13
– longitudinal designs: 3 (2 descr./1 experim.)
– single-subject designs: 81
– design flawed, etc. : 30
95 studies met inclusion criteria re: research design quality
Example 4 Lessons
•
•
•
•
Searches for systematic reviews are different from EBP searches,
more comprehensive and multiple search strategies that
complement one another
Electronic databases, even if combined, do not reveal all available
evidence
– AAC relevant journals may not be indexed
– Database indeces may not be accurate
“Nontraditional ways” to search can reveal further useful
evidence, i.e., hand searches and ancestry searches, but
practitioners have to decide how feasible
– Hand search in example lead to 28 additional studies that did
not show up in databases
Research evidence can also be found in unpublished literature
such as PhD and MS theses (although not peer reviewed); 6 PhD
and 1 MS theses located in example
Example 4 Lessons (cont.)
•
In this particular case of AAC & autism:
–
–
Search reveals lots of anecdotal evidence, less data-based
Considerable amount of studies with flaws in research
design or insufficient (pre-experimental) design quality
 Researchers and clinicians need to be cautious in
interpreting and evaluating these findings, need to keep
EBP guidelines and evidence hierarchy in mind
Further Sources for EBP Searches
• ProQuest Digital Dissertations formerly known as
Dissertation Abstracts International (DAI)
• Linguistics and Language Behavior Abstracts
(LLBA)
• Web of Science
• Internet
–
–
–
–
General search engines
Specialized search engines
Meta-searches and meta-search engines
Evaluation of Internet information
ProQuest Digital Dissertations
(PQDD)
• PQDD is an online version of the print counter-part
Dissertation Abstracts International (DAI)
• Dissertations and theses may contain valuable
research findings
• Exclusion can introduce bias in the EBP process
• Most theses undergo internal peer-review process,
i.e., acceptance by committee
• PQDD holds more than 2 million entries from 1861
to the present (updated each semester)
ProQuest Digital Dissertations
(PQDD)
• More than 1.7 million full-text versions available in
paper or microform, can be ordered online
• More than 450,000 full-text versions available in
digital format for online download from Dissertation
Express website
– Entries from 1997 forward
• Only last 2 years can be accessed free of charge
• Broader searches require subscription
• Downloading full-text versions often requires
additional fee payment
Example 5: AAC & Early
Intervention
5. Implement search - CINAHL
6. Examine the results - CINAHL
Example 5 continued
5. Implement search - PsycINFO
6. Examine the results - PsycINFO
ProQuest Digital Dissertations
(PQDD)
• Brief search example: AAC & Early intervention
• Keywords:
– “augmentative communication”
 65 hits
– “alternative communication”  86 hits
– “augmentative and alternative communication”
 51 hits
– “AAC”  156 hits, but many non-relevant
– “sign language”  621 hits
– “sign language” AND NOT “deaf”  204 hits
• No filters available for research design quality
ProQuest Digital Dissertations
(PQDD)
Online
Demo
5. Implement search - expand to the internet
(ixquick  http://www.ixquick.com)
• 5. Implement search - expand to the inter-net
(Dogpile  http://www.dogpile.com)
• 5. Implement search - expand to the internet
(scirus  http://www.scirus.com)
ISI Web of Science
• Five high-quality databases containing
information from journals in all research areas
– Three relevant for AAC:
• Science Citation Index Expanded
• Social Sciences Citation Index
• Arts & Humanities Citation Index
– Contain references cited by authors of indexed
journals
– Option to use forward citation search where cited
references are used as search terms
– Can’t delimit citations to meet design criteria
ISI Web of Science (cont.)
– May also search by topic, author, source title, and
address
– Searching by author may be more cost-efficient
because of possible fees for forward citation search
– Requires either an individual or university library
subscription
ISI Web of Science (cont.)
Searching for evidence on the Internet
• Internet offers a vast array of information pertaining
to AAC
• Challenge when obtaining too many sites of interest
but different quality
• Lack of quality control is a major concern
• Points to consider when conducting evidence
searches
– Knowing which search engines will yield best
results
– Knowing how to assess the quality of the retrieved
information
General search engines
• Search engine:
– Software program built on top of large databases
containing web page files
– Matches the database content to specified search
terms
– Databases compiled through “robots”, ”spiders”, or
“crawlers (programs searching and indexing
webpages)
– Each search engine uses own set of criteria to decide
what to include in its database
– Different ways to organize search results for users,
e.g., link popularity vs. common themes
General search engines (cont.)
• Web directories:
– Instead of “spiders” human editors review and
index links
– Guidelines for web sites to be included in index
– Directory editor looks at quality of a site:
functionality, content and design
– Directory indexes
•  small number of, but higher quality links
– Example: Yahoo! (in the past), Open Directory
Project (http://dmoz.org)
General search engines (cont.)
• Hybrid search engines:
– Latest generation
– Combination of traditional search engine with a
directory
– Top ten search sites are hybrids
– Example: Google uses Open Directory Project to
supplement its automatically generated listings
(Shapiro & Lehoczky, 2004)
General search engines (cont.)
• Most popular search engines by percentage of home
and work users (as of June 2004)
– Google, http://www.google.com (41,6%)
– Yahoo, http://www.yahoo.com (31,5%)
– MSN, http://www.msn.com (27.4%)
– AOL Search, http://www.aolsearch.com (13.6%)
– Ask Jeeves, http://www.ask.com (7.0%)
– Overture, http://www.overture.com (5.1%)
(Source: Nielsen//Netratings, 2004)
– Others: Altavista (http://www.altavista.com); Hotbot
(http://www.hotbot.com); Lycos (http://www.lycos.com)
• Currently, Google and Yahoo leading the market
General search engines (cont.)
• Search engines differ significantly in terms of
power and user-friendliness of their interfaces
• Advanced search options offered:
– Boolean operators (AND, NEAR, OR)
– Truncation (e.g., communicat*)
– Search within results (only Google, Hotbot, and
Lycos)
General search engines (cont.)
• Internet directories (e.g., dmoz):
good for beginning searches and to distinguish
between different categories, but may not be up to
date and limited in scope
• Not every item appears in every search engine
• Even the most current search engines cover less than
half of all the content on the internet (e.g., Google
accesses only 15-16%)
- in fall of 2005 Yahoo reported to possess an index of
19 billion web documents; Google’s index at the time
was estimated at 8 billion documents (Sherman,
2005)
General search engines (cont.)
• No recent updates on index sizes by either Yahoo or
Google
• Google, Yahoo and other search engines only access
“surface web”  static web documents that are
accessible to search engines
• “Deep web” 50x bigger than “surface web”, consists
of web documents dynamically generated by database
servers: this information tends to be more recent,
more topic-focused, and more relevant, but these sites
are difficult to retrieve for search engine crawlers
(Sherman, 2005)
 Repeat search using different search engines!
Specialized search engines
• Consider for locating higher quality research
evidence on the internet
• Scirus (http://www.scirus.com)
–
–
–
–
–
Focuses on scientific content only
Searches the web and electronic journal sources
Locates more peer reviewed articles
Recognizes formats such as PDF and Postscript
Most beneficial to locate research data
Specialized search engines (cont.)
• Google Scholar (http://www.scholar.google.com)
– Peer-reviewed papers, theses, books, preprints,
abstracts and technical reports from all broad areas
of research are available
– Results are ordered by how relevant they are to the
query
– Automatically analyzes and extracts references and
presents them as separate results, even if the
documents they refer to are not online
Meta-searches and meta-search
engines
• Simultaneously explore the databases of multiple sets
of individual search engines
• Summarize results from a broad range of different
search engines
• Search is more comprehensive without loosing the
general overview perspective
• Meta-search engines
– Ixquick (http://www.ixquick.com); features include
coverage of 12 most familiar search engines and
directories, result rankings, and ratings for most appropriate
individual search engine
– Copernic (http://www.copernic.com)
– DogPile (http://www.dogpile.com)
– Metacrawler (http://www.meta-crawler.com)
Evaluation of information located on
the Internet
• A systematic evaluation of the obtained web
documents should be performed to determine the
quality and usefulness of the information or data
presented
• Evaluation of validity is necessary because there is a
lack of quality control on the internet
• Five criteria for evaluation
– Accuracy
– Authority
– Objectivity
– Currency
– Coverage
Accuracy
• Need for content to be valid and “without errors
of fact, interpretation, or judgment” (Eysenbach et al.,
2002)
• Information needs to be disclosed re:
– Who authored the web document
– Whether or not that person is qualified to write the
information provided
– If the provided content is reliable and error-free
– What kind of sources are cited
Authority
• Overall credibility of the author and publisher
of a web document
• To determine authority look at:
– Header or footer for an affiliation
– The Internet domain (e.g., .edu, .com, .ac.uk, .org,
and .net)
• Credentials of the author should be reviewed
Authority:
Review of Top Level Domains
Domain
Meaning
educational
.edu
.gov
.org
.com
.net
.mil
.ca
ac.uk
.fl.us
Example
aac.unl.edu
U.S. government
www.nih.gov
organization/association
www.ussaac.org
commercial (mostly)
www.prentrom.com
internet service provider
www.assistivetech.net
U.S. military
www.bethesda.med.navy.mil
Canada
www.apraxia.ca
UK (academic)
callcentre.education.ed.ac.uk
U.S., Florida State
aten.scps.k12.fl.us
Objectivity
• Depending on website’s purpose (advertising,
advocacy, opinion, scholarship, etc.) the
provided information may be biased
• To determine objectivity consider:
– Why the document was created
– For whom it was written
– What opinions, if any, are expressed by the author
• Disclosure of ownership and sponsorship are
essential
Currency
• Look at when the web document or its web site
was first produced and updated
• Check the number of dead links located on the
web page
– If links cannot be accessed, there is a high
likelihood that information on the web page from
which the link was accessed is no longer current
Coverage, comprehensiveness, and
accessibility of information
• Web documents containing more detail most
likely provide more information and increased
coverage of a topic
• Credibility increases with links to reputable
web sites
• Many web pages contain links to biased sites
and advertisements
• Accessibility of web pages is important.
Accessible sites provide a text equivalent for
all non-text elements to be read in screenreaders to users with visual impairments
Look out for
• Schlosser, R. W., Wendt, O., Angermeier, K. L., &
Shetty, M. (2005). Searching for evidence in
augmentative and alternative communication:
Navigating a scattered literature. Augmentative and
alternative communication, 21 (4), 233-255.
• Schlosser, R. W., Wendt, O., Bhavnani, S., & NailChiwetalu, B. (under review). Benefits of “adapted
pearl growing” for evidence-based practice and
implementing systematic reviews: A short report.
Manuscript submitted for publication.
References
• American Speech-Language-Hearing Association (ASHA) (2003). Evaluating Web sites.
Retrieved November 23, 2003 from the ASHA web site:
http://www.asha.org/sitehelp/websites.htm
• Beaven, O. (2002). Searching the literature. In J. V. Graig, & R. L. Smyth (Eds.), The
evidence-based practice manual for nurses (pp. 45-85). Edinburgh: Churchill Livingstone.
• Beck, S. (1997). The good, the bad, and the ugly: or, why it’s a good idea to evaluate web
sources. Retrieved March 22, 2004, from the New Mexico State University Library web
site: http://lib.nmsu.edu/instruction/evalcrit.html
• Blair-Lockney, E. (2003). AJSLP reaches major milestone – MEDLINE indexing. The
ASHA Leader, April 1, 2003, 3.
• Boynton, J., Glanville, J., McDaid, D., Lefebvre, C. (1998). Identifying systematic reviews
in MEDLINE: developing an objective approach to search strategy design. Journal of
Information Science, 24,137-157.
• Chamberlain, E. (2003). Bare bones 101: A basic tutorial on searching the web. Retrieved
March 10, 2004, from the University of South Carolina, Beaufort Library web site:
http://www.sc.edu/beaufort/library/bones.html
• Cook, D. J., Mulrow, C. D., & Haynes, R. B. (1997). Synthesis of best evidence for clinical
decisions. Annals of Internal Medicine, 126, 376-380.
• Cook, D. J., Sackett, D.L., Spitzer, W. O. (1995). Methodologic guidelines for systematic
reviews of randomized control trials in health care from the Potsdam Consultation on MetaAnalysis. Journal of Clinical Epidemiology, 48, 167-171
• Cooper, H. & Hedges, L. V. (1994). The handbook of research synthesis. New York: Russell
Sage Foundation.
• Corrall, C. J., Wyer, P. C., Zick, L. S. et al. (2002). How to find evidence when you
need it, part 1: databases, search programs, and strategies. Annals of Emergency
Medicine, 39, 302-306.
• De Vet, C. W. H., de Bie, R., van der Heijden, G., Verhagen, A., Sijpkes, P., &
Knipschild, P. (1997). Systematic reviews on the basis of methodological criteria.
Physiotherapy, 83, 284-289.
• Dickson, R. (1999). Systematic reviews. In S. Hamer & G. Collinson (Eds.),
Achieving evidence-based practice. A handbook for practitioners (pp. 41-60).
Edinburgh: Tindall.
• Eysenbach, G., Powell, J., Kuss, O., Sa, E.-R. (2002). Empirical studies assessing
the quality of health information for consumers on the world wide web: A
systematic review. Journal of the American Medical Association, 287, 2691-2700.
• Gallagher, P. E., Allen, T. Y., & Wyer, P. C. (2002a). How to find evidence when you
need it, part 2: A clinician’s guide to MEDLINE: The basics. Annals of Emergency
Medicine, 39, 436-440.
• Gallagher, P. E., Allen, T. Y., & Wyer, P. C. (2002b). How to find evidence when you
need it, Part 3: A clinician’s guide to MEDLINE: Tricks and special skills. Annals of
Emergency Medicine, 39, 547-551.
• Golper, L. A., Wertz, R. T., Frattali, C. M., Yorkston, K., Meyers, P., Katz, R.,
Beeson, P., Kennedy, M. R. T., Bayles, K., & Wambaugh, J. (2001). Evidence-based
practice guidelines for the management of communication disorders in
neurologically impaired individuals: Project introduction. Minneapolis, MN:
Academy of Neurologic Communication Disorders and Sciences.
• Guyatt, G., & Rennie, D. (2002). Users' guides to the medical literature. Essentials
of evidence-based clinical practice. Chicago, AMA Press.
• Hass, U., Andersson, A., Brodin, H., Persson, J. (1997). Assessment of computeraided assistive technology: analysis of outcomes and costs. Augmentative and
Alternative Communication, 13, 125-135.
• Haynes, R. B., Walker, C. J., McKibbon, A., et al. (1994). Performances of 27 MEDLINE
systems tested by searches with clinical questions. Journal of the American Medical
Informatics Association, 1, 285-295.
• Haynes, R. B., Wilcynski, N., McKibbon, A., et al. (1994). Developing optimal search
strategies for detecting clinically sound studies in MEDLINE. Journal of the American
Medical Informatics Association, 1, 447-458.
• Haynes, R. B., Wilcynski, N., McKibbon, A., et al. (1994). Developing optimal search
strategies for detecting clinically sound studies in MEDLINE. Journal of the American
Medical Informatics Association, 1, 447-458.
• Helewa, A., & Walker, J. M. (2000). Critical evaluation of research in physical
rehabilitation: Towards evidence-based practice. Philadelphia: W.B. Saunders Company.
• Humphris, D., Littlejohns, P., Victor, C., O’Halloran, P., & Peacock, J. (2000). Implementing
evidence-based practice: Factors that influence the use of research evidence by Occupational
Therapists. British Journal of Occupational Therapy, 63, 516-522.
• Jackson, G. B. (1980). Methods for integrative reviews. Review of Educational Research, 50,
438-460.
• Kapoun, J. (1998). Teaching undergrads Web evaluation: A guide for library instruction.
College & Research Libraries News, 59 (7), 522-523.
• Kunka, J. L. (2004). Research and the Internet. Retrieved March 10, 2004, from Purdue
University, Online Writing Lab web site:
http://owl.english.purdue.edu/workshops/pp/index.html
• Kuster, J. M. (2002, May). Web-based information resources for evidence-based practice in
speech-language pathology. Perspectives on Language Learning and Education, 9 (1), 6-14.
• Lloyd, L. L., Arvidson, H., & Koul, R. (1996). AAC Master’s and doctoral theses update.
Augmentative and Alternative Communication, 12, 200-208.
• Lloyd, L. L., Koul, R., & Arvidson, H. (1993). AAC Theses and Dissertations. Augmentative
and Alternative Communication, 9, 196-225.
• Lloyd, L. L., Koul, R., & Arvidson, H. (1994). Master's and doctoral theses update.
Augmentative and Alternative Communication, 10, 61-65.
• Lohr, K. N. (1994). Guidelines for clinical practice: Applications for primary care.
International Journal for Quality in Health Care, 6, 17-25.
• Lou, J. Q. (2002). Searching for the evidence. In M. Law (Ed.), Evidence-based
rehabilitation (pp. 71- 94). Thorofare, NJ: Slack Incorporated.
• McAuley, L., Pham, B., Tugwell, P., & Moher, D. (2000). Does the inclusion of grey
literature influence estimates of intervention effectiveness reported in meta-analyses? The
Lancet, 356, 1228-1231.
• Melnyk, B. M., & Fineout-Overholt, E. (2002). Key steps in implementing evidence-based
practice: Asking compelling, searchable questions and searching for the best evidence.
Pediatric Nursing, 22, 262-266.
• Millar, D., Light, J. C., & Schlosser, R. W. (2004). The impact of augmentative and
alternative communication on speech development: A best evidence research review.
Manuscript submitted for publication.
• Morrisey, L. J., & DeBourgh, G. A. (2001). Finding evidence: Refining literature searching
skills for the advanced practice nurse. AACN Clinical Issues, 12, 560-577.
• NHS Centre for Reviews and Dissemination (CRD) (2001, March). Undertaking systematic
reviews of research on effectiveness (CRD Report No. 4, 2nd ed.). Retrieved June 18, 2003,
from the University of York, Centre for Reviews and Dissemination web site
http://www.york.ac.uk/inst/crd/report4.htm
• NHS Centre for Reviews and Dissemination (2003). DARE guide to searching. Version
2003:1. York, UK: Author.
• Nielsen//NetRatings (2004). Top 15 Search Destinations. Home and Work Users, June
2004. Retrieved November 1, 2004, from http://www.nielsen-netratings.com/news.jsp
• Raghavendra, P. (2000). Evidence-Based Practice: Where are we in AAC? Proceedings of
the 9th Biennial Conference of the International Society for Augmentative and Alternative
Communication (pp. 742-743). Washington DC.: International Society for Augmentative
and Alternative Communication.
• Sackett, D. L., Haynes, R. B., Tugwell, P. (1985). Clinical epidemiology. A basic science
for clinical medicine. Boston: Little, Brown & Co.
• Scherer, S., & Smith, M. B. (2002). Teaching evidence-based practice in academic and
clinical settings. Cardiopulmonary Physical Therapy, 13, 23.
• Schlosser, R. W. (2003). The efficacy of augmentative and alternative communication:
Toward evidence-based practice. San Diego, CA: Academic Press.
• Schlosser, R. W., & Raghavendra, P. (2004). Evidence-based practice in augmentative and
alternative communication. Augmentative and Alternative Communication, 20, 1-21.
• Schlosser, R. W., & Sigafoos, J. (in press). Augmentative and alternative communication
interventions for persons with developmental disabilities: Narrative review of
comparative single-subject experimental studies. Research in Developmental Disabilities.
• Shapiro, Y., & Lehoczky, E. (2004). Search engines. Retrieved November 1, 2004, from
http://www.searchengines.com/search_engines_101.html
• Sherman, C. (2005). Search Day. Electronic newsletter. Retrieved January 24, 2006, from
http://searchenginewatch.com/searchday/
• White, H. D. (1994). Scientific communication and literature retrieval. In H. Cooper, & L.
V. Hedges (Eds.), The handbook of research synthesis (pp. 42–55). New York: Russell
Sage Foundation.
For further information contact
• Oliver Wendt:
– E-mail: [email protected]
• Ralf W. Schlosser:
– E-mail: [email protected]
• Handout available on
Purdue AAC website:
http://www.edst.purdue.edu/aac
(click on “Recent Presentations”)