Transcript Development and Validation of the Acute Otitis Media

Acute Otitis Media Severity of
Symptom Scale (AOM-SOS)
Development and Validation
Nader Shaikh, MD
Alejandro Hoberman, MD
Jack Paradise, MD
Howard Rockette, PhD*
General Academic Pediatrics
Children’s Hospital of Pittsburgh
*Graduate School of Public Health
Why measure symptoms?
Symptoms are important
 Bring children to medical attention
 AOM treated to improve symptoms
Need for symptom measurement in AOM trials
 Current research limited due to lack of validated
symptom measurement strategy
 Use of surrogate outcomes (bacteriologic
eradication) problematic
Potential uses
 Comparison of treatment modalities
 Natural history of symptoms in AOM
 Relationship between bacteriology and
symptoms
 Relationship between otoscopy and
symptoms
Conceptual model
Middle ear
infection
Physiologic
changes
Intended scope of
the AOM-SOS
Symptoms
Activity
limitation
Quality of life
Scale Development
 List of 28 symptoms
 Reduction using triangulation
Literature review
Expert opinion
Parent interview (n=33)
• Which of 28 symptoms present
•
How much symptom affects child
Validation study # 1
Objective
 Establish the reliability, validity and responsiveness of SOS
by comparing it with otoscopy
Methods
 Cohort of children 6-24 mo with/without AOM (n = 327)
 2003 and 2004 respiratory seasons
 Followed for one season (~3 visits/child, 949 visits)
 At each visit
- Examined by validated otoscopist
- SOS completed
Internal Reliability
Definition
 Are items in scale measuring the same concept?
Methods
 Correlation of items with each other
 Assessed by Cronbach’s α
 Cronbach’s α >0.7 indicates good reliability
Results
 α = 0.83
 Excellent inter-item correlation given short scale and
heterogeneous population
Content Validity
Definition
 Does each item measure what it is supposed to?
 Are items associated with AOM (or URI)?
Methods
 Examined association between items and
otoscopic diagnosis adjusting for URI
Content validity
AOM vs.
No AOM
AOM vs.
OME
OME vs.
Normal
Ear pain
p < .001
p < .001
p = .14
Tugging
p < .001
p < .001
p = .02
Irritability/crying
p < .001
p < .001
p =.049
Difficulty sleeping
p < .001
p < .001
p = .09
Eating less
p < .001
p < .004
p = .08
Less playful
p < .001
p = .01
p = .04
*All p values adjusted for URI
Construct Validity
Does the scale measure what it is supposed to?
AOM
N = 211
OME
N = 154
NORMAL
N = 584
Mean AOM-SOS score
4.4
1.6
0.9
Standard Deviation
4.6
2.8
1.9
Otoscopic diagnosis
AOM vs. Normal
AOM vs. OME
OME vs. Normal
p < .001
p < .001
p < .03
Responsiveness
Definition
 Can the scale detect change?
Methods
 Examined changes in score in children seen twice in
3-week period
 Responsiveness measured by standardized response
mean (SRM)
 SRM > 0.5  good responsiveness
Responsiveness –
Change in score within 3-week period
Unchanged
No AOM 
No AOM
N = 24
Change
P value
SRM
Changed
AOM 
AOM
N = 38
No AOM 
AOM
N = 23
AOM
No AOM
N = 108
Mean
Std
Mean
Std
Mean
Std
Mean
Std
-0.1
4.8
-1.8
6.4
+4.9
4.9
-3.4
4.9
.57
.08
<.001
<.001
0
0.28
1.00
0.69
Study # 2 – Further Validation
Needed to further evaluate:
 Day-to-day responsiveness
 Construct validity
Design
 Cohort study
 3 mo to 3 yr with AOM treated with antibiotics
 Otoscopic exam on days 1 and 5
 SOS administered days 1 through 5 (q 12 hrs)
 Reference measures also administered
 56 children enrolled
Construct validity
Correlation of SOS with reference measures on day 1
Reference Measures
AOM-SOS
p value
Pain Visual analog scale
r = .71
< .05
Post operative pain scale†
r = .83
< .05
Functional status IIR¥
r = .88
< .05
†
Chambers
¥
Stein
Responsiveness – Change in score
AOM-SOS
7
score
6
5
4
3
2
1
0
Day 1
Day 2
Day 3
Day 4
Day 5
Standardized response mean (day 1 to 5) = 0.76
Conclusions
 Developed scale for measurement of AOM
symptoms
 Demonstrated:
- Good measurement properties
- Correlates with otoscopy
- Correlates with validated measures of pain and
functional status
 Anticipated use in NIH funded RCT to evaluate the
efficacy of antibiotics versus placebo in young
children with AOM
Acknowledgements
Consultants
 Gordon Guyatt, MD – McMaster U.
 Galen Switzer, PhD – U. of Pittsburgh
Study Team
 Diana Kearney, Study Coordinator
 Kathleen Colborn, Data Manager
 Lisa Zoffel, CRNP
 Stephanie Konieczka, RN