Transcript Diapositivo 1

Introdução à Medicina
Did the quality of diagnostic test accuracy studies in
infectious diseases improve in the last twelve years ?
Systematic Review
April 26, 2006
Class 9
Tutor Professor: Dra. Cristina Santos
Introduction
Better diagnostic test
Faster and better diagnostic
Better treatment
Better prognostic
Improvement in patients’ quality of life
April 26, 2006
Introduction
STARD
• To improve the quality of reporting of diagnostic accuracy studies
the Standards for Reporting of Diagnostic Accuracy (STARD) was
published. (1)
• The STARD project group developed a checklist of 25 items and a
prototypical flow chart to represent the design of the study and the flow
of patients through the study. (2)
(1) Bossuyt PM et al, de Vet HC: Towards complete and accurate reporting of studies of diagnostic accuracy, the STARD initiative.
Clinical Chemistry 2003
(2) Bossuyt PM et al, de Vet HC: Towards complete and accurate reporting of studies of diagnostic accuracy, the STARD initiative.
Clinical Chemistry 2003
April 26, 2006
Introduction
Aim
•
Evaluate the quality of reporting of diagnostic studies in infectious
diseases in high impact factor journals between 1995 and 2006 using
the Standards for Reporting of Diagnostic Accuracy (STARD) initiative
parameters.
• Compare the quality of reporting of diagnostic studies in infectious
diseases before and after STARD  any improvement?
April 26, 2006
Methods
Query used on PubMed
•
Expressions related to the evaluation of diagnostic tests accuracy (1)
•
Expressions related to infectious diseases
•
13 journals: - 8 journals – table 1
- 5 journals – table 2
•
Exclusion of review articles and letters to the author
(1) Devillé, W.L. et al.: Conducting systematic reviews of diagnostic studies: didactic guidelines. BMC Medical Research Tethodology Jully
2002
April 26, 2006
Methods
Rank
April 26, 2006
Abbreviated Journal Title
(linked to journal information)
Impact
Factor
1
LANCET INFECT DIS
2
CLIN INFECT DIS
6.510
3
AIDS
5.835
4
EMERG INFECT DIS
5.308
5
ANTIVIR THER
5.286
6
J INFECT DIS
4.953
7
CURR OPIN INFECT DIS
4.258
8
INFECT IMMUN
3.933
9
J ANTIMICROB CHEMOTH
3.886
10
JAIDS-J ACQ IMM DEF
3.871
11
PEDIATR INFECT DIS J
3.047
12
CURR HIV RES
3.000
13
DIAGN MICR INFEC DIS
2.738
13
SEX TRANSM DIS
2.738
15
CLIN MICROBIOL INFEC
2.679
16
SEX TRANSM INFECT
2.668
17
J VIRAL HEPATITIS
2.550
18
AIDS RES HUM RETROV
2.531
19
INT J ANTIMICROB AG
2.428
20
INFECT CONT HOSP EP
2.413
1. Smidt et al. Quality of Reporting of Diagnostic Accuracy Studies. Radiology, 2005. 235:350
10.008
Methods
Articles review
•
Years selected: 1995 to 2006
•
Random selection of 20 articles / year
•
10 reviewers
•
Each abstract was read by 2 reviewers
•
Apply inclusion / exclusion criteria
•
Disagreement  3rd reviewer  consensus between 3 reviewers
April 26, 2006
Methods
Exclusion criteria
1 – Doesn’t refer to infectious diseases
2 – It’s not an evaluation of diagnostic tests
3 – Other language than English or Portuguese
4 – It’s not an article
Note: If the abstract isn’t available, the article should be included.
April 26, 2006
Methods
• Articles evaluation  STARD’s 25
items
• Each article is evaluated by 2
reviewers
• Article score = number of STARD
items
• Average score per year
April 26, 2006
Methods
• Agreement between reviewers:
•In abstract inclusion – PA and kappa
•In full text evaluation – ICC and Bland and Altman limits
• Check if the distribution of the items is normal
• One - WAY ANOVA test to compare the different biennium
• T-test to compare the quality of pre and post STARD studies
April 26, 2006
Results
Query
2128 studies found
April 26, 2006
Results
• 1995 – 95
Articles / year
• 1995 – 6
• 1996 – 92
• 1996 – 8
• 1997 – 114
• 1997 – 6
• 1998 – 104
• 1998 – 2
• 1999 – 121
• 1999 – 3
• 2000 – 96
• 2001 – 125
• 2002 – 132
• 2003 – 112
Random
selection of
20 studies
per year
Application
of exclusion
criteria
• 2000 – 7
• 2001 – 4
• 2002 – 4
• 2003 – 10
• 2004 – 161
• 2004 – 5
• 2005 – 143
• 2005 – 5
• 2006 – 138
• 2006 – 8
April 26, 2006
Results
Articles / year
• 1995 – 10
• 1996 – 8
• 1997 – 10
• 1998 – 3
New random
selection to
obtain 10
articles/year
• 1999 – 10
Application
of exclusion
criteria
• 2000 – 10
• 2001 – 10
• 2002 – 10
• 2003 – 10
• 2004 – 10
• 2005 – 10
• 2006 – 10
April 26, 2006
Total = 111 articles
Results
Agreement between the reviewers in abstract inclusion
PA = 0. 89
Kappa = 0.77
April 26, 2006
Results
Means of the STARD
score/year
April 26, 2006
Year of publication
Mean
1995
14,00
1996
13,75
1997
13,60
1998
15,67
1999
17,60
2000
17,40
2001
14,90
2002
14,00
2003
15,10
2004
18,00
2005
16,80
2006
14,70
Total
15,48
Results
Means of the STARD score/
biennium
Year of publication
April 26, 2006
Mean
95 /96
13,89
97 / 98
14,08
99 /00
17,50
01 / 02
14,45
03 / 04
16,55
05 / 06
15,75
Total
15,48
Results
Agreement between the reviewers in full text evaluation
ICC = 0,92
Bland and Altman plot
April 26, 2006
Results
Kolmogorov-Smirnov Test
p=0.383
Normal distribution
April 26, 2006
Results
One-way ANOVA test
p = 0.008
Differences in STARD score between years
ARE statistically significant
April 26, 2006
Results
Post Hoc Tests
Differences between 95/96 and 99/00 ARE
statistically significant
(p = 0, 027)
April 26, 2006
Results
Graph of the means of the
STARD score/ biennium
April 26, 2006
Results
Compare means before and after STARD
T - test (independent samples)
p = 0,072
April 26, 2006
Results
Presence of
STARD items
April 26, 2006
Discussion
Quality of the studies:
• medium
Mean = 15.48
Std. Deviation = 3.648
• improve between 95/96 and 99/00
• didn’t improve after 2000
• didn’t improve after STARD (2003)
April 26, 2006
Discussion
50% or more of the studies analysed didn’t refer:
• The number of the persons executing the tests
• if the readers were blind (masked)
• beginning and ending dates of patients’ recruitment
• time interval from the index tests to the reference
standard
• adverse events and indeterminate results and how they
were handled
April 26, 2006
Discussion
•
The items 13 and 24
13
Methods for calculating test reproducibility, if done
24
Measures of test reproducibility, if done
These STARD items should NOT be facultative!!
If the evaluations followed the STARD statement and
measured the reproducibility, they would probably be better.
April 26, 2006
The End
April 26, 2006