Transcript TITOLO TITOLO TITOLO TITOLO TITOLO TITOLO

Development of a harmonized protocol for
hippocampal tracing
An EADC-ADNI joint effort
4th Meeting, Paris, July 20, 2011
Principal Investigators: Giovanni B Frisoni & Clifford R Jack
Project Coordinator: Marina Boccardi
Housekeeping Information
Webinar attendees will be muted to limit surround
noise
Please write your questions in the “Questions”
window. Your question will be forwarded to the
speakers.
THE WORKGROUP
EADC
F Barkhof/P Scheltens Amsterdam
B Dubois/S Leherici Paris
N Fox/ J Barnes
London
GB Frisoni
Brescia
H Hampel/J Pantel
Frankfurt
C Hock
Zurich
A Simmons
London
H Soininen
Kuopio
S Teipel
Rostock
LO Wahlund
Stockholm
OTHERS
R Camicioli/NV Malykhin Edmonton, AL
J O’Brien
Newcastle, UK
J Pruessner
Montreal, QC
C Watson
Detroit, MI
ADNI
CR Jack Jr
Rochester, MN
M Albert
Baltimore, MD
G Barzokis
UCLA, CA
D Bennet
Chicago, IL
J Csernansky NorthW U, WA
C DeCarli
UC Davis, CA
M De Leon
New York, NY
L DeToledo-Morrell Chicago, IL
J Kaye
Portland, OR
R Killiany
Boston USM, MA
PM Thompson LoNI, UCLA, CA
M Weiner/S Mueller UCSF/VAMC
THE WORKGROUP
Advisory board
Clinical Advisor
PJ Visser
Maastricht
EADC P.I.s
B Winblad
Stockholm
Lutz Froelich, Mannheim
Dissemination/Education
ADNI P.I.
M Weiner
Statistical Board
UCSF, US
G Waldemar
Copenhagen
S Duchesne Québec City
L Collins
Montreal
P Pasqualetti Rome
Population-based studies
P Sachdev/JJ Maller
T Den Heijer
L Launer
W Jagust
Mirjam I. Geerlings
PATH Through Life Study
Rotterdam Scan Study
Honolulu Asia Aging Study
SALSA Study
SMART MEDEA Study
DISCLOSURES
Funded by Alzheimer’s Association grant n. 174022
Funded in part thanks to an unrestricted grant by
Lilly and Wyeth/Pfizer
Bartzokis Research support – Janssen; Consultant Lilly, Pfizer, Novartis; Csernansky Consultant Eli Lilly;
Duchesne Research support - Agfa HealthCare Inc.;
funding as a partner in the project; Frisoni Consultant
Wyeth; Jack Contract – Pfizer; consulting – Élan;
Lehericy Consultant - EISAI, Janssen-Cilag
Preliminary Phase
Areas explicitly
included
Areas explicitly
excluded
Andreas-Retzius
CA regions, dentate
gyrus (ARG), the part
gyrus, subiculum,
of the FG that is
alveus, fimbria, part of
adjacent to ARG, crus
the fasciolar gyrus (FG)
of fornix
Most anterior slice
Most posterior slice
slice w here one of the
follow ing is visible: alveus,
temporal horn of lateral
ventricle (uncal recess) or
amygdala
slice w here an ovoid mass of
gray matter started to appear
inferomedially to the trigone of the
lateral ventricle
- Survey of literature
- Certified extraction of landmarks
(Boccardi et al., J Alzheimers Dis, in press)
Inferior border
Medial border
Superior border
HEAD
uncal cleft (if visible)
CSF of ambient cistern
temporal horn of lateral ventricle
(uncal recess) and alveus
BODY
temporal horn of
lateral ventricle
(uncal recess)
White matter of the
parahippocampal
gyrus
CSF of ambient cistern
superior excess of the
quadrigeminal cistern
TAIL
BOUNDARIES
Lateral border
temporal horn of
lateral ventricle
(uncal recess)
Discrimination of
HT from FG and
crus of fornix
using arbitrary
borders
adjacent w hite matter
atrium of lateral ventricle
Discrimination of HT from ARG
using arbitrary borders
Preliminary Phase
Preliminary Phase
Areas explicitly
included
Areas explicitly
excluded
Andreas-Retzius
CA regions, dentate
gyrus (ARG), the part
gyrus, subiculum,
of the FG that is
alveus, fimbria, part of
adjacent to ARG, crus
the fasciolar gyrus (FG)
of fornix
Most anterior slice
Most posterior slice
slice w here one of the
follow ing is visible: alveus,
temporal horn of lateral
ventricle (uncal recess) or
amygdala
slice w here an ovoid mass of
gray matter started to appear
inferomedially to the trigone of the
lateral ventricle
- Survey of literature
- Certified extraction of landmarks
(Boccardi et al., J Alzheimers Dis, in press)
Inferior border
Medial border
Superior border
HEAD
uncal cleft (if visible)
CSF of ambient cistern
temporal horn of lateral ventricle
(uncal recess) and alveus
BODY
temporal horn of
lateral ventricle
(uncal recess)
White matter of the
parahippocampal
gyrus
CSF of ambient cistern
superior excess of the
quadrigeminal cistern
TAIL
BOUNDARIES
Lateral border
temporal horn of
lateral ventricle
(uncal recess)
Discrimination of
HT from FG and
crus of fornix
using arbitrary
borders
adjacent w hite matter
atrium of lateral ventricle
Discrimination of HT from ARG
using arbitrary borders
- Operationalization of differences
(Segmentation Units, SUs)
- Quantification of relevant features of SUs
(reliability, impact on hippo volume,
informative value for AD-related atrophy)
- Quantitative data used for Delphi Questionnaires
Delphi panel → harmonized prot
VARIABILITY
EVALUATION
GOLD STANDARD
5 expert tracers
1 tracer
20 naive tracers
Local Protocol:
Experimental set (1.5T ADNI):
2 x each of the 5 Scheltens’s
atrophy score x 2 sides
(SAME on 3T ADNI scans)
(total for each rater: 40 hippos)
VALIDATION vs
PATHOLOGY
Training
(tracing 20 hippos on 1.5T ADNI
scans with each SU)
Local Protocol:
1.5T 3D T1-weighted scans
from (Bobinski et al., 2000)
pathologically verified set
(total for rater: 30 hippos)
Benchmark Harmonized hippos:
Qualification
Harmonized Protocol:
Experimental set (1.5T ADNI):
2 x each of the 5 Scheltens’s
atrophy score x 2 sides
(SAME on 3T ADNI scans)
(total for each rater: 40 hippos)
RM-ANOVA: test
of rater and rater by
center terms
1.5T ADNI scans
2 x each of the 5
Scheltens’s atrophy score x 2 sides
(SAME on 3T ADNI scans)
(total for each rater: 40 hippos)
Qualification
Harmonized Protocol:
global and local
95% confidence
intervals
1.5T 3D T1-weighted scans
from (Bobinski et al., 2000)
pathologically verified set
(total for rater: 30 hippos)
Harmonized Protocol:
The best 5
naive tracers
1.5T ADNI scans
2 sides x 5 Scheltens’s atrophy scores x
3 time points (0-12°month-24°month)
x 3 scanners + retracing for timepoint 1
(SAME on 3T ADNI scans)
(total for each rater: 240 hippos – including
40 hippos already traced)
RM-ANOVA: test
of protocol main
effect
RM-ANOVA: test of
main effects side, traceretrace, atrophy, time,
scanner, rater
Project Steps Completed so far
- 2 Delphi rounds
- Tracing with local protocol (7/20)
- Master tracers practice (4/5)
- Pilot platform for tracers qualification
(will be reported in the interim report for September)
Delphi Panel
Murphy et al.,
Health
Technology
Assessment,
1998:2(3)
Delphi Panelists
PI
Barkhoff/Scheltens
George Bartzokis
Richard Camicioli
John Cserrnansky
Charles DeCarli
Leyla DeToledo-Morrell
Nick Fox
Mirjam Geerlings
Clifford Jack
Ronald Killiany
John O’Brien
Jens Pruessner
Hilkka Soininen
Paul Thompson
Craig Watson
Christoph Hock
Delphi Panelist
Wouter Henneman
George Bartzokis
Richard Camicioli / Nikolai Malikhin
Lei Wang
Charles DeCarli
Leyla DeToledo-Morrell
Josephine Barnes
Lotte Gerritsen
Clifford Jack / Greg Preboske
Ronald Killiany
Michael Firbank
Jens Pruessner
Hilkka Soininen / Maija Pihlajamäki
Liana Apostolova
Craig Watson
Henrike Wolf
Delphi method
First round:
Questions: Choose based on experience and helped by the
quantitative data; Motivate answers.
Space for free comments/criticism/suggestions.
Second round:
Feedback about answers and reasons;
Level of agreement about reasons converging towards
most frequent answers.
We answer previous comments / criticism / suggestions.
Space for new free comments/criticism/suggestions.
Delphi Panel – Landmarks
Statistics by Patrizio Pasqualetti, Rome
Delphi Panel – I Round
Delphi I Round – Alveus/Fimbria Inclusion
P=0.21
31%
69%
0 = Exclude Alveus/Fimbria
Frequency
1 = Include Alveus/Fimbria
P = exact p at Binomial Test (2x2)
Delphi II Round – Alveus/Fimbria Inclusion
1 = Minimum agreement
TO
9 = Maximum agreement
Frequency
19%
P1 < 0.0005
P2 =0.021
Median = 9
81%
P1 = Fisher’s exact test (2x9)
P2 = exact p at Binomial Test (<5 vs >5) (2x2)
Delphi I Round – Which criterion for subiculum
Delphi I Round – Subiculum
Frequency
0 = No subiculum
P=0.004
1 = Oblique line criterion
0% 6% 38% 56%
2 = Horizontal line criterion
3 = Morphology criterion
P at Fisher’s Exact Test (2x4)
Delphi II Round – Subiculum Morphology
Frequency
P1 = 0.056
P2 =0.057
Median = 7
19% (12%) 69%
1 = Minimum agreement
TO
9 = Maximum agreement
* Explicitly expresses no
strong objection for
morphology
*
P1 = Fisher’s exact test (2x9)
P2 = exact p at Binomial Test (<5 vs >5) (2x2)
Delphi Panel I Round
Delphi Panel I Round
Delphi I Round – Tail
Frequency
1 = No tail
P=0.006
2 = Crura criterion
6% 25% 69%
3 = Crura+End Tail criterion
P at Fisher’s Exact Test (2x3)
Delphi II Round – Tail End
TO
9 = Maximum agreement
Frequency
P1 = 0.146
P2 =0.035
Median = 8
19% (6%) 75%
1 = Minimum agreement
P1 = Fisher’s exact test (2x9)
P2 = exact p at Binomial Test (<5 vs >5) (2x2)
Delphi Panel – Preliminary Results
P=0.035
P for agreement
among panelists)
Covers 100% of
hippocampus proper
Captures 100% of
AD-related atrophy
P=0.057
Intra-rater (BS-BS): 0.993
Inter-rater (BS-BS): 0.985
Preliminary ICC (BS-Mayo-LONI): 0.95
P=0.021
Delphi – Preliminary Results
This model includes some tissue
which is not hippocampus proper:
- A/F is white matter
- Tail end may include some
vestigial tissue
- The medial boundary of the body
may include some entorhinal cortex
Delphi II Round – Harmonized Hippo
6%
P1 <0.0005
P2 =0.001
Median = 8
6%
88%
1 = Minimum agreement
TO
9 = Maximum agreement
Frequency
* Panelist who voted for different criterion for
every landmark, all reasons reported for
other panelists
*
§
§ Panelist who has problem to agree
because some boundaries not discussed
(we only questioned landmark
heterogeneities in the available literature)
P1 = Fisher’s exact test (2x9)
P2 = exact p at Binomial Test (<5 vs >5) (2x2)
Delphi Panel - Segmentation Modalities
Delphi I round – Disambituating Hippo/Amy in 3D
12%
P=0.004
88%
0 = (sometimes) not possible to
disambiguate with 3D visualization *
Frequency
1 = Possible to disambiguate with 3D
visualization
* Suggested editing in 3D
*
P = exact p at Binomial Test (2x2)
Delphi II Round – Disambiguating Hippo/Amy in 3D
1 = Minimum agreement
TO
9 = Maximum agreement
Frequency
0%
P1 < 0.0005
P2 <0.0005
Median = 8
0%
100%
P1 = Fisher’s exact test (2x9)
P2 = exact p at Binomial Test (<5 vs >5) (2x2)
Delphi I round – Disambiguating Vestigial in 3D
0 = not possible (or opportune)
Frequency
P=1
50%
1 = possible (or opportune)
50%
* Proposed again into two questions in
the II round
P = exact p at Binomial Test (2x2)
Delphi II Round – Possibility to exclude
vestigial tissue in 3D
44%
P=0.804
56%
0 = No
Frequency
1 = Yes
P = exact p at Binomial Test (2x2)
Delphi II Round – Opportunity to exclude
vestigial tissue (with 3D)
Frequency
19%
P1 = 0.797
P2 = 0.057
Median = 6.5
12%
69%
1 = Minimum agreement
TO
9 = Maximum agreement
P1 = Fisher’s exact test (2x9)
P2 = exact p at Binomial Test (<5 vs >5) (2x2)
Frequency
Delphi I Round – Internal CSF
P1 for exclusion
(somehow) P1 =0.004
1 = Include CSF in segmentation
P2 =0.397
3 = Exclude always
12% 44% 25% 19%
4 = Other **
2 = Exclude when connected *
* Criterion to be sure that it is CSF
** Should use other criterion to be
sure it is CSF, then exclude
P1 = exact p at Binomial Test (2x2: 1 vs 2-4)
P2 = Fisher’s exact test (2x4)
Delphi II Round – Exclude internal csf
when connected
1 = Minimum agreement
TO
9 = Maximum agreement
Frequency
31%
P1 = 0.304
P2 = 0.302
Median = 7
6%
63%
P1 = Fisher’s exact test (2x9)
P2 = exact p at Binomial Test (2x2)
Delphi I-II Round – Not visible structures
Trace only if you see, but train to look for tissue there?
Delphi I Round – Plane of tracing
1 = Long axis of the hippocampus
2 = AC-PC
Frequency
P=0.345
50%
31%
3 = Other *
19%
* Both are fine
P at Fisher’s Exact Test (2x3)
Delphi II Round – Hippo long axis
6%
P1 = 0.002
P2 = 0.012
Median = 8
31%
62%
*
Frequency
1 = Minimum agreement
TO
9 = Maximum agreement
* both AC-PC or long axis hippo are ok
P1 = Fisher’s exact test (2x9)
P2 = exact p at Binomial Test (2x2)
Discussion - Landmarks
Alveus/Fimbria: significant agreement for inclusion
(not Hippo proper, but > reliability, and some increase in
sensitivity).
Subiculum: non significant agreement for morphology;
Morphology > Horizontal, but further round needed;
Possible inclusion of non hippocampal tissue would apply
to both criteria.
Tail End: (almost significant) agreement for inclusion of
the whole tail;
Harmonized Hippo: significant agreement for inclusion of
all SUs and using the morphology criterion for subiculum.
OK
3° round
3° round
3° round
Discussion – Segmentation Modalities
Disambiguation hippo/amy in 3D: significant agreement;
Suggestion of “editing” in 3D will be considered in further
steps.
Disambiguation of vestigial tissue: no agreement on the
possibility or opportunity to exclude it from the whole
hippocampal structure.
CSF exclusion: significant agreement on exclusion. no
agreement on using the “connection” criterion to be sure.
Suggestions will be used to seek consensus on more
articulated ways to recognise for sure the CSF and thus
exclude it.
OK
3° round
OK +
3° round
Discussion – Segmentation Modalities
Segmentation of not visible structures:
agreement on inclusion of tissue only if visible
Plane of tracing:
significant agreement on long axis of the
hippocampus. Further rounds will better specify (left,
right, mean?)
OK
OK +
3° round
Discussion – Other Issues
- Landmarks not discussed: because no disagreement in
the literature.
-Partial volumes: interesting issue, proposed to panelists,
but only 2 answered. Will formulate an item based on these
2 in next trial.
Discussion – Software
- Multitracer (free; 3D simultaneous visualization; sub-voxel
segmentation and volume computation; tracing of different
labels on each slice)
- Possibility of editing in 3D may be helpful
-Please send your suggestions about a free software fulfilling
all (or more!) of these needs
- Zoom requirements are specific to this project due to
common image processing, but do not necessarily apply to
the future requirements for the harmonized protocol
Acknowledgements
Mike & Barbara Urbut, Stuart & Amy Savitz, Harriet K.
Burnstein, Chicago, IL
Maria Carrillo, Meredith McNeil
Alzheimer’s Association, Chicago IL
Martina Bocchetta, Daniele Tolomeo, Gabriele Corbetta
(Brescia, Italy)
Patrizio Pasqualetti
(Rome, Italy)
All partners!