Justin Chumbley Laboratory for Social and Neural Systems Research University of Zurich With many thanks for slides amp images to FIL Methods group Detect an effect of unknown extent amp location ID: 484578
Download Presentation The PPT/PDF document "Multiple testing" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Multiple testing
Justin ChumbleyLaboratory for Social and Neural Systems ResearchUniversity of Zurich
With many thanks for slides & images to:
FIL Methods groupSlide2
Detect an effect of unknown extent & location
Realignment
Smoothing
Normalisation
General linear model
Statistical parametric map (SPM)
Image time-series
Parameter estimates
Design matrix
Template
Kernel
Voluminous
Dependent
p <0.05
Statistical
inferenceSlide3
t
=
contrast ofestimatedparametersvarianceestimate
t
Error at
a single voxelSlide4
H
0 ,
H1: zero/non-zero activationt = contrast ofestimatedparameters
variance
estimate
t
Error at
a single voxelSlide5
Decision:
H0 ,
H1: zero/non-zero activationt = contrast ofestimatedparameters
variance
estimate
t
h
Error at
a single voxelSlide6
Decision:
H0 ,
H1: zero/non-zero activationt = contrast ofestimatedparameters
variance
estimate
t
h
Error at
a single voxelSlide7
Decision:
H0 ,
H1: zero/non-zero activationt = contrast ofestimatedparameters
variance
estimate
t
h
Error at
a single voxelSlide8
Decision:
H0 ,
H1: zero/non-zero activationt = contrast ofestimatedparameters
variance
estimate
t
h
Decision rule (threshold)
h
,
determines related error rates
,
Convention: Penalize complexity
Choose
h
to give acceptable under
H
0
Error at
a single voxelSlide9
Types of error
Reality
H
1
H
0
H
0
H
1
True
negative (TN
)
True positive (TP
)
False positive (FP)
False negative (FN)
specificity:
1-
= TN / (TN + FP)
= proportion of actual negatives which are correctly identified
sensitivity (power): 1-
= TP / (TP + FN)= proportion of actual positives which are correctly identified
DecisionSlide10
Multiple tests
t
= contrast ofestimatedparametersvarianceestimate
t
h
t
h
h
t
h
What is the problem?Slide11
Multiple tests
t
= contrast ofestimatedparametersvarianceestimate
t
h
t
h
h
t
h
Penalize each independent
o
pportunity for error. Slide12
Multiple tests
t
= contrast ofestimatedparametersvarianceestimate
t
h
t
h
h
t
h
Convention: Choose
h
to limit
assuming family-wise
H
0Slide13
Issues
1. Voxels or regions
2. Bonferroni too harsh (insensitive)Unnecessary penalty for sampling resolution (#voxels/volume) Unnecessary penalty for independence Slide14
intrinsic
smoothness
MRI signals are aquired in k-space (Fourier space); after projection on anatomical space, signals have continuous supportdiffusion of vasodilatory molecules has extended spatial supportextrinsic smoothnessresampling during preprocessingmatched filter theorem deliberate additional smoothing to increase SNRRobustness to
between
-
subject
anatomical
differencesSlide15
Apply
high threshold: identify improbably high
peaks
Apply lower
threshold: identify improbably
broad peaks
Total number of regions?
Acknowledge/estimate
dependence
Detect effects in smooth landscape, not
voxelsSlide16
Null distribution?
1.
Simulate null experiments
2.
Model null experimentsSlide17
Use
continuous random field
theoryimage discretised continuous random field
Discretisation
(“lattice approximation”)
Smoothness
quantified
:
resolution
elements
(‘
resels
’)
similar, but not identical to
# independent observations
computed from spatial
derivatives of the residualsSlide18
Euler characteristic
(
h
)
threshold
an image at high
h
h
# blobs
FWER
E [
h]
= p (blob)
Slide19
General form for expected Euler characteristic 2,
F, & t fields
E[h(W)] = Sd Rd (W) rd (h)Small volumes: Anatomical atlas, ‘
functional
localisers
’, orthogonal
contrasts
,
volume
around
previously reported coordinates…
Unified Formula
R
d
(
W
):
d-dimensional
Minkowski functional of W
– function of dimension, space W and smoothness:
R0(W) =
(W) Euler characteristic of W
R1(W) = resel diameter R
2(W) = resel surface area
R3(
W) = resel volume
r
d
(
W
):
d-dimensional EC density of Z(x) – function of dimension and threshold,
specific for RF type:E.g. Gaussian RF: r0(h)
= 1- (h)
r1(h) = (4 ln2)1/2
exp(-h2/2) / (2
p)
r2(h) = (4 ln2)
exp(-h2/2) / (2
p
)
3/2
r
3
(
h
)
= (4 ln2)
3/2
(
h
2
-1)
exp
(-
h
2
/2
) / (2
p
)
2
r
4
(
h
)
= (4 ln2)
2
(
h
3
-
3
h
)
exp
(-
h
2
/2
) / (2
p
)
5/2
Slide20
Euler characteristic (EC) for 2D images
R = number of resels
h = threshold Set h such that E[EC] = 0.05 Example: For 100 resels, E [EC] = 0.049 for a Z threshold of 3.8. That is, the probability of getting one or more blobs where Z is greater than 3.8, is 0.049.
Expected EC values for an image of 100 reselsSlide21
Spatial extent: similarSlide22
Voxel, cluster and set level tests
e
u
hSlide23Slide24
ROI
Voxel
Field‘volume’
resolution
*
volume
independence
FWE
FDR
*
voxels/volume
Height
Extent
ROI
Voxel
F
ield
Height
Extent
There is a multiple testing problem (‘voxel’ or ‘blob’ perspective).
More corrections needed as ..
Detect an effect of
unknown
extent &
location
Volume
,
Independence
Slide25
Further reading
Friston KJ, Frith CD, Liddle PF, Frackowiak RS. Comparing functional (PET) images: the assessment of significant change. J Cereb Blood Flow Metab. 1991 Jul;11(4):690-9.
Genovese CR, Lazar NA, Nichols T. Thresholding of statistical maps in functional neuroimaging using the false discovery rate. Neuroimage. 2002 Apr;15(4):870-8.Worsley KJ Marrett S Neelin P Vandal AC Friston KJ Evans AC. A unified statistical approach for determining significant signals in images of cerebral activation. Human Brain Mapping 1996;4:58-73.