Im so confident The universe is not one of clockwork Everything we measure has uncertainty How can we confidently draw conclusions and make decisions without sacrificing our intelligence Quantify Your Uncertainty ID: 680806
Download Presentation The PPT/PDF document "How to Lie With Statistics" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
How to Lie With StatisticsSlide2Slide3
I'm so confident!
The universe is not one of clockwork
Everything we measure has uncertainty
How can we confidently draw conclusions and make decisions without sacrificing our intelligence?Slide4
Quantify Your Uncertainty!
Family of intervals to handle this
Confidence
Prediction
Comprised of
Point estimator
Degree of Confidence
Uncertainty
Rely on
Underlying distribution (assumption)Data collectionSlide5
Formula
Z*,t*
Margin of Error
Slide6
What could go wrong?
Poor data collection
"I talked to my 10 closest friends – Romney has got the election in the BAG!"
Wrong assumptions
Excessive uncertainty
Unrealistic expectations
Degree of confidence is too high/low
Wrong type of interval
Prediction versus ConfidenceSlide7
Degree of Confidence
What are we confident in?
The Process
If we collect data from the same population, over and over and over again, we can expect our interval to contain the true (correct) value x% of the
time where x is our
degree of confidence
(confidence level)
"The feeling or belief that
one can rely on someone
or something"
-- What I got
Googling
"What is Confidence"Slide8
Degree of Confidence
What is confidence NOT?
A probability of any single interval containing the true value
The % of all population values contained in the interval
A product of your self-esteemSlide9
Lancet Survey
October 29, 2004
1
The Lancet
published a study on the effects of the US invasion of Iraq, specifically on the number of additional civilian deaths due to the invasion
The data estimated 98,000 excess deaths
They were 95% confident that the true number of excess deaths was between 8,000 and 194,000
1
Les, R.,
Lafta
, R., Garfield, R.,
Khudhairi
, J., & Burnham, G. (2004).
"Mortality before and after the 2003 invasion of Iraq
: Cluster sample survey. The Lancet, 1857-1864.Slide10
Lancet Survey
October
11, 2006
2
The Lancet
published a follow-up study
The
data now
estimated 654,965 excess deathsThey were 95% confident that the true number of excess deaths was between 392,979
and
942,636
Claimed that they asked for death certificates 87% of the time and that 92% of the time they asked they were shown a certificate (80% of reported deaths had a certificate to verify the death)
Iraqi Ministry of Health only issued ~50,000 death certificates for post-war violent deaths
2 Burnham, G.,
Lafta, R., Doocy, S., & Roberts, L. (2006). Mortality after the 2003 invasion of Iraq
: a cross-sectional cluster sample survey. The Lancet, Retrieved from http://www.brussellstribunal.org/pdf/lancet111006.pdfSlide11
Statistical Ties
Des Moines Register Iowa Poll Results Published 9/29/2012
Obama 49%
Romney 45%
Other 4%
Undecided 2%
Margin of Error ±3.8%
Who is winning?
Register claims Obama has a 4% lead
Statistically inconclusiveThe poll results give an ~84% chance that Obama is really aheadSlide12
Multiplicity of ErrorSlide13
Multiplicity of Error
Dr. Elisabeth
Targ
made headlines when she identified an effect of remote faith healing
The study results were published in the peer-reviewed journal
The Western Journal of Medicine
under the title "A randomized double-blind study of the effect of distant healing in a population with advanced AIDS. Report of a small scale study"
Results from the double blind study showed that members of the control group spent 600% more days in the hospital and contracted 300% as many AIDS-related illnesses
Considered the most rigorous, scientifically valid experiment to demonstrate the effect of remote faith healingSlide14
Multiplicity of Error
What was learned until after the fact:
Study was designed to look for differences in mortality rates
Triple-drug Anti-Retroviral therapy became common practice one month into the six month
study
Only one person died – insufficient data to make conclusions
Adjusted the scope to look at HIV physical symptoms and Quality of Life
Also inconclusive
Adjusted scope again to look at Mood State scores
Control group performed better than treatment group
Adjusted scope again to look at Hospital Stays and Doctor Visits
Treatment group performed statistically significantly better
Adjusted scope to include at 23 illnesses associated with AIDS
Data hadn't been collected as part of the study design, so blindness was broken and data was aggregated from hospital charts
Sharpshooter fallacySpray bullets down range, then draw your target around a clusterSlide15
Multiplicity of Error
Keeping ourselves honest
Bonferroni
correction
Don't use
α
, use
where m is the number of comparisons
Conservative
Holm-
Bonferroni
correction
Compare the smallest p-value to
If it is significant, continue testing
Otherwise, stop
Compare the next smallest p-value to
Continue comparing the p-values until one is not significant, then stopLess conservativeOthers that are implemented in statistics packages (Minitab, JMP, etc…)Tukey's HSDDunnet'sAnd many more (you can't get a Ph.D. in statistics without inventing a correction to name after yourself)
Slide16
Biggest Lie Ever Told
Vitamin C prevents or helps cure the Common Cold
Linus Pauling (an incredibly bright, talented scientist) championed the need for
megadoses
of Vitamin C to prevent or cure the Common Cold (and cancer) with the publishing of his book
Vitamin C and the Common Cold
in 1970
Had a whole institute setup to further study the benefits of
megadosing
Vitamin CAt least 30 experiments have been performed to test the ability of Vitamin C to prevent or cure the Common Cold in large groups of people (dates of experiments go from 1967-2001)
Not a single one has found a link between Vitamin C and the
Common Cold