Why amp How Corey Mackenzie PhD C Psych http wwwlatrobeeduau scitecheng aboutstaff profileuname GDCumming http wwwlatrobeeduau psy researchcognitiveanddevelopmentalpsychology ID: 255295
Download Presentation The PPT/PDF document "The New Statistics:" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
The New Statistics:
Why & How
Corey Mackenzie, Ph.D., C. PsychSlide2
http://
www.latrobe.edu.au
/
scitecheng/about/staff/profile?uname=GDCumming
http://www.latrobe.edu.au/psy/research/cognitive-and-developmental-psychology/esciSlide3
Outline
Need for changes to how we conduct research
Three threats to research integrity
Shift from Null Hypothesis Sig Testing (NHST)3 “new” solutionsEstimationEffect sizesMeta-analysisSlide4
1
st
change to how we do research: Enhance research integrity by addressing three threatsSlide5
Threat to Integrity #1
We must have complete reporting of findings
Small or large effects, important or not
Challenging because journals have limited space and are looking for novel, “significant” findingsPotential solutionsOnline data repositoriesNew online journalsOpen-access journalsSlide6
Threat to Integrity #2
We need to avoid selection and bias in data analysis (e.g., cherry picking)
How?
Prespecified research in which critical aspects of studies are registered beforehandDistinguishing exploratory from prespecified studiesSlide7
Threat to Integrity #3
We need published replications (ideally with more precise estimates than original study)
Key for meta-analysis
Need greater opportunities to report themSlide8
2
n
change to how we do research: stop evaluating research outcomes by testing the null hypothesisSlide9
Problems with p-values
In April 2009, people rushed to Boots pharmacies in Britain to buy No. 7 Protect & Perfect Intense Beauty Serum. They were prompted by media reports of an article in the British Journal of Dermatology stating that the anti-ageing cream “produced statistically significant improvement in facial wrinkles as compared to baseline assessment (p = .013), whereas [placebo-treated] skin was not significantly improved (p = .11)”. The article claimed a statistically significant effect of the cream because p < .05, but no significant effect of the control placebo cream because p > .05. In other words, the cream had an effect, but the control material didn’t.Slide10
Problems with NHST
Kline (2004) What’s Wrong with Stats Tests
8 Fallacies about null hypothesis testing
Encourages dichotomous thinking, but effects come in shades of greyP = .001, .04, .06, .92NHST is strongly affected by sample sizeSlide11
Solution #1
Support for Bill 32 is 53% in a poll with an error margin of 2%
i.e., 53 (51-55 with 95% confidence)
vsSupport is statistically significantly greater than 50%, p < .01Slide12
Solution #2
http://en.wikipedia.org/wiki/
Effect_size
http://lsr-wiki-01.mrc-cbu.cam.ac.uk/statswiki/FAQ/effectSizeG*PowerSlide13
Solution #3
Meta-analysis
P-values have no (or very little) role except their negative influence on the file-drawer effect
Overcomes wide confidence intervals often given by individual studiesCan makes sense of messy and disputed research literaturesSlide14
Why do we love P?
Suggests importance
We’re reluctant to change
Confidence intervals are sometimes embarrassingly wide9 ±12But this accurately indicates unreliability of dataSlide15
Why might we change?
30 years of damning critiques of NHST
6
th edition of APA publication manualUsed by more than 1000 journals across disciplinesResearchers should “wherever possible, base discussion and interpretation of results on point and interval estimates”http://www.sagepub.com/journals/Journal200808/
manuscriptSubmissionSlide16
Epi
Example