Power Is The conditional probability that one will reject the null hypothesis given that the null is really false by a specified amount and given certain other specifications such as sample size and the criterion of statistical significance alpha ID: 1034295
Download Presentation The PPT/PDF document "Power Analysis An Overview" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. Power AnalysisAn Overview
2. Power IsThe conditional probabilitythat one will reject the null hypothesisgiven that the null is really falseby a specified amountand given certain other specifications such as sample size and the criterion of statistical significance (alpha).
3. A Priori Power AnalysisYou want to find how many cases you will need to have a specified amount of power givena specified effect sizethe criterion of significance to be employedwhether the hypotheses are directional or nondirectionalA very important part of the planning of research.
4. A Posteriori Power AnalysisYou want to find out what power would be for a specifiedeffect sizesample sizeand type of analysisBest done as part of the planning of research.could be done after the research to tell you what you should have known earlier.
5. Retrospective Power AnalysisAlso known as “observed power.”What would power be if I were torepeat this researchwith same number of cases etc.and the population effect size were exactly what it was in the sample in the current researchSome stat packs (SPSS) provide this.
6. Hoenig and Heisey The American Statistician, 2001, 55, 19-24Retrospective power asks a foolish question.It tells you nothing that you do not already know from the p value.After the research you do not need a power analysis, you need confidence intervals for effect sizes.
7. One Sample Test of MeanExperimental treatment = memory drug H0: µIQ 100; σ = 15, N = 25Minimum Nontrivial Effect Size (MNES) = 2 points.Thus, H1: µ = 102.
8.
9. = .05, MNES = 2, Power = ?Under H0, CV = 100 + 1.645(3) = 104.935will reject null if sample mean 104.935Power = area under H1 104.935Z = (104.935 102)/3 = 0.98 P(Z > 0.98) = .1635 = 1 - .16 = .84Hope you like making Type II errors.
10.
11. = .05, ES = 5, Power = ?What if the Effect Size were 5?H1: µ = 105 Z = (104.935 105)/3 = 0.02 P(Z > 0.02) = .5080 It is easier to find large things than small things.
12. H0: µ = 100 (nondirectional)CVLower = 100 1.96(3) = 94.12 or less CVUpper = 100 + 1.96(3) = 105.88 or more If µ = 105, Z = (105.88 105)/3 = .29 P(Z > .29) = .3859Notice the drop in power.Power is greater with directional hypotheses IF you can correctly PREdict the direction of the effect.
13. Type III Errorµ = 105 but we happen to get a very low sample mean, at or below CVLower. We would correctly reject H0But incorrectly assert the direction of effect.P(Z < (94.12 105)/3) = P(Z < 3.63), which is very small.
14. H0: µ = 100, N = 100Under H0, CV = 100 + 1.96(1.5) = 102.94If µ = 105, Z = (102.94 105)/1.5 = -1.37 P(Z > -1.37) = .9147Anything that reduces the SE increases power (increase N or reduce σ)
15.
16. Reduce to .01CVUpper = 100 + 2.58(1.5) = 103.87 If µ = 105, Z = (103.87 105)/1.5 = -0.75P(Z > 0.75) = .7734Reducing reduces power, ceteris paribus.
17. Power, , N, and dA Visualizationhttp://rpsychologist.com/d3/NHST/
18. z versus tUnless you know σ (highly unlikely), you really should use t, not z.Accordingly, the method I have shown you is approximate.If N is not small, it provides a good approximation.It is primarily of pedagogical value.
19. Howell’s MethodThe same approximation method, butYou don’t need to think as muchThere is less arithmeticYou need his power table
20. H0: µ = 100, N = 25, ES = 5IQ problem, minimum nontrivial effect size at 5 IQ points, d = (105 100)/15 = 1/3.with N = 25, = (1/3)5 = 1.67.Using the power table in our text, for a .05 two-tailed test, power = 36% for a of 1.60 and 40% for a of 1.70
21. = .05, = 1.67power for = 1.67 is 36% + .7(40% 36%) = 38.8%
22. I Want 95% PowerFrom the table, is 3.60.If I get data on 117 cases, I shall have power of 95%.With that much power, if I cannot reject the null, I can assert its near truth.
23. The Easy Way: GPowerTest family: t testsStatistical test: Means: Difference from constant (one sample case)Type of power analysis: Post hoc: Compute achieved power – given α, sample size, and effect sizeTails: TwoEffect size d: 0.333333 (you could click “Determine” and have G*Power compute d for you)α error prob: 0.05Total sample size: 25This is NOT an approximation, it uses the t distribution.
24.
25.
26. Significant Results, Power = 36%Bad news – you could only get 25 casesGood news – you got significant resultsBad news – the editor will not publish it because power was low.Duh. Significant results with low power speaks to a large effect size.But also a wide confidence interval.
27. Nonsignificant ResultsPower = 36%You got just what was to be expected, a Type II error.Power = 95%If there was anything nontrivial to be found, you should have found it, so the effect is probably trivial.The confidence interval should show this.
28. I Want 95% PowerHow many cases do I need?
29. Sensitivity AnalysisI had lots of data, N = 1500, but results that were not significant.Can I assert the range null that d 0.Suppose that we consider d 0 if -0.1 d +0.1.For what value of d would I have had 95% power?
30. If the effect were only .093, I would have almost certainly found it.I did not find it, so it must be trivial in magnitudeI’d rather just compute a CI.
31. Two Independent Samples Test of MeansEffective sample size, .The more nearly equal n1 and n2, the greater the effective sample size.For n = 50, 50, it is 50. For n =10, 90, it is 18.
32. Howell’s Method: Aposteriorin1 = 36, n2 = 48, effect size = 40 points, SD = 98From the power table, power = 46%.
33. I Want 80% PowerFor effect size d = 1/3.From power table, = 2.8 with alpha .05I plan on equal sample sizes.Need a total of 2(141) = 282 subjects.
34. G*PowerWe have 36 scores in one group and 48 in another.If µ1 - µ2 = 40, and σ = 98, what is power?
35. I Want 80% Powern1 = n2 = ? for d = 1/3, = .05, power = .8.You need 286 cases.
36. Allocation Ratio = 9n1/n2 = 9. How many cases needed now?You need 788 cases!
37. Two Related Samples, Test of MeansIs equivalent to one sample test of null that mean difference score = 0.With equal variances, The greater , the smaller the SE, the greater the power.
38. dDiffAdjust the value of d to take into account the power enhancing effect of this design.
39. Howell’s Method: A PosterioriEffect size = 20 points:Cortisol level when anxious vs. when relaxedσ1 = 108, σ2 = 114 = .75N = 16Power = ?
40. Howell’s MethodPooled SD = d = 20/111 = .18. From the power table, power = 17%.
41. I Want 95% Power
42. G*PowerDependent means, post hoc.Set the total sample size to 16. Click on “Determine.”Select “from group parameters.”Calculate and transfer to main window.
43. Power = 16%
44. I Want 95% PowerYou need 204 subjects.
45. Type III ErrorsYou have correctly rejected H0: µ1= µ2.Which µ is greater?You conclude it is the one whose sample mean was greater.If that is wrong, you made a Type III error.This probability is included in power.To exclude it, see http://core.ecu.edu/psyc/wuenschk/StatHelp/Type_III.htm
46. Bivariate Correlation/RegressionH0: Misanthropy-AnimalRights = 0For power = .95, = .05, = .2, N = ?
47. One-Way ANOVA, Independent Samplesf is the effect size statistic. Cohen considered .1 to be small, .25 medium, and .4 large.In terms of 2, this is 1%, 6%, 14%.
48. Comparing three populations on GRE-QMinimum nontrivial effect size is if each ordered mean differs from the next by 20 points (about 1/5 SD), = 100, n = 11.(µj - µ)2 = 202 + 02 + 202 = 800
49. Let G*Power Calculate f
50. Power is only .115
51. I Want 70% Power
52. Analysis of CovarianceAdding covariates to the ANOVA model can increase power.If they are well correlated with the dependent variable.Adjust the f statistic this way, where r is the corr between covariate(s) and Y.
53. k = 3, f = .1, power = .95, N = ?f = .1 is a small effect.Ouch, that is a lot of data we need here.
54. Add a Covariate, r = .7
55. Reduce the error df by 1 for each covariate
56. Factorial ANOVA, Independent SamplesWe plan a 3 x 4 ANOVA.Want power = 80% for medium-sized effect.Sample sizes will be constant across cellsWill be three F tests, with df = 2 (the three level factor)3 (the four level factor)6 (the interaction)
57. The Three-Level FactorFor a medium effect, you need 158 cases, = 158/12 = 13.2 per cell. Bump N up to 14(12) = 168 cases.
58. The Four Level Factor
59. The Interaction
60. Which N to Obtain?You will not have the same power for each effect.If only interested in main effects, get the N required for them.Suppose we are interested in the interaction. 225/12 = 18.75 cases/cell, bump up to 19(12) = 228 cases.This would give you 93% power for the one main effect and almost 90% for the other.
61. Let GPower Determine the fWhat f corresponds to 2 of 6% ?Click Determine and enter 2 and 1- 2
62. Adjusting f for Other EffectsThat f ignores the fact that other effects in the model reduce the error variance.Suppose that I expect other effects to account for 14% of the total variance.I enter 6% for the effect and (100-6-14) = 80% for error.
63. ANOVA With Related FactorsFor the univariate-approach analysis, you need add two more parametersThe correlation between scores in one condition and those in another conditionEpsilon, if you suspect that correlation to differ across pairs of conditionsk = 4, f = .25 (medium), power = .95, r = .5, = 1.
64. Need only 36 Cases
65. Increase r to .75
66. Estimate to be .6
67. Multivariate Approach: No Sphericity Assumption
68. Contingency Table Analysis (Two-Way)Effect size = P0i is the population proportion in cell i under the null hypothesis. P1i is the population proportion in cell i under the alternative hypothesis..1 is small, .3 medium, .5 largeFor a 2 x 2, w is identical to
69. 2 x 4, 95% Power, w = .1:Need 1,717 Cases !
70. MANOVA and DFAThere will be one root (discriminant function, canonical variate) for each treatment df.Each is a weighted linear combination of the Y variables.Each maximizes the ratio of the among groups SS to within group SS (the eigenvalue, ).Within a set, each root is independent of the others.
71. Test Statistics for a Given EffectFor each df there will be one and Hotellings Trace: Wilks Lambda: Pillai’s Trace: Roy’s Greatest Root: for the first root
72. The Effect Size ParameterIt is f..1 is small, .25 medium, .4 large.GPower will convert from value of trace to f if you wish.We plan a one-way MANOVA, four groups, two Y variables.Want 95% power for a medium effect.
73. Planning the 1-Way MANOVA
74. Planning the Post-MANOVAWhat will do you do if the MANOVA is significant?You decide to do two univariate ANOVAs, one on each outcome variable.How much power would you have for each of those?
75. Oh My, Only 78% Power
76. But I Want 95% Power !You have it, for the canonical variate you have created, just not for the original variables.Maybe you should just work with the canonical variates.But maybe you, or your editors, don’t really understand canonical variates.
77. 95% Power for the Post-MANOVA Analyses of VarianceDoes the significant MANOVA protect you from inflating familywise error?You decide to employ the Bonferroni correction.To keep familywise error capped at .05, you use a .025 criterion for each of the two ANOVAs.How many cases do you need?
78. Need 320 Cases. Ouch !
79. The Type I Boogey ManParanoid obsession with this creature can really mess up your research life.If that univariate ANOVA is significant, you plan to make, for each Y, six comparisons (1-2, 1-3, 1-4, 2-3, 2-4, 3-4).Bonferroni per comparison alpha = .05/12 = .00416.How many cases now?
80. Need 165 x 4 = 660 Cases !330/2 groups = 165 per group.
81. LinksAssorted Stats LinksG*Power 3 – download siteUser Guide – sorted by type of analysisUser Guide – sort by test distributionInternet Resources for Power AnalysisList of the analyses available in G*Power