Calculating Sample Size: Cohen’s Tables and G*Power. A practical example - PowerPoint Presentation

Slide1

Calculating Sample Size: Cohen’s Tables and G*Power. A practical example

Sample 4

Females

43%

Work on management positions

14%

Variable One

22%

Variable Two

31%

Variable Three

34%

Variable Four

39%

Variable Five

2 in 5

Additional Descriptive statistics

80%

More descriptive statistics

450

Subjects live in the city

.

Males

22%

Work on Management positions

.

3 in 5

We found some descriptive statistics

Variable One

14%

Variable Two

22%

Variable Three

31%

Variable Four

34%

Variable Five

39%

275

Subjects live in a rural area

.

29%

Of some other statistics.

Calculating Sample Size

Sample 4

Ouline

:

General Research Proposal Scenario

Cohen’s d effect concept

Pearson’s r effect concept

Type I and Type II errors

Cohen’s d tablesCalculating sample size

Pearson’s r tablesCalculating sample size

G*Power Tool

Linear Regression a prioriANOVA a priori

ANOVA post hocQuestions?

Calculating Sample Size

Sample 4

Common Scenario on Proposals on URM (Pre QRM) or Statistic Classes:

“I am conducting a correlational design and my chosen sample size is 25 subject” (no explanations provided)

My typical answer:

The sample size is something that we cannot just arbitrarily select, but must calculated based on our type of tests, the expected power, and the expected effect. The size, the power, and the effect are intimately related. Also, the specific tests to be performed play a role in this calculation (For example factor analysis).

About effect size:

An effect size is simply an objective and (usually) standardized measure of the magnitude of observed effect. The fact that the measure is standardized just means that we can compare effect sizes across different studies that have measured different variables . . . Many measures of effect size have been proposed, the most common of which are Cohen's d, Pearson's correlation coefficient r and the odds ratio" (Field, 2009, p. 57)

 Effect is very important because in addition to our test being significant, we can test "how significant' is the effect. There are many tools and tables to calculate the effect size.

Calculating Sample Size

Sample 4

Cohen’s d

The Cohen’s effect size is used as a complement to the significance test to show the magnitude of that significance or to represent the extent to which a null hypothesis is false. This calculation shows an estimated to calculate the size of observed differences between groups: small, medium or large. “Cohen's d statistic represents the standardized mean differences between groups. Similar to other means of standardization such as z scoring, the effect size is expressed in standard score units” (Salkind, 2010, p. 2)

In general, Cohen's d is defined as

where

d

represents the effect size,

μ

1 and

μ

2 represent the two population means, and σ∊ represents the pooled within-group population standard deviation, but in practice we use the sample data means. Cohens’ suggestions about what constitutes a large, medium or large effects are:

d = 0.2 (small),

d = 0.5 (medium) d = 0.8 (large).

Cohen’s d

Sample 4

Pearson’s r

Pearson’s r “correlation coefficient” that is typically known as the measure of relationships between continuous variables, can also be used to quantify the differences in means between two groups (similar to Cohen’s d). Cohen’s also suggested some common sizes (Field, 2017)

r = 0.10 (small effect): In this case the effect explains 1% of the total variance.

r = 0.30 (medium effect): The effect accounts for 9% of the total variance.

r = 0.50 (large effect): The effect accounts for 25% of the variance.

Pearson’s r

Sample 4

A Type I error (or false positive) is when we believe that there is a genuine effect when it is not. The opposite (or false negative) is when we believe that there is no effect where in reality there is. The most common acceptable probability of this error is .2 (or 20%) and it is called the β-level. This means that if we took 100 samples (in which the effect exists) we will fail to detect the effect in 20 of those samples. (Field, 2017).

“The power of a test is the probability that a given test will find an effect assuming that one exists in the population. This is the opposite of the probability that a given test will not find an effect assuming that one exists in the population, which, as we have seen, is the β-level (i.e., Type II error rate” (Field, 2017, p. 47).

The problem with the significance (whether is .01, .05, or .10 ) is that does not tell us the importance of the effect, but we can measure the size of the effect in a standardized way. So the effect size is an standardized measure of the magnitude of the observed effect

Type I and Type II Errors

Sample 4

Type I and Type II Errors

Sample 4

TYPE I and TYPE II error

.

Calculating Sample Size

p value

significance

Sample 4

TYPE I and TYPE II error

Calculating Sample Size

significance

p value

Sample 4

TYPE I and TYPE II error

.

Calculating Sample Size

Power is often expressed as 1 − β, where β represents the likelihood of committing a Type II error (i.e., the probability of incorrectly retaining the null hypothesis). Betas can range from .00 to 1.00. When the beta is very small (close to .00), the statistical test has the most power. For example, if the beta equals .05, then statistical power is .95. Multiplying statistical power by 100 yields a power estimate as a percentage. Thus, 95% power (1 − β = .95 × 100%) suggests that there is a 95% probability of correctly finding a significant result if an effect exists (Christopher &

Nyaradzo

, 2010, p. 3)

Sample 4

Calculating Sample Size using Cohen’s Tables

Using d Effects

Sample 4

Calculating Sample Size using Cohen’s Tables

Using d Effects

Sample 4

Calculating Sample Size using Cohen’s Tables

Using r Effects

Sample 4

Calculating Sample Size using Cohen’s Tables

Using r Effects

Sample 4

Calculating Sample Size using Cohen’s Tables

Using d Effects

Sample 4

G*Power Download

http://www.G*Power.hhu.de/

G*Power Manual

http://www.G*Power.hhu.de/fileadmin/redaktion/Fakultaeten/Mathematisch-Naturwissenschaftliche_Fakultaet/Psychologie/AAP/G*Power/G*PowerManual.pdf

https://youtu.be/Kvz5AHFBEvQ

G*Power F-test: Linear Multiple Regression, Fixed Model, R-squared deviation from zero

Using G*Power to calculate Sample Size (A Priori) HD MANOVA special Effects and Interactions

http://youtu.be/aOnZKEj3Wmg

Calculating Sample Size using G*Power

Sample 4

Calculating Sample Size

Sample 4

Calculating Sample Size

Sample 4

Calculating Sample Size

Sample 4

References

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York: Lawrence Erlbaum Associates.

Christopher, A. S., &

Nyaradzo

, H. M. (2010). Statistical Power, Sampling, and Effect Sizes: Three Keys to Research Relevancy. Counseling Outcome Research and Evaluation, 1(2), 1-18. doi:10.1177/2150137810373613

Field, A. (2017). Discovering statistics using SPSS (5th ed.). Thousand Oaks, CA: Sage Publications.

Salkind, N. (2010). Encyclopedia of Research Design. doi:10.4135/9781412961288

Calculating Sample Size