Evaluating Differences and Changes Our overall customer satisfaction score increased from 92 percent 3 months ago to 935 percent today Did customer satisfaction really increase Should we celebrate ID: 912505
Download Presentation The PPT/PDF document "14 S tatistical Testing of Differences a..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
14 Statistical Testing of Differences and Relationships
Slide2Evaluating Differences and Changes
“Our overall customer satisfaction score increased from 92 percent 3 months ago to 93.5 percent today.”
Did customer satisfaction really increase? Should we celebrate?
“In a recent product concept test, 19.8 percent of those surveyed said they were very likely to buy the new product they evaluated.”
Is this good? Is it better than the results we got last year for a similar product? What do these results suggest in terms of whether to introduce the new product?
Slide3These are the common questions in marketing and marketing research. Although considered boring by some, statistical hypothesis testing is important because it helps researchers get
closer*
to answers to these questions.
(
Remark:
“
closer
” because certainty is never achieved in answering these questions in marketing research.)
Slide4Statistical Significance
Statistical significance:
A difference that is large enough that is not likely to have occurred because of chance or sampling error.
Slide5Things to keep in mind on significance tests
Random samples are assumed.
Big data does not mean “good” data.
Don’t
overrely
on significance testing.
Slide6Hypothesis Testing
Hypothesis:
Assumption or theory that a researcher or manager makes about some characteristic of the population under study.
The marketing researcher is often faced with the question of whether research results are different enough from the norm that some element of the firm’s marketing strategy should be changed.
Slide7Considering the following situations:
The results of a tracking survey show that awareness of a product is lower than it was in a similar survey conducted six months ago.
Are these results significantly lower? Are the results sufficiently lower to call for a change in advertising strategy?
Slide8Considering the following situations:
A product manager believes that the average purchaser of his product is 35 years of age. A survey is conducted to test this hypothesis and the survey shows that the average purchaser of the product is 38.5 years of age.
Is the survey result different enough from the product manager’s belief to cause him to conclude that his belief is incorrect?
Slide9All of these questions can be evaluated with some kind of statistical test. In hypothesis testing, the researcher determines whether a hypothesis concerning some characteristic of the population is likely to be true, given the evidence. A statistical hypothesis test allows us to calculate the probability of observing a particular result if the stated hypothesis is actually true.
Slide10Steps in Hypothesis Testing
Step 1: Stating the Hypothesis
Hypotheses are stated using 2 basic forms:
The null hypothesis H
0
(the
hypothesis of status quo
) is the hypothesis that is tested against its compliment
The alternative hypothesis H
a
(the
research hypothesis of interest
)
Slide11Steps in Hypothesis Testing
Step 1: Stating the Hypothesis
Ex:
Suppose the manager of Burger City believes that his operational procedures will guarantee that the average customer will wait 2 minutes in the drive-in window line. He conducts research, based on the observation of 1,000 customers at randomly selected stores at randomly selected times. The average customer observed in this study spends 2.4 minutes in the drive-in window line.
Slide12Steps in Hypothesis Testing
Step 1: Stating the Hypothesis
The null hypothesis and the alternative hypothesis might be stated as follows:
The null hypothesis H
0
:
Mean waiting time = 2 minutes
The alternative hypothesis H
a
:
Mean waiting time ≠ 2 minutes
Slide13Steps in Hypothesis Testing
Step 2: Choosing the Appropriate Statistical Test
The analyst must choose the appropriate statistical test, given the characteristics of situation under investigation.
Slide14Commonly Used Statistical Hypothesis Tests
Independent vs. Related Samples
Independent samples:
Samples in which measurement of a variable in one population has no effect on measurement of the variable in the other.
Ex.
Men and women were interviewed in a particular survey regarding their frequency of eating out, there is no way that man’s response could affect/change the way woman would respond to a question in the survey.
Slide15Commonly Used Statistical Hypothesis Tests
Independent vs. Related Samples
Related samples:
Samples in which measurement of a variable in one population may influence measurement of the variable in the other.
Ex.
The researcher needed to determine the effect of a new advertising campaign on consumer awareness of a particular brand. To do this, the researcher might survey a random sample of consumers before introducing the new campaign and then survey the same sample of consumers 90 days after the new campaign was introduced. These samples are not independent. The measurement of awareness 90 days after the start of the campaign may be affected by the first measurement.
Slide16Commonly Used Statistical Hypothesis Tests
Degrees of Freedom
Degree of freedom:
Number of observations in a statistical problem that are free to vary.
Many statistical tests require the researcher to specify degrees of freedom in order to find the critical value of the test statistic from the table for that statistic.
The number of degree of freedom (
d.f
.) is equal to the number of observations minus the number of assumptions/constraints necessary to calculate a statistic.
Slide17Goodness of Fit
Chi-Square Test
Test of the goodness of fit between the observed distribution and the expected distribution of a variable.
The
test is applied when you have two
categorical variables
from a single population. It is used to determine whether there is a significant association between the two variables.
Slide18Hypotheses about One Mean
Z
Test
Hypothesis test used for a single mean if the sample is large enough (n ≥ 30)and drawn at random.
t Test
Hypothesis test used for a single mean if the sample is too small (n < 30) to use the
Z
test.
Slide19Analysis of Variance (ANOVA)
Analysis of variance (ANOVA):
Test for the differences among the means
of several independent groups (3 or more sample means).
Slide20Steps in Hypothesis Testing
Step 3: Developing a Decision Rule
Decision rule:
rule or standard used to determine whether to reject or fail to reject the null hypothesis.
The significance level ( ) is critical in the process of choosing between the null and alternative hypotheses. The level of significance---.10, .05, or .01, for example---is the probability that is considered too low to justify acceptance of the null hypothesis.
Slide21Steps in Hypothesis Testing
Step 3: Developing a Decision Rule
Consider a situation in which the researcher has decided that she wants to test a hypothesis at the .05 level of confidence. This means that she will reject the null hypothesis if the test indicates that the probability of occurrence of the observed result because of the chance or sampling error is less than 5 percent.
Rejection of the null hypothesis is equivalent to supporting the alternative hypothesis, but statistically, we can only state that the null hypothesis is not true.
Slide22Steps in Hypothesis Testing
Step 4: Calculating the Value of the Test Statistic
The researcher does the following:
Uses appropriate formula to calculate the value of statistic for the test chosen.
Compares the value just calculated to the critical value of the statistic (from appropriate table), based on the decision rule chosen.
Based on the comparison, determines to either reject or fail to reject the null hypothesis H
0
.
Slide23Steps in Hypothesis Testing
Step 5: Stating the Conclusion
The conclusion summarizes the results of the test. It should be stated from the perspective of the original research question.
Slide24Types of Errors in Hypothesis Testing
Type I error ( error):
Rejection of the null hypothesis when, in fact, it is true.
The researcher may reach this incorrect conclusion because the observed difference between the sample and population values is due to sampling error.
The probability of committing a type I error is referred to as the
alpha ( ) level
. Conversely, 1 - is the probability of marking a correct decision by not rejecting the null hypothesis when, in fact, it is true.
Slide25Types of Errors in Hypothesis Testing
Type II error (
β
error):
Failure to reject the null hypothesis when, in fact, it is false.
A type II error is referred to as a
beta (
β
)
error.
The value 1-
β
reflects the probability of marking a correct decision in rejecting the null hypothesis when, in fact, it is false.
Slide26Types of Errors in Hypothesis Testing
The level of is set by the researcher, after consulting with his/her client, considering the resources available for the project, and considering the implication of marking type I and type II errors.
However, the estimation of
β
is more complicated and it is beyond the scope of our discussion.
Actual
State of the Null Hypothesis
Fail to Reject H
0
Reject
H
0
H
0
is true
Correct (1 - )
Type I error ( )
H
0
is false
Type II error (
β
)
Correct (1 -
β
)
Slide27One-Tailed vs. Two-Tailed Test
Tests are either one-tailed or two-tailed. The decision as to which to use depends on the nature of the situation and what the researcher is trying to demonstrate.
Ex.
When the quality control department of a fast-food org. receives a shipment of chicken breasts from one of its vendors and needs to determine whether the product meets specifications in regard to fat content, a one-tailed test is appropriate. The shipment will be rejected if it does not meet minimum specifications.
Slide28One-Tailed vs. Two-Tailed Test
Ex.
On the other hand, the managers of the meat company that supplies the product should run two-tailed tests to determine two factors. First, they must make sure that the product meets the min. specifications of their customer before they ship it. Second, they want to determine whether the product exceeds specifications because this can be costly to them. If they are consistently providing a product that exceeds the level of quality they have contracted to provide, their costs may be unnecessarily high.