STAT 250 Dr Kari Lock Morgan SECTION 43 Significance level 43 Statistical conclusions 43 pvalue and H 0 If the pvalue is small then a statistic as extreme as that observed would be unlikely if the null hypothesis were true providing significant evidence against H ID: 484795
Download Presentation The PPT/PDF document "Hypothesis Testing: Significance" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Hypothesis Testing: Significance
STAT 250Dr. Kari Lock Morgan
SECTION 4.3
Significance level (4.3)
Statistical conclusions (4.3)Slide2
p-value and H0
If the p-value is small, then a statistic as extreme as that observed would be unlikely if the null hypothesis were true, providing significant evidence against H0The smaller the p-value, the stronger the evidence against the null hypothesis and in favor of the alternativeSlide3
The
smaller the p-value, the stronger the evidence against Ho.The smaller the p-value, the stronger the evidence against Ho.The smaller the p-value, the stronger the evidence against H
o
.
p-value and H
0Slide4
Which of the following p-values gives the strongest evidence
against H0? 0.005 0.1 0.32 0.56 0.94
p-value and H
0Slide5
Which of the following p-values gives the strongest evidence
against H0? 0.22 0.45 0.03 0.8 0.71
p-value and H
0Slide6
Two different studies obtain two different p-values. Study A obtained a p-value of 0.002 and Study B obtained a p-value of 0.2. Which study obtained stronger evidence
against the null hypothesis? Study A Study Bp-value and H0Slide7
Formal Decisions
If the p-value is small:the sample would be extreme if H0 were true the results are statistically significantREJECT H0we have evidence for HaIf the p-value is not small:the sample would not be too extreme if H0 were truethe results are not statistically significantDO NOT REJECT H0the test is inconclusive; either H0 or Ha may be trueSlide8
A formal hypothesis test has only two possible conclusions:
The p-value is small: reject the null hypothesis in favor of the alternativeThe p-value is not small: do not reject the null hypothesisFormal DecisionsHow small?Slide9
Significance Level
The significance level, , is the threshold below which the p-value is deemed small enough to reject the null hypothesis p-value < Reject H0 p-value > Do not Reject H0Slide10
Significance Level
If the p-value is less than , the results are statistically significant, and we reject the null hypothesis in favor of the alternativeIf the p-value is not less than , the results are not statistically significant, and our test is inconclusiveOften = 0.05 by default, unless otherwise specifiedhttps://www.openintro.org/stat/why05.php Slide11
Statistical Significance
p-value < αResults would be rare, if the null were trueReject H0We have evidence that the alternative is true!
p-value ≥ α
Results would not be rare, if the null were true
Do not reject H
0
We can make no conclusions either waySlide12
Statistical Conclusions
In a hypothesis test of H0: = 10 vs Ha: < 10 the p-value is 0.002. With α = 0.05, we conclude: Reject H0 Do not reject H0 Reject Ha Do not reject HaSlide13
Statistical Conclusions
In a hypothesis test of H0: = 10 vs Ha: < 10 the p-value is 0.002. With α = 0.01, we conclude: There is evidence that = 10 There is evidence that < 10 We have insufficient evidence to conclude anythingSlide14
Statistical Conclusions
In a hypothesis test of H0: = 10 vs Ha: < 10 the p-value is 0.21. With α = 0.01, we conclude: Reject H0 Do not reject H0 Reject Ha Do not reject HaSlide15
Statistical Conclusions
In a hypothesis test of H0: = 10 vs Ha: < 10 the p-value is 0.21. With α = 0.01, we conclude: There is evidence that = 10 There is evidence that < 10 We have insufficient evidence to conclude anythingSlide16
H
0 : X is an elephantHa : X is not an elephantWould you conclude, if you get the following data? X walks on two legs X has four legs
Elephant ExampleSlide17
“
For the logical fallacy of believing that a hypothesis has been proved to be true, merely because it is not contradicted by the available facts, has no more right to insinuate itself in statistical than in other kinds of scientific reasoning…” -Sir R. A. Fisher Never Accept H0“Do not reject H0” is not the same as “accept H0”! Lack of evidence against H0 is NOT the same as evidence for H0!Slide18
Resveratrol, an ingredient in red wine and grapes, has been shown to promote weight loss in rodents, and has recently been investigated in primates (specifically, the Grey Mouse Lemur).
A sample of lemurs had various measurements taken before and after receiving resveratrol supplementation for 4 weeksRed Wine and Weight LossBioMed Central (2010, June 22). “Lemurs lose weight with ‘life-extending’ supplement resveratrol. Science Daily.Slide19
Red Wine and Weight Loss
In the test to see if the mean resting metabolic rate is higher after treatment, the p-value is 0.013.Using = 0.05, is this difference statistically significant? (should we reject H0: no difference?) Yes NoSlide20
Red Wine and Weight Loss
In the test to see if the mean resting metabolic rate is higher after treatment, the p-value is 0.013.Using = 0.05, what can we conclude? Mean resting metabolic rate is higher after resveratrol supplementation in lemurs Mean resting metabolic rate is not higher after resveratrol supplementation in lemurs NothingSlide21
Red Wine and Weight Loss
In the test to see if the mean body mass is lower after treatment, the p-value is 0.007.Using = 0.05, is this difference statistically significant? (should we reject H0: no difference?) Yes NoSlide22
Red Wine and Weight Loss
In the test to see if the mean body mass is lower after treatment, the p-value is 0.007.Using = 0.05, what can we conclude? Mean body mass is higher after resveratrol supplementation in lemurs Mean body mass is not higher after resveratrol supplementation in lemurs NothingSlide23
Red Wine and Weight Loss
In the test to see if locomotor activity changes after treatment, the p-value is 0.980.Using = 0.05, is this difference statistically significant? (should we reject H0: no difference?) Yes NoSlide24
Red Wine and Weight Loss
In the test to see if locomotor activity changes after treatment, the p-value is 0.980.Using = 0.05, what can we conclude? Locomotor activity changes after resveratrol supplementation in lemurs Locomotor activity does not change after resveratrol supplementation in lemurs NothingSlide25
Red Wine and Weight Loss
In the test to see if mean food intake changes after treatment, the p-value is 0.035.Using = 0.10, is this difference statistically significant? (should we reject H0: no difference?) Yes NoSlide26
Red Wine and Weight Loss
In the test to see if mean food intake changes after treatment, the p-value is 0.035.Using = 0.01, is this difference statistically significant? (should we reject H0: no difference?) Yes NoSlide27
Informal strength of evidence against H
0:Formal decision of hypothesis test, based on = 0.05 :
Statistical ConclusionsSlide28
Multiple Sclerosis and Sunlight
It is believed that sunlight offers some protection against multiple sclerosis, but the reason is unknownResearchers randomly assigned mice to one of: Control (nothing) Vitamin D Supplements UV Light All mice were injected with proteins known to induce a mouse form of MS, and they observed which mice got MS
Seppa, Nathan. “Sunlight may cut MS risk by itself”,
Science News,
April 24, 2010 pg 9, reporting on a study appearing March 22, 2010 in the
Proceedings of the National Academy of Science.Slide29
Multiple Sclerosis and Sunlight
For each situation below, write downNull and alternative hypotheses Informal description of the strength of evidence against H0Formal decision about H0, using α = 0.05Conclusion in the context of the questionIn testing whether UV light provides protection against MS (UV light vs control group), the p-value is 0.002.In testing whether Vitamin D provides protection against MS (Vitamin D vs control group), the p-value is 0.47.Slide30
Multiple Sclerosis and Sunlight
In testing whether UV light provides protection against MS (UV light vs control group), the p-value is 0.002.Slide31
Multiple Sclerosis and Sunlight
In testing whether Vitamin D provides protection against MS (Vitamin D vs control group), the p-value is 0.47.Slide32
Conclusions
p-value < αGeneric conclusion:Reject H0Conclusion in context:We have (strong?) evidence that [fill in alternative hypothesis]
p-value ≥ α
Generic conclusion
:
Do not reject
H
0
Conclusion in context:
We
do not have enough evidence to conclude that
[
fill in alternative
hypothesis
]
H
0
H
aSlide33
To Do
Read Section 4.3 (will cover errors next class)Do HW 4.3 (due Friday, 3/20)Enjoy spring break!!!