Topic 5 Mean separation Multiple comparisonsration Multiple compari - PDF document

Topic 5. Mean separation: Multiple comparisonsration: Multiple comparisons5. 1. Basic concepts If there are more than 2 treatments the problem is to determine which means are significantly different. This process can take 2 forms. Planned – Motivated by treatment structure Independent Limited to (t-1) comparisons (multiple comparison tests, Topic 5) No preliminary grouping is known onships exists among treatments. No limited number of comparisons Many methods/philosophies to choose from 5. 2. Error ratesSelection of the most appropriate mean separation test is heavily influenced rejects a null true HType I a Type I error is made. In a single comparison this is the value When comparing 3 or more treatment means there are at least 2 kinds of

This is the number of type I errors divided by the total number or comparisons EER This is the number of experiments in which occurs, divided by the total number of experiments 100 experiments5 treatments In each experiment, there are 10 Total possible pairwise comparisons 2)1( 1,000 possible pairwise comparisonsno true differences among the treatments riments one Type I error is made. = (100 mistakes) / (1000 comparisons) = 0.1 or 10%. = (100 experiments with mistakes) Relationship between has to be kept very low. Conversely, to e two Type I error ratios depends on the objectives of the study. experiment or when the consequetly rejecting a number of comparisons, experiment, the ng these two kinds of errors. is difficult to comp

ute because possible to compute an for the EER by assuming that the probability of a Type I error in any comparison is of the other comparisons. In this case: Upper bound EER1 - (1 - )p of pairs to compare = rror in one comparison = Pb of not making an error in p comparisons = 10* 9/ 2= 45 pairwise comparisons rror in one comparison= 0.95 Pb. of not making a Type 1 error in comparisons: Probability of making 1 or more Type 1 error in comparisons: 1- 0.1=0.9 This formula can be used to fix and then calculate the required : Suppose there are 10 treatments and one shows a making a Type I error between The experimenter will incorrectly conclude that some pair of similar effects are different 84% of the time. Treatment number2 4 6 8 10

x x x x x x x x x . Results (mg dry weight) of an experiment (CRD) to determine the effect of seed treatment by acids Treatment iY iY Control 4.23 4.38 4.1 3.99 4.25 20.95 4.19 HCl 3.85 3.78 3.91 3.94 3.86 19.34 3.87 Propionic 3.75 3.65 3.82 3.69 3.73 18.64 3.73 Butyric 3.66 3.67 3.62 3.54 3.71 18.2 3.64 Overall ..Y= 3.86 Table 4.2. ANOVA of data in Table 4.1. Source of Variation Sum of Squares Total 19 1.0113 Treatment 3 0.8738 0.2912 33.87 Exp. error 16 0.1376 Weight gains (lb/animal/day) as affected by three D with unequal replications. Treat. 1.21 1.19 1.17 1.231.291.146 7.23 1.20 1.34 1.41 1.38 1.291.361.421.371.328 10.89 1.36 1.45 1.45 1.51 1.391.445 7.24 1.45 1.31 1.32 1.28 1.351.411.271.377 9.31 1.33 Overall

26 34.67 1.33 Table 5-2. ANOVA of data in Table 5-1. Source of Variation Sum of Squares Mean Squares Total 25 0.2202 Treatment 3 0.17090.0569625.41 Exp. Error 22 0.0493 Complete and partial null hypothesis in SAS EER under the comple: all population means are equal : some means are equal but some subdivides the error rates into: = comparison-wise error rate = experiment-wise error rate under complete null hypothesis (the = experiment-wise error rate = maximum experiment-wise erropartial null hypothesis. ST&D Ch. 8 and SAS/STAT (GLM)more inferences while controlling the probability of making at leassimultaneous Multiple comparison techniques divide themselves into two groups: Fixed-range tests: Those which can provide confid

ence intervals and tests of hypothesis. One range for testing all differences in balanced 5. 3. 1. Fixed-range tests These tests provide e is most appropriate. We will present four commonly used procedures starting from the less conservative (high power) to the moTukey LSD test is one of the simplest and one of the most widely misusedLSD test declares the between means Y i and Y i' to be significant when: | Y i - Y � | t/2, MSE df = pooled swas calculated by PROC is coming from the XFrom 00086 Y i - Y � | 0.1243, the treatments are said to be significantly different. to arrange the means in descending Treatment Control 4.19 Propionic 3.73 First compare the largest with the smallest mean. If these two means are s

ignificantly different, then compare the next largest with the smallest. Identify these two and any means in between with a common lower case When all the treatments are equally LSD is readily used to construct confid confidence limits are = Y A - Y B LSD of replications Different LSD must be calculated for each comparison involving different numbers of replications. Y i - Y � | t/2, MSE df with unequal replications: Feed-C Feed-A 1.20 c 1.33 b 1.36 b 1.45 a The 5% LSD for comparing the (1.45-1.20=0.25) is, 000224.() Y i - Y among different pairs of mean comparisons A vs. Control = 0.0531 A vs. B = 0.0560 A vs. C = 0.0509 B vs. C = 0.0575 C vs. Control = 0.0546 The LSD test is much sa compared are selected of the exp

eriment. It is primarily intended for use when there is no predetermined structure to the treatments (e.g. in variety trials). The LSD test is the only test comparison wise error rate. This is often regarded as too liberal (i.e. to performing the overall ANOVA test at thecomparisons only if the F test is significant (Fisher's Protected LSD if there are more than three means . A preliminary but not the 8 Compare a with each of several other treatments Dunnett's test holds the maximum under any complete or partial null hypothesis (In this method a value is calculated for each comparison. /2, MSE df /2, MSE df From Table 4-1, MSE = 0.0086 with 16 df and p= 3, t* DLSD 0.025 = 2.59 000860.152= least significant difference betwee comp

uted as = Y o - Y DLSD. The limits of these differences are, Control - butyric = 0.32 ± 0.15 Control - HC1 = 0.46 ± 0.15 Control - propionic = 0.55 ± 0.15 simultaneously To compare 0.025, 22 , p=3= 2.517 (from SAS) 000224.() - Y = 0.125 is larger than 0.06627, it is significant. The other 9 Propionic5. 3. 1. 3. Tukey's w procedure Tukey's test was designed for all possible pairwise comparisonsThe test is sometimes called "honestly It controls the when the sample sizes are equal. It uses a statistic similar to the LSD but with a number ,(p, MSE df)obtained from ,(p, MSE df) for equal r ( is missing because is inside ,(p, MSE df) ()/(SAS manual) 0.05,(4, 16) w= 4.05 00086 of contrasts is larger (all pairs of means). Table

4.1 Treatment Control 4.19 Propionic 3.73 0.05,(4, 22) 000224.()/ - Y = 0.125 is larger than 0.0731, it is significant. As in the LSD the only pairwise comparison that is Y =1.33) and Feed-A ( Y =1.36). PropionicScheffe's test is compatible with the overall ANOVA in that it never declares a contrastScheffe's test pairwise comparisons. re kinds of comparisons, it is less comparison procedures. For pairwise comparisons with equal r, the Scheffe's critical difference SCD has a similar structure as that dfFdfdfTRTRMSE(,) 2 for equal r SCD = dfFdfdfTRTRMSE(,) MSE SCD 0.05= 3324 00086 Treatment Control 4.19 Propionic 3.73 d for each comparison. The contrast vs. Feed-C (Table 5-1), SCD 0.05, (3, 22) = 3305 000224.() - Y = 0.125

is larger than 0.0796, it is significant. Scheffe's procedure is also readily used for interval estimation. confidence limits are = Y A - Y SCD. The resulting intervals are simultaneous. The probability is at least 1- simultaneously true. The most important use of Scheffe's test is for arbitrary If we are interested only in testing means, Tukey is more sensitive than Scheffe. To make comparisons among i. withci0 (or rc 0 for unequal replication) We will reject the hypothesis (H= 0 if the absolute value of Q is larger than a critical vale S. This is the general form for Scheffe's F test: dfFdfdfTRTRMSE(,) In the previous pairwise comparisons the contrast is 1 vs. -1 If we want to compare the control vs. the average of the th4.1, th

e contrast coefficients are multiplied by the 0.05, (3, 16) .3*3 5/))1()1()1(3(0086.02222= 0.4479 , therefore we reject H. The control (4.190-mg) is significantly different from the average of the three acid treatments (3.745-mg). By giving up the facility for simultaneous estimation with one value, it is possible to obtain tests with greater power: Duncan Student-Newman-Keuls (SNK)studentized range statistic range tests: no longer accepted gnificance between means as they are more widely separated in the array. It controls the CER at the level but it has a high type I error rate Duncan's test used to be the most popular method but many journals no Largest mean p p p With means arrayed from the lowest to the highest, a multiple-

range test gives significant ranges that become smaller as the pairwise means to be compared are closer in the array most distant means and thenand smaller critical value in Table 8 5. 3. 2. 2. The StudenThis test is more conservative than Duncan's in that the type I error rate is It is often accepted by journals that do not accept Duncan's test. The SNK test behavior in terms of the The procedure is to compute a set of critical values (ST&D ,(p, MSE df) P 2 3 4 3.0 3.65 4.05 Note that for 0.124 0.151 0.168 = LSD Treatment p Control 4.19 Propionic 3.73 Assume the following (means from smallest to largest) pairs compared by LSD Remember that HCl vs. Propionic SNK is more sensitive than Tukey This test does not detect significan

t differences between HCl and Propionic. The price of the better 5. 3. 2. 3. The REGWQ method A variety of MSTs that control ME for The REGWQ method does the comparisons using a range test. This method appears to be among the most powerful multiple Assuming the sample means have been arranged in order from Y 1 , the homogeneity of means Y i., ..., Y , is rejected by REGWQ if: Y i q(p; p, dfMSE (Use ST&D) P 2 3 4 0.050.05 3.493.654.05Critical value 0.1450.1510.168 � SNK =S p / t Note that the difference between HCl and propionic is significant with SNK but no significant with REGWQ (3.87 - 3.73 ) Treatment 5 Control 4.19 Propionic 3.73 5. 4. Conclusions and recommendations and multivariate no consensusmost

appropriate procedure to recommend to all users. One main difficulty in comparing versus comparison-wise. The difference in performance of any choice between a comparison-wise error ratend an experiment-wise error rateScheffe's test). very general technique to test all possible comparisons among means. For just pairwise comparisons, Scheffe's Tukey’s 's test should be used if the comparisons between each of several treatments and a manual makes the following additional recommendations: for Tukey and C each have 11. No significant Treatment Data Mean A 10, 17 13.5 B 10, 11, 12, 12, 13, 14, 15, 16, 16, 17, 18 14.0 C 14, 15, 16, 16, 17, 18, 19, 20, 20, 21, 22 18.0 D 16,21 18.5