collinearity Collinearity between independent variables High r 2 High vif of variables in model Variables significant in simple regression but not in multiple regression Variables not significant in multiple regression but multiple regression model as whole significant ID: 1027390
Download Presentation The PPT/PDF document "Collinearity Symptoms of" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. Collinearity
2. Symptoms of collinearityCollinearity between independent variables High r2High vif of variables in modelVariables significant in simple regression, but not in multiple regressionVariables not significant in multiple regression, but multiple regression model (as whole) significantLarge changes in coefficient estimates between full and reduced modelsLarge standard errors in multiple regression models despite high power
3. Collinearity and confounding independent variablesTwo independent variables, correlated with each other, where both influence the response
4. MethodsTruth: y = 10 + 3x1 + 3x2 + N(0,2)x1 = U[0,10]x2 = x1 + N(0,z) wherez = U[0.5,20]Run simple regression between y and x1Run multiple regression between y and x1 + x2No interactions!
5. Simple regression: y~x1
6. Simple regression: y~x1
7. Simple regression: y~x1
8. Simple regression: y~x1
9. Multiple regression: y~x1+x2
10. Multiple regression: y~x1+x2
11. Multiple regression: y~x1+x2
12. Collinearity and redundant independent variablesTwo independent variables, correlated with each other, where only one influences the response, although we don’t know which one
13. MethodsTruth: y = 10 + 3x1 + N(0,2)x1 = U[0,10]x2 = x1 + N(0,z) wherez = U[0.5,20]Run simple regression between y and x1Run multiple regression between y and x1 + x2No interactions!
14. Simple regression: y~x1
15. Simple regression: y~x1
16. Simple regression: y~x1
17. Simple regression: y~x2
18. Simple regression: y~x2
19. Simple regression: y~x2
20. Multiple regression: y~x1+x2
21. Multiple regression: y~x1+x2
22. Multiple regression: y~x1+x2
23. Multiple regression: y~x1+x2
24. Multiple regression: y~x1+x2
25. Multiple regression: y~x1+x2
26. What to do?Be sure to calculate collinearity and vif among independent variables (before you start your analysis)Pay attention to how coefficient estimates and variable significance change as variables are removed or addedBe careful to identify potentially confounding variables prior to data collection
27. Is a variable redundant or confounding?Think!Extreme collinearityRedundantLarge changes in coefficient estimates of both variables between full and reduced modelsConfoundingLarge changes in coefficient estimates of one variable between full and reduced modelsRedundant – full model estimate close to zeroUncertain – assume confoundingMultiple regression always produces unbiased estimates (on average) regardless of type of collinearity
28. What to do? Confounding variablesBe sure to sample in a manner that eliminates collinearityCollinearity may be due to real collinearity or sampling artifactUse multiple regressionMay have large standard errors if strong collinearityInclude confounding variables even if non-significantGet more dataDecreases standard errors (vif)
29. What to do? Redundant variablesDetermine which variable explains response best using P-values from regression and changes in coefficient estimates with variable addition and removalDo not include redundant variable in final modelReduces vifTry a variable reduction technique like PCA