Solutions to the <a href="/keyword/mountain-states-potato-company/" >mountain states potato company</a> Case 101 The <a href="/keyword/mountain-states-potato-company/" >mountain states potato company</a> The independent variables involved are the pH of the filter cake (PH), the pressure in the vacuum line below the fluid line on the rotating drum (LOWER), the pressure of the vacuum line above the fluid line on the rotating drum (UPPER), cake thickness measured on the drum (THICK), setting used to control the drum speed (VARIDRIV), and the speed at which the drum was rotated when collecting filter cake (DRUMSPD). These data are contained in the POTATO file. We begin our analysis of the potato processing data by first measuring the amount of collinearity that exists between the explanatory variables through the use of the variance inflationary factor. The following figure represents partial PHStat output for a multiple linear regression model in which the percent of solids is predicted from the six explanatory variables. We observe that four of the VIF are above 5.0, ranging from 9.9 for Varidriv to 8.4 for Upper. Thus, based on the criteria developed by Snee, there is evidence of collinearity among at least some of the explanatory variables. A reasonable strategy is to remove the independent variable with the largest VIF above 5, and determine what effect this has on the VIF of the remaining independent variables. Regression Analysis PH and all other X <a href="/keyword/regression-statistics-multiple/" >regression statistics multiple</a> R R Square Adjusted R Square Standard Error Observations VIF Regression Analysis Lower Pressure and all other X <a href="/keyword/regression-statistics-multiple/" >regression statistics multiple</a> R 0.939267 R Square 0.882222 Adjusted R Square 0.869954 Standard Error 0.716466 Observations 54 VIF 8.490574 Regression Analysis Cake Thickness and all other X <a href="/keyword/regression-statistics-multiple/" >regression statistics multiple</a> R 0.616598 R Square 0.380193 Adjusted R Square 0.31563 Standard Error 0.108131 Observations 54 VIF 1.613407 Regression Analysis Drum speed setting and all other X <a href="/keyword/regression-statistics-multiple/" >regression statistics multiple</a> R 0.946259 R Square 0.895406 Adjusted R Square 0.884511 Standard Error 2.150324 Observations 54 VIF 9.560793 0.561502 0.315284 0.243959 0.232255 54 1.460459 Regression Analysis Upper Pressure and all other X <a href="/keyword/regression-statistics-multiple/" >regression statistics multiple</a> R 0.93823 R Square 0.880276 Adjusted R Square 0.867805 Standard Error 0.766658 Observations 54 VIF 8.352558 Regression Analysis Varidriv speed and all other X <a href="/keyword/regression-statistics-multiple/" >regression statistics multiple</a> R 0.948243 R Square 0.899165 Adjusted R Square 0.888661 Standard Error 0.179475 Observations 54 VIF 9.917201 Figure 1 Regression Model obtained from PHStat to Predict Percent of Solids based on Six Explanatory Variables 102 Solutions to the <a href="/keyword/mountain-states-potato-company/" >mountain states potato company</a> Case The following figure represents the regression model obtained from PHStat with only the Varidriv variable removed from the model. Regression Analysis PH and all other X <a href="/keyword/regression-statistics-multiple/" >regression statistics multiple</a> R 0.527979 R Square 0.278761 Adjusted R Square 0.219885 Standard Error 0.235924 Observations 54 VIF 1.386504 Regression Analysis Upper Pressure and all other X <a href="/keyword/regression-statistics-multiple/" >regression statistics multiple</a> R 0.937879 R Square 0.879617 Adjusted R Square 0.869789 Standard Error 0.760882 Observations 54 VIF 8.306799 Regression Analysis Drum speed setting and all other X <a href="/keyword/regression-statistics-multiple/" >regression statistics multiple</a> R 0.330551 R Square 0.109264 Adjusted R Square 0.036551 Standard Error 6.210805 Observations 54 VIF 1.122667 Regression Analysis Lower Pressure and all other X <a href="/keyword/regression-statistics-multiple/" >regression statistics multiple</a> R 0.938614 R Square 0.880996 Adjusted R Square 0.871281 Standard Error 0.7128 Observations 54 VIF 8.403063 Regression Analysis Cake Thickness and all other X <a href="/keyword/regression-statistics-multiple/" >regression statistics multiple</a> R 0.614269 R Square 0.377327 Adjusted R Square 0.326496 Standard Error 0.10727 Observations 54 VIF 1.605978 Figure 2 Regression Model obtained from PHStat to Predict Percent of Solids based on Five Explanatory Variables excluding the Varidriv independent variable From the figure above, we see that the VIF for the Drumspeed independent variable has been reduced from 9.6 to 1.1. This indicates that Varidriv and Drumspeed were very correlated with each other, but uncorrelated with the other independent variables. However, we also observe that the VIF values for Lower and Upper are still above 5, being equal to 8.4 and 8.3 respectively. Using the criteria of removing the independent variable with the highest VIF above five, we can remove the Lower independent variable from the model. The following figure represents PHStat output for a model that has excluded the Lower and Varidriv independent variables. Solutions to the <a href="/keyword/mountain-states-potato-company/" >mountain states potato company</a> Case 103 Regression Analysis PH and all other X <a href="/keyword/regression-statistics-multiple/" >regression statistics multiple</a> R 0.516068032 R Square 0.266326213 Adjusted 0.222305786 R Square Standard 0.235557799 Error Observati 54 ons VIF 1.363003583 Regression Analysis Cake Thickness and all other X <a href="/keyword/regression-statistics-multiple/" >regression statistics multiple</a> R 0.603839337 R Square 0.364621944 Adjusted 0.326499261 R Square Standard 0.10726935 Error Observati 54 ons VIF 1.573866128 Regression Analysis Upper Pressure and all other X <a href="/keyword/regression-statistics-multiple/" >regression statistics multiple</a> R 0.530366 R Square 0.281288 Adjusted R Square 0.238165 Standard Error Observations VIF 1.840452 54 1.391378 Regression Analysis Drum speed setting and all other X <a href="/keyword/regression-statistics-multiple/" >regression statistics multiple</a> R 0.29969 R Square 0.089814 Adjusted R Square 0.035203 Standard Error Observations VIF 6.215147 54 1.098677 Figure 3 Regression Model obtained from PHStat to Predict Percent of Solids based on Four Explanatory Variables excluding the Lower and Varidriv independent variables From the figure above, we see that none of the remaining four independent variables has a VIF value above 1.6. The Lower independent variable was undoubtedly highly correlated with the Upper independent variable and its removal left us with four relatively uncorrelated independent variables, pH, Upper, Thick, and Drumspeed. The Stepwise Regression Approach to Model Building We now continue our analysis of these data by attempting to determine the explanatory variables that might be deleted from the complete model. We shall first utilize stepwise regression. 104 Solutions to the <a href="/keyword/mountain-states-potato-company/" >mountain states potato company</a> Case The figure below represents a partial output obtained from the PHStat add-in for Microsoft Excel for the potato processing data. Stepwise Analysis Table of Results for General Stepwise PH entered. df Regression Residual Total Intercept PH 1 52 53 Coefficients 0.782076396 2.589618022 SS MS 25.35907426 25.35907426 74.61796278 1.434960823 99.97703704 Standard Error t Stat 2.45805091 0.318169324 0.616011802 4.203844815 F Significance F 17.67231123 0.000103538 P-value Lower 95% 0.751630817 -4.150360268 0.000103538 1.353500744 Upper 95% 5.714513059 3.825735299 Upper Pressure entered. df Regression Residual Total Intercept PH Upper Pressure 2 51 53 Coefficients 3.839576804 2.834310878 -0.263257541 SS MS 41.46414438 20.73207219 58.51289266 1.147311621 99.97703704 Standard Error t Stat 2.344527788 1.637675963 0.554678372 5.109827637 0.070265187 -3.74662834 F Significance F 18.07013179 1.16804E-06 P-value Lower 95% 0.107645513 -0.86725551 4.87591E-06 1.720748436 0.00045751 -0.404320681 Upper 95% 8.546409118 3.947873319 -0.1221944 No other variables could be entered into the model. Stepwise ends. Figure 4 Stepwise regression output obtained from the PHStat add-in for Microsoft Excel for the potato processing data For this example, a significance level of .05 was utilized either to enter a variable into the model or to delete a variable from the model. The first variable entered into the model is pH. Since the p value of .0001 is less than .05, pH is included in the regression model. The next step involves the evaluation of the second variable to be included in this model. The variable to be chosen is the one will make the largest contribution to the model, given that the first explanatory variable has already been selected. For this model, the second variable is Upper pressure. Since the p value of .00046 for Upper pressure is less than .05, Upper pressure is included in the regression model. Now that Upper pressure has been entered into the model, we determine whether pH is still an important contributing variable or whether it may be eliminated from the model. Since the p value of .000004876 (4.87591E-06 in scientific notation) for pH is also less than .05, pH should remain in the regression model. The next step involves the determination of whether any of the remaining variables should be added to the model. Since none of the other variables meet the .05 criterion for entry into the model, the stepwise procedure terminates with a model that includes pH and Upper pressure. Solutions to the <a href="/keyword/mountain-states-potato-company/" >mountain states potato company</a> Case 105 The Best Subset Approach to Model Building The best subset approach evaluates either all possible regression models for a given set of independent variables or at least the best subset of models for a given number of independent variables. The figure below represents partial output obtained from the PHStat add-in for Microsoft Excel in which all regression models for a given number of parameters were evaluated according to two widely used criteria, the adjusted r 2 and the Cp statistic. Best Subsets Analysis Intermediate Calculations R2T 0.428728 1 - R2T 0.571272 N 54 T 5 n-T 49 Model X1 X1X2 X1X2X3 X1X2X3X4 X1X2X4 X1X3 X1X3X4 X1X4 X2 X2X3 X2X3X4 X2X4 X3 X3X4 X4 Cp 14.0171 2.200053 3.038685 5 4.136941 15.01919 16.79688 15.80859 25.90084 25.97922 27.03666 26.46649 28.92168 30.54398 34.92666 k 2 3 4 5 4 3 4 3 2 3 4 3 2 3 2 R Square Adj. R Square 0.253649 0.239296084 0.414737 0.391785177 0.428277 0.393973229 0.428728 0.382093161 0.415472 0.380400828 0.265283 0.236470785 0.267875 0.223947575 0.25608 0.226906562 0.115101 0.098083637 0.137504 0.103681097 0.148493 0.097402952 0.131823 0.097777342 0.079882 0.062187562 0.084286 0.048375212 0.009872 -0.009168586 Consider Std. Error This Model? No 1.197899 Yes 1.071126 Yes 1.069198 Yes 1.079627 No 1.081104 No 1.200121 No 1.209923 No 1.207614 No 1.304354 No 1.3003 No 1.304846 No 1.304575 No 1.330057 No 1.339816 No 1.37973 Figure 5 Best subsets regression output obtained from the PHStat add-in for Microsoft Excel for the potato processing data The first criterion that is often used is the adjusted r 2, which adjusts the r2 of each model to account for the number of variables in the model. Since models with different numbers of independent variables are to be compared, the adjusted r2 is the appropriate criterion here rather than r2. Referring to the figure above, we observe that the adjusted r2 reaches a maximum value of .39397 for the model that includes the independent variables pH, Upper, and Thick plus the intercept term (for a total of four terms). We note that the model selected by using stepwise regression, that includes pH and Upper has an adjusted r2 of .39179. Thus, the best subset approach, unlike stepwise regression, has provided us with several alternative models to evaluate in greater depth using other criteria such as parsimony, interpretability, and departure from model assumptions (as evaluated by residual analysis). 106 Solutions to the <a href="/keyword/mountain-states-potato-company/" >mountain states potato company</a> Case A second criterion often used in the evaluation of competing models is based on the Cp statistic developed by Mallows. When a regression model with p independent variables contains only random differences from a true model, the average value of Cp is p + 1, the number of parameters. Thus, in evaluating many alternative regression models our goal is to find models whose Cp is close to or below p + 1. From the previous figure, we observe that there are three models that contains a Cp value equal to or below p + 1. These are the models with X1 and X2, with X1, X2, and X3, and with X1, X2, X3, and X4. Since the models with X1 and X2 and with X1, X2, and X3 have fewer variables and also have Cp less than p + 1, we will focus on these two models. One approach for choosing between models that meet the criteria of Cp less than p + 1 is to determine whether the models contain a subset of variables that are common, and then test whether the contribution of the additional variables is significant. In this case, that would mean testing whether variable X3 made a significant contribution to the regression model given that variables X1 and X2 were already included in the model. If the contribution was statistically significant, then variable X3 would be included in the regression model. If variable X3 did not make a statistically significant contribution, variable X3 would not be included in the model. The following figure represents a regression model that includes variables X1, X2, and X3 (pH, Upper, and Thick). Regression Analysis <a href="/keyword/regression-statistics-multiple/" >regression statistics multiple</a> R 0.654428477 R Square 0.428276632 Adjusted R 0.393973229 Square Standard Error 1.069197909 Observations 54 ANOVA df SS Regression 3 42.81782866 Residual 50 57.15920838 Total 53 99.97703704 Coefficients Standard Error Intercept 2.625587479 2.592611079 PH 3.148146196 0.624290082 Upper Pressure -0.309346256 0.081934686 Cake Thickness 1.531919658 1.407781964 MS F Significance F 14.27260955 12.48496083 3.26679E-06 1.143184168 t Stat 1.012719378 5.042761826 -3.775522569 1.088179631 P-value Lower 95% Upper 95% 0.316070006 -2.581827253 7.833002211 6.41161E-06 1.89422215 4.402070241 0.000424913 -0.473916983 -0.144775529 0.281733384 -1.295694788 4.359534103 Figure 6 Regression Model obtained from PHStat to Predict Percent of Solids based on Three Explanatory Variables including the pH, Upper, and Thick independent variables From this figure, we observe that Thick (X3) has a t value of 1.09 and a p-value of .282. Since the p-value of .282 &gt; .05, we can conclude that Thick (X3) does not make a significant contribution to the regression model given that pH (X1 )and Upper pressure (X2) are included. Therefore, a reasonable approach is to eliminate Thick (X3) from the model and fit the regression model that includes pH (X1 )and Upper pressure (X2). The following figure represents PHStat output for this model. Solutions to the <a href="/keyword/mountain-states-potato-company/" >mountain states potato company</a> Case 107 Regression Analysis <a href="/keyword/regression-statistics-multiple/" >regression statistics multiple</a> R 0.644000528 R Square 0.41473668 Adjusted R 0.391785177 Square Standard Error 1.071126333 Observations 54 ANOVA df Regression Residual Total 2 51 53 SS MS F Significance F 41.46414438 20.73207219 18.07013179 1.16804E-06 58.51289266 1.147311621 99.97703704 Intercept PH Upper Pressure Coefficients Standard Error t Stat P-value Lower 95% Upper 95% 3.839576804 2.344527788 1.637675963 0.107645513 -0.86725551 8.546409118 2.834310878 0.554678372 5.109827637 4.87591E-06 1.720748436 3.947873319 -0.263257541 0.070265187 -3.74662834 0.00045751 -0.404320681 -0.1221944 Figure 7 Regression Model obtained from PHStat to Predict Percent of Solids based on Two Explanatory Variables including the pH, and Upper independent variables The following residual plots do not suggest any need for non-linear transformation. The Durbin-Watson statistic of 1.5509 is greater than dU 1.47 at 10% level of significance. So there is not sufficient evidence to conclude that there is negative autocorrelation in the model. Thus, we can conclude that raising the pH and/or reducing the Upper pressure should result in an increased percentage of solids. Upper Pressure Residual Plot Residuals 10 5 0 -5 0 5 10 Upper Pressure 15 20 PH Residual Plot Residuals 10 5 0 -5 0 1 2 PH 3 4 5