PPT7 Variable Screening Techniques0 - McGill University...

McGill University
Advanced Business Statistics
MGSC-272

Variable Screening Methods
Stepwise Regression
All-Possible-Regressions (Best subsets)

Stepwise Regression
Variables are entered into the model in order of
significance (based on  -values or F-values) until the
new value being added is not significant.
If introducing a new variable causes an existing
variable to become non-significant it is dropped from
the model.

Data
Y X
1
X
2
X
3
X
4
STORE PROFIT ADVERT SPECIAL SIZE PLACE
1 9.4 3 1 30 1
2 10.3 3 5 37 1
3 10.9 4 5 38 1
4 9.9 4 2 35 1
5 12.9 5 6 40 0
6 11.8 5 6 40 0
7 11.5 6 2 39 1
8 13.2 6 5 45 0
9 12.8 7 5 41 0
10 12.1 7 1 41 0

Correlation Matrix
Profit Advert Special Size Place
Profit 1
Advert .8056 1
Special .5037 0 1
Size .9145 .7577 .5014 1
Place -.8604 -.7071 -.4126 -.7318 1
First Second

Minitab’s Stepwise Regression

Minitab’s Methods Screen

Minitab: Stepwise Regession output

Minitab’s Stepwise Regression Results
Minitab selected Model 6
0 3 3 4 4
3 4
ˆ
4.231 .201 1.033
y x x
Y x x
β β β ε = + + +
= + -
This model explains Profit ( Y ) in terms of variables
Size ( x
3
) and Place ( x
4
)

Forcing a variable into the model
Suppose that the manager in charge of advertising
budgets insists that she needs a model that contains
the variable advert ( x
1
).
In the next slide we show how to force the variable
advert into the model.

Forcing a variable into the model

Stepwise Regression with Advert in Model

Stepwise Regression with X
1
= advert
forced into the model
Stepwise regression has selected Model 2
0 1 1 2 2
1 2
ˆ
6.666 .715 .326
y x x
Y x x
β β β ε = + + +
= + +
Advert ( x
1
) and Special ( x
2
) are the selected variables

Summary
We see that Stepwise regression can help the
analyst find a suitable model with important variables
included as required for a specific application.
Once we have decided upon the
independent variables to include in a model
it is important to check for possible
interaction among the variables.

R
2
adj
= 89.1%

No interaction between Size and Place
p -value > .05
R
2
adj
= 87.4%

All-Possible Regressions
Best Subsets