Stepwise versus Enter method in regression

[Norberto Hernandez]

Hi everybody, I am looking for a recommendation, I heard that is better to use Enter besides Stepwise method in regression, but I couldn't find any useful reference about it. 

Could you please help me!

Norberto

[Bruce Weaver]

Here's one article you may find useful.  

  http://people.duke.edu/~mababyak/papers/babyakregression.pdf

Norberto Hernandez wrote

Hi everybody, I am looking for a recommendation, I heard that is better to
use Enter besides Stepwise method in regression, but I couldn't find any
useful reference about it.

Could you please help me!
Norberto

[Rich Ulrich]

Google < stepwise regression ulrich>

The Stata site, among others, shows the comments of Frank
Harrell that I collected and started promoting in 1996,
along with references.

The general point is that using intelligence and intention
is far better than using any method that capitalizes on chance.

--
Rich Ulrich

________________________________

> Date: Tue, 11 Feb 2014 13:45:14 -0600
> From: [hidden email]
> Subject: Stepwise versus Enter method in regression
> To: [hidden email]
>
> Hi everybody, I am looking for a recommendation, I heard that is better
> to use Enter besides Stepwise method in regression, but I couldn't find
> any useful reference about it.
>
> Could you please help me!
> Norberto

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Salbod]

Dale Berger's Introduction to Regression Analysis addresses the problem. But, more important are the references. I would check out Cohen et al to get a good understanding of the Stepwise Curse.

Regression.doc

I hope this helps, Steve

[Yang, Hongwei]

My understanding is that these days people do not like the stepwise method any more in applied research.

 

Instead, the preferred approach seems to be a hierarchical approach to building regression models. This approach is certainly based on the Enter method, instead of the Stepwise method. The Enter method is used each time a candidate in a hierarchy of models is fitted.

 

In addition, there is also a recommendation in the literature on the use of the all-possible-subsets method (rather than stepwise method) to evaluate all possible models from the model space. Then, the top models from the all-possible-subsets method is further evaluated after taking into account theoretical considerations. In SPSS, the all-possible-subsets method is available under the Automatic Linear Modeling platform, not under the traditional Linear Regression platform.

 

Hongwei Yang, University of Kentucky

 

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Norberto Hernandez
Sent: Tuesday, February 11, 2014 2:45 PM
To: [hidden email]
Subject: Stepwise versus Enter method in regression

 

Hi everybody, I am looking for a recommendation, I heard that is better to use Enter besides Stepwise method in regression, but I couldn't find any useful reference about it. 

 

Could you please help me!

Norberto

[Richard Ristow]

At 03:15 PM 2/11/2014, Rich Ulrich wrote:

>The general point, [about preferring specifying a regression model
>to using stepwise variable selection], is that using intelligence
>and intention is far better than using any method that capitalizes on chance.

I'd have put it a little differently -- I'm not sure whether this is
saying the same thing in different words, or something different.

First off, in our trade, capitalizing on chance is often better than
using intelligence and intention; hence, the emphasis on random
selection of samples, or random assignment of subjects to treatment groups.

I'd have said that the problems with stepwise selection are twofold,
and mutually reinforcing: first, that by exploring an effectively
very large set of models, it loses statistical power; and second,
that by reporting the final model as if it were a chosen model, it
conceals that loss of power and reports significance levels, and
confidence intervals, that are simply too strong. (In that, it's
closely analogous to  making many comparisons and reporting only
those that show significance; in fact, it's more or less a special
case of that.)

Concretely: Given a dependent variable, and a large number of
'independents' drawn with a random-number generator, stepwise
selection could well produce a model that looks pretty good, though
it has nothing to do with the data at all.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Rich Ulrich]

I see what you say.  I think that I would draw a distinction
between "making good use of chance" -- which is what we
do "in the trade" -- and "taking unfair advantage of chance",
which is how I have used the phrase.  Your final paragraph
describes the practical result.

Maybe I should stop using the phrase.  When I think of
other contexts, I come up with "capitalizing on the blunder",
which I remember from chess commentary; and similar
uses in various sports.  "Over-capitalize on chance"?

--
Rich Ulrich

> Date: Sat, 22 Feb 2014 13:59:45 -0500

> From: [hidden email]
> Subject: Re: Stepwise versus Enter method in regression
> To: [hidden email]
>
> At 03:15 PM 2/11/2014, Rich Ulrich wrote:
>
> >The general point, [about preferring specifying a regression model
> >to using stepwise variable selection], is that using intelligence
> >and intention is far better than using any method that capitalizes on chance.
>
> I'd have put it a little differently -- I'm not sure whether this is
> saying the same thing in different words, or something different.
>
> First off, in our trade, capitalizing on chance is often better than
> using intelligence and intention; hence, the emphasis on random
> selection of samples, or random assignment of subjects to treatment groups.
>
> I'd have said that the problems with stepwise selection are twofold,
> and mutually reinforcing: first, that by exploring an effectively
> very large set of models, it loses statistical power; and second,
> that by reporting the final model as if it were a chosen model, it
> conceals that loss of power and reports significance levels, and
> confidence intervals, that are simply too strong. (In that, it's
> closely analogous to making many comparisons and reporting only
> those that show significance; in fact, it's more or less a special
> case of that.)
>
> Concretely: Given a dependent variable, and a large number of
> 'independents' drawn with a random-number generator, stepwise
> selection could well produce a model that looks pretty good, though
> it has nothing to do with the data at all.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD