Re: Stepwise versus Enter method in regression

Posted by Rich Ulrich on Feb 22, 2014; 8:01pm
URL: http://spssx-discussion.165.s1.nabble.com/Stepwise-versus-Enter-method-in-regression-tp5724452p5724600.html

I see what you say.  I think that I would draw a distinction
between "making good use of chance" -- which is what we
do "in the trade" -- and "taking unfair advantage of chance",
which is how I have used the phrase.  Your final paragraph
describes the practical result.

Maybe I should stop using the phrase.  When I think of
other contexts, I come up with "capitalizing on the blunder",
which I remember from chess commentary; and similar
uses in various sports.  "Over-capitalize on chance"?

--
Rich Ulrich

> Date: Sat, 22 Feb 2014 13:59:45 -0500

> From: [hidden email]
> Subject: Re: Stepwise versus Enter method in regression
> To: [hidden email]
>
> At 03:15 PM 2/11/2014, Rich Ulrich wrote:
>
> >The general point, [about preferring specifying a regression model
> >to using stepwise variable selection], is that using intelligence
> >and intention is far better than using any method that capitalizes on chance.
>
> I'd have put it a little differently -- I'm not sure whether this is
> saying the same thing in different words, or something different.
>
> First off, in our trade, capitalizing on chance is often better than
> using intelligence and intention; hence, the emphasis on random
> selection of samples, or random assignment of subjects to treatment groups.
>
> I'd have said that the problems with stepwise selection are twofold,
> and mutually reinforcing: first, that by exploring an effectively
> very large set of models, it loses statistical power; and second,
> that by reporting the final model as if it were a chosen model, it
> conceals that loss of power and reports significance levels, and
> confidence intervals, that are simply too strong. (In that, it's
> closely analogous to making many comparisons and reporting only
> those that show significance; in fact, it's more or less a special
> case of that.)
>
> Concretely: Given a dependent variable, and a large number of
> 'independents' drawn with a random-number generator, stepwise
> selection could well produce a model that looks pretty good, though
> it has nothing to do with the data at all.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD