Posted by
Mike on
Jan 14, 2016; 4:00am
URL: http://spssx-discussion.165.s1.nabble.com/Confusing-SPSS-outputs-in-Linear-Regression-Analysis-tp5731271p5731275.html
Here's another way to understand what Greenberg is saying:
(1) The overall F test is an omnibus test that in the case of
multiple regression is testing whether the multiple R is significantly
different from zero. A non-significant R in this situation implies
that there is not correlation between the dependent/outcome
variable and the predictors (actually, it can also be interpreted
as the Pearson r between the actual values of Y and the predicted
values of Y (Y-hat)).
(2) You have 7 predictors in your equation, each can be evaluated
for significance (either the slope b is not equal to zero or the
increase in R^2 produced by the predictor is not greater than zero).
Each predictor is evaluated with a per comparison alpha (or
alpha-pc) p= .05. With 7 predictors, you have 7 tests done
each at alpha-pc. But the problem with multiple testing like
this is that we have to remember that there is an overall Type I
error rate or alpha-overall, which represents the probability
of falsely rejecting a true null hypothesis (in this case, correlations
are all equal to zero) after doing 7 tests.
(3) The formula for alpha-overall = 1 (1 - alpha-pc)^k
where k is the number of tests being done -- in this case
k = 7 (the ^7 means raised to the power of 7).. If the
alpha-pc = 0.05, then alpha-overall is
alpha-overall = 1 - (1-.05)^7 = 1 - (.95)^7 = 1 - (0.6983)
alpha-overall = 0.3017
In words, after 7 tests there is a 30% chance that one
has committed a Type I error. This is often considered
to be unacceptably high and people will tend to set
alpha-overall = 0.05 which implies that each alpha-pc
has to be reduced. One method is to do the following:
"corrected" alpha-pc = alpha-overall/k = .05/7 = 0.007.
Now, compare the p-value of each predictor in the equation
and see if it less than 0.007. It likely that none will be.
(4) The Bonferroni correction is the reduction of the Type I
error rate or alpha used with a group of tests. The omnibus F
of the regression only tells you that there either is significant
relationship between the dependent/criterion variable and
the independent/predictors or not. In the case of the multiple
regression it does not tell you which predictor is involved
in the relationship which is why you have to do additional
testing (this is a two-stage testing process). As the number
of predictors used increases, the probability that one or
more of them will statistically significant by chance (Type I
errors) increases and this is what one want to guard against.
I hope I was clear.
-Mike Palij
New York University
[hidden email]
----- Original Message -----
From: "David Greenberg" <
[hidden email]>
To: <
[hidden email]>
Sent: Wednesday, January 13, 2016 9:41 PM
Subject: Re: Confusing SPSS outputs in Linear Regression Analysis
> You are totally mistaken. The point is not to do the correction on
> the overall regression. That needs no correction. But you are doing 7
> tests on the coefficients. Imagine a world in which, in the
> population, all those coefficients are zero. If you use a nominal
> alpha of .05 the probability of getting any one estimate significant
> by chance is 1 in 20, but with 7 tests, the probability of 2 in 7 is
> elevated. It is quite a bit higher than .05. David Greenberg
>
> On Wed, Jan 13, 2016 at 9:23 PM, E. Bernardo
> <
[hidden email]> wrote:
>> Dear David,
>>
>> All the seven predictors were entered together into a multiple
>> regression
>> model (using ENTER method). The overall F was nonsignificant at the
>> same
>> time two of the seven predictors were significant (p<.05). Bonferroni
>> correction is out of context in this discussion because all
>> predictors were
>> entered into the model simultaneously. That is, only one multiple
>> regression
>> was analyzed.
>>
>> Thank you.
>> E.
>>
>>
>> On Thursday, January 14, 2016 9:54 AM, David Greenberg <
[hidden email]>
>> wrote:
>>
>>
>> The overall F not being significant should tell you to stop there.
>> With seven individual predictors each being tested individually, you
>> are multiplying the chances of obtaining 2 significant t tests by
>> chance. In other words, you think you are testing at alpha = .05, but
>> actually are testing with a larger value of alpha. Many researchers
>> correct for this by doing a Bonferroni correction.. Chances are that
>> your significant findings will not be significant once that is done.
>> David Greenberg, Sociology Department, New York U.
>>
>> On Wed, Jan 13, 2016 at 8:45 PM, E. Bernardo
>> <
[hidden email]>
>> wrote:
>>> Dear members,
>>>
>>> My linear regression analysis has seven binary predictors, n=47, and
>>> (of
>>> course) a continuous dependent. The overall regression anova is
>>> nonsignificant (F=1.489, p = .200). The confusing is that two out of
>>> the
>>> seven predictors are significant (p<.05). I dont think there is
>>> multicollinearity problem because the collinearity diagnostics
>>> statistics
>>> seem look fine. For example, no beta coefficients of predictors
>>> greater
>>> than
>>> 1.0; Tolerance of the predictors range between .559 and .814; VIF of
>>> predictors range from 1.224 and 1.669; correlation coefficients
>>> among
>>> predictors are between .009 and .757 but most are below .30.
>>>
>>> Any comments are welcome.
>>>
>>> Thank you.
>>> E.
>>> ===================== To manage your subscription to SPSSX-L, send a
>>> message
>>
>>> to
>>
[hidden email] (not to SPSSX-L), with no body text except
>> the
>>> command. To leave the list, send the command SIGNOFF SPSSX-L For a
>>> list of
>>> commands to manage subscriptions, send the command INFO REFCARD
>>
>> =====================
>> To manage your subscription to SPSSX-L, send a message to
>>
[hidden email] (not to SPSSX-L), with no body text except
>> the
>> command. To leave the list, send the command
>> SIGNOFF SPSSX-L
>> For a list of commands to manage subscriptions, send the command
>> INFO REFCARD
>>
>>
>>
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
>
[hidden email] (not to SPSSX-L), with no body text except
> the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD