multiple regression

[Siavoush Dastmalchi]

Dear list,

I am doing rather unusual MLR using spss and am using Enter method to use all of the independent variables. However, two of the independent variables are excluded from the regression. The output says "Tolerance = .000 limits reached". I have did some changes to F value or its probability, but never get to the point to have all variables selected in the MLR.
Now the question is that how one can force spss to use all of the variables?
Many thanks in advance for your kind help.

Cheers, Siavoush

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Art Kendall]

For some reason, additional predictors do not increase the fit.
do you have a large enough ratio of variables to cases?
a guess.
some subsets of variables are linear combinations of others. are
measuring the same
first check the list of independent variables to be sure that you do not
have the same variable in twice, e.g., a variable called sex, and a
variable called gender.
If you have items for a scale you should not also have the scale score.
If you have dummies to represent levels of a nominal level variable,
there should be one less dummy variable than there are categories of the
group.

If all that fails, use RELIABLILITY pretending your independent
variables are items on a scale.
 be sure to include
 /statistics = descriptives scale correlations
 /summary = total

The descriptives will help you find variables that have no or almost no
variance.
If you look at the pairwise zero order correlations, you can see if some
approach 1.00.
If you look at the SMC (squared multiple correlations) you can see if
any of these approach 1.00.

See if CATREG gives you different messages.

Are you doing the regression for predictive or exploratory or
confirmatory purposes?

If you still have not resolved the problem, please post your DISPLAY
with the variable definitions in the analysis and the syntax you are using.
Then give us a description of what you are trying to do and a rationale
for including _all_ these independents.


Art Kendall
Social Research Consultants

Siavoush Dastmalchi wrote:

> Dear list,
>
> I am doing rather unusual MLR using spss and am using Enter method to use all of the independent variables. However, two of the independent variables are excluded from the regression. The output says "Tolerance = .000 limits reached". I have did some changes to F value or its probability, but never get to the point to have all variables selected in the MLR.
> Now the question is that how one can force spss to use all of the variables?
> Many thanks in advance for your kind help.
>
> Cheers, Siavoush
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>
>
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Hector Maletta]

It is not exactly that "additional predictors do not increase the fit" (this
may happen for reasons other than lack of tolerance), but that additional
predictors have a very high correlation with some combination of previous
predictors", as Art correctly explains in his examples.
It is not a case of perfect collinearity, such as having two identical
variables among your predictors. If you had "sex" and "gender", or if you
did not leave one category out while converting a polithomous variable into
dummies, the covariance matrix would be singular and the error message will
be different. That is not the case. What happens is that some variable has a
very high correlation with some of the rest, and is therefore highly
redundant.
One possible way to detect the culprit is using a stepwise method, with all
the caveats such a method deserves.
Hector

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Art
Kendall
Sent: 07 December 2008 10:52
To: [hidden email]
Subject: Re: multiple regression

For some reason, additional predictors do not increase the fit.
do you have a large enough ratio of variables to cases?
a guess.
some subsets of variables are linear combinations of others. are
measuring the same
first check the list of independent variables to be sure that you do not
have the same variable in twice, e.g., a variable called sex, and a
variable called gender.
If you have items for a scale you should not also have the scale score.
If you have dummies to represent levels of a nominal level variable,
there should be one less dummy variable than there are categories of the
group.

If all that fails, use RELIABLILITY pretending your independent
variables are items on a scale.
 be sure to include
 /statistics = descriptives scale correlations
 /summary = total

The descriptives will help you find variables that have no or almost no
variance.
If you look at the pairwise zero order correlations, you can see if some
approach 1.00.
If you look at the SMC (squared multiple correlations) you can see if
any of these approach 1.00.

See if CATREG gives you different messages.

Are you doing the regression for predictive or exploratory or
confirmatory purposes?

If you still have not resolved the problem, please post your DISPLAY
with the variable definitions in the analysis and the syntax you are using.
Then give us a description of what you are trying to do and a rationale
for including _all_ these independents.


Art Kendall
Social Research Consultants

Siavoush Dastmalchi wrote:
> Dear list,
>
> I am doing rather unusual MLR using spss and am using Enter method to use
all of the independent variables. However, two of the independent variables
are excluded from the regression. The output says "Tolerance = .000 limits
reached". I have did some changes to F value or its probability, but never
get to the point to have all variables selected in the MLR.
> Now the question is that how one can force spss to use all of the
variables?

> Many thanks in advance for your kind help.
>
> Cheers, Siavoush
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>
>
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Nancy Darling-2]

Hector is absolutely correct.

I would suggest looking at your correlation matrix.  If you want to use all your measures, you may want to combine some of your variables by building a composite variable or using a factor score instead of individual indicators.

Siavoush Dastmalchi wrote:
  

Dear list,

I am doing rather unusual MLR using spss and am using Enter method to use
    

all of the independent variables. However, two of the independent variables
are excluded from the regression. The output says "Tolerance = .000 limits
reached". I have did some changes to F value or its probability, but never
get to the point to have all variables selected in the MLR.
  

Now the question is that how one can force spss to use all of the
    

variables?
  

Many thanks in advance for your kind help.

Cheers, Siavoush
    

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Hector Maletta]

Looking at the bivariate correlation matrix may not help. The various
correlation coefficients may look all normal and not very high, and still
one predictor may be an almost exact function of some combination of other
predictors.

Another way of "forcing SPSS to accept all variables" is lowering the
TOLERANCE criterion in the REGRESSION command, CRITERIA subcommand (the
default value is 0.001, so one could set it to 0.000001 or something
similar, to see what happens). However, this does not eliminate the problem
addressed by the TOLERANCE criterion, which is multi-collinearity. When one
predictor is an ALMOST exact linear function of other predictors, the
resulting estimates are highly unstable. This means that a slight change in
one or a few values, perhaps in one or a few cases, may fundamentally alter
the results.



Thus it is NOT advisable to force the inclusion of all variables. Even the
SPSS default level of tolerance (0.001) is too low, in my humble opinion,
and should be increased rather than decreased, to be on the safe side.



After all, the aim of science is to predict more with less predictors --and
not the converse.



Hector

  _____

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Nancy Darling
Sent: 07 December 2008 12:39
To: [hidden email]
Subject: Re: multiple regression



Hector is absolutely correct.

I would suggest looking at your correlation matrix.  If you want to use all
your measures, you may want to combine some of your variables by building a
composite variable or using a factor score instead of individual indicators.




Siavoush Dastmalchi wrote:


Dear list,

I am doing rather unusual MLR using spss and am using Enter method to use


all of the independent variables. However, two of the independent variables
are excluded from the regression. The output says "Tolerance = .000 limits
reached". I have did some changes to F value or its probability, but never
get to the point to have all variables selected in the MLR.


Now the question is that how one can force spss to use all of the


variables?


Many thanks in advance for your kind help.

Cheers, Siavoush




===================== To manage your subscription to SPSSX-L, send a message
to [hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command SIGNOFF SPSSX-L For a list of
commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Art Kendall]

The SMC (squared multiple correlations) of each variable from all of the
_*other*_ variables from RELIABILITY is helpful here.

Although RELIABILITY is designed for summative scales,  it is a quick
and dirty way to see these some stats about the distribution of the
correlations, as well as the SMCs. The higher the SMC the more the
multicollinearity. 1.00 means that a variable is completely predicted by
the others that have 1.00.  It is possible for a user to have more than
one set the come close to be perfect linear functions of each other.


Art Kendall
Social Research Consultants

Hector Maletta wrote:

> Looking at the bivariate correlation matrix may not help. The various
> correlation coefficients may look all normal and not very high, and still
> one predictor may be an almost exact function of some combination of other
> predictors.
>
> Another way of "forcing SPSS to accept all variables" is lowering the
> TOLERANCE criterion in the REGRESSION command, CRITERIA subcommand (the
> default value is 0.001, so one could set it to 0.000001 or something
> similar, to see what happens). However, this does not eliminate the problem
> addressed by the TOLERANCE criterion, which is multi-collinearity. When one
> predictor is an ALMOST exact linear function of other predictors, the
> resulting estimates are highly unstable. This means that a slight change in
> one or a few values, perhaps in one or a few cases, may fundamentally alter
> the results.
>
>
>
> Thus it is NOT advisable to force the inclusion of all variables. Even the
> SPSS default level of tolerance (0.001) is too low, in my humble opinion,
> and should be increased rather than decreased, to be on the safe side.
>
>
>
> After all, the aim of science is to predict more with less predictors --and
> not the converse.
>
>
>
> Hector
>
>   _____
>
> From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
> Nancy Darling
> Sent: 07 December 2008 12:39
> To: [hidden email]
> Subject: Re: multiple regression
>
>
>
> Hector is absolutely correct.
>
> I would suggest looking at your correlation matrix.  If you want to use all
> your measures, you may want to combine some of your variables by building a
> composite variable or using a factor score instead of individual indicators.
>
>
>
>
> Siavoush Dastmalchi wrote:
>
>
> Dear list,
>
> I am doing rather unusual MLR using spss and am using Enter method to use
>
>
> all of the independent variables. However, two of the independent variables
> are excluded from the regression. The output says "Tolerance = .000 limits
> reached". I have did some changes to F value or its probability, but never
> get to the point to have all variables selected in the MLR.
>
>
> Now the question is that how one can force spss to use all of the
>
>
> variables?
>
>
> Many thanks in advance for your kind help.
>
> Cheers, Siavoush
>
>
>
>
> ===================== To manage your subscription to SPSSX-L, send a message
> to [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command SIGNOFF SPSSX-L For a list of
> commands to manage subscriptions, send the command INFO REFCARD
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>
>
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Guerrero, Rodrigo]

One way to test for this may be to load each variable at the time and
see how the coefficients change as you add more variables.  If the
variable is truly independent, the coefficient should not change much.
You might be able to narrow the list and identify which variables are
misbehaving.

 

 

 

RG

 

Rodrigo A. Guerrero | Director Of Marketing Research and Analysis | The
Scooter Store | 830.627.4317

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Nancy Darling
Sent: Sunday, December 07, 2008 8:39 AM
To: [hidden email]
Subject: Re: multiple regression

 

Hector is absolutely correct.

I would suggest looking at your correlation matrix.  If you want to use
all your measures, you may want to combine some of your variables by
building a composite variable or using a factor score instead of
individual indicators.



 
Siavoush Dastmalchi wrote:
 

        Dear list,
         
        I am doing rather unusual MLR using spss and am using Enter
method to use
           

all of the independent variables. However, two of the independent
variables
are excluded from the regression. The output says "Tolerance = .000
limits
reached". I have did some changes to F value or its probability, but
never
get to the point to have all variables selected in the MLR.
 

        Now the question is that how one can force spss to use all of
the
           

variables?
 

        Many thanks in advance for your kind help.
         
        Cheers, Siavoush
           

 

===================== To manage your subscription to SPSSX-L, send a
message to [hidden email] (not to SPSSX-L), with no body text
except the command. To leave the list, send the command SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command INFO
REFCARD



The information transmitted is intended only for the addressee(s) and may contain confidential or privileged material, or both.  Any review, receipt, dissemination or other use of this information by non-addressees is prohibited.   If you received this in error or are a non-addressee, please contact the sender and delete the transmitted information.

====================To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Neda Faregh]

Hi all,
I've been running multinomial logistic regressions (SPSS 16) and most of the
outputs have had this warning:
"There is possibly a quasi-complete separation in the data. Either the
maximum likelihood estimates do not exist or some parameter estimates are
infinite."
"The NOMREG procedure continues despite the above warning(s). Subsequent
results shown are based on the last iteration. Validity of the model fit is
uncertain.:

What does this mean exactly? And how does it affect the results?

Are there any solutions to it?

Thank you very much,

Neda

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Anthony Babinec]

Complete separation is a condition where one predictor or a linear
combination of predictors perfectly predicts the target variable. A simple
example might be:

X G
1 0
2 0
3 0
4 0
5 1
6 1
7 1
8 1

Quasi-complete separation occurs when values of the target variable overlap
or are tied on a single or only a few values of the predictor variable.

X G
1 0
2 0
3 0
4 0
4 1
6 1
7 1
8 1

Symptoms are extremely large regression coefficients or large standard
errors.

A remedy might be to remove the predictor from the model, IF you knew which
predictor is the problem. But the problem might involve the interaction of
more than one predictor. This may or may not be straightforward to detect.
See Hosmer and Lemeshow's Applied Regression Analysis 2nd edition for a
suggested principled approach to model building. This includes preliminary
steps such as fitting single predictor models using each predictor in turn.

Anthony Babinec
[hidden email]


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Neda Faregh
Sent: Monday, December 08, 2008 11:32 AM
To: [hidden email]
Subject: Questions about Multinomial logistic regression output

Hi all,
I've been running multinomial logistic regressions (SPSS 16) and most of the
outputs have had this warning:
"There is possibly a quasi-complete separation in the data. Either the
maximum likelihood estimates do not exist or some parameter estimates are
infinite."
"The NOMREG procedure continues despite the above warning(s). Subsequent
results shown are based on the last iteration. Validity of the model fit is
uncertain.:

What does this mean exactly? And how does it affect the results?

Are there any solutions to it?

Thank you very much,

Neda

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[SR Millis-3]

I'd suggest using the 2-stage method of Belsley, Kuh, and Welsch (1980, 2004) in which you examine both the condition indexes and their corresponding variance decomposition proportions.  SPSS will provide these indices when you request collinearity diagnostics.


Scott R Millis, PhD, MEd, ABPP (CN,CL,RP), CStat
Professor & Director of Research
Dept of Physical Medicine & Rehabilitation
Wayne State University School of Medicine
261 Mack Blvd
Detroit, MI 48201
Email:  [hidden email]
Tel: [phone]
Fax: [phone]


--- On Mon, 12/8/08, Guerrero, Rodrigo <[hidden email]> wrote:

> From: Guerrero, Rodrigo <[hidden email]>
> Subject: Re: multiple regression
> To: [hidden email]
> Date: Monday, December 8, 2008, 11:02 AM
> One way to test for this may be to load each variable at the
> time and
> see how the coefficients change as you add more variables.
> If the
> variable is truly independent, the coefficient should not
> change much.
> You might be able to narrow the list and identify which
> variables are
> misbehaving.
>
>
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD