Logistic Regression Evaluation

[Brock-15]

Hello all,

 

I have been thrown headfirst into SPSS (limited experince) and have been asked to model outcomes.  Based on my research, I see that modeling a variable that is binary can be done with logistic regression.  I have created a "model" to the best of my ability based on theory and statistics, but wonder on its application.  I know (think) that models can be created for a variety of reasons, but to put one into action, is there a statistically accepted cutoff?  Is it the various r-square measures?  Predicted group membership success?  Statistically significant variables?  I know all of this matters, but to make decisions based on a model, it seems that there should be some criterion that can be used.  Not to bring in multiple regression, but an r-square of .05 might not be applicable in a practical sense.

 

Anyway, I know this may be off topic, but I figure SPSS users have war stories/experience that they can share on the matter.

 

Many thanks in advance.

 

Brock

[Jason McNellis]

Brock,



I was in the same situation several years ago.  I found Andy Field's book
tremendously helpful in understanding the basics.  Several analogous
R-squared measures have been proposed for logistic regression and a few of
those are talked about in Andy Field's book.



As far as cut-offs in my experience it depends on how the data is being
used.  I have had times where the selected cases were going to be mailed a
very expensive (for the company producing it) offer and the model had to be
very accurate (high predicted membership success).  I have had other times
where a client is going to mail (a relatively cheap offer) to half the
customers on file and they want more guidance than picking half randomly.



Have a great day, Jason







  _____

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
[hidden email]
Sent: Thursday, October 26, 2006 9:01 PM
To: [hidden email]
Subject: Logistic Regression Evaluation



Hello all,



I have been thrown headfirst into SPSS (limited experince) and have been
asked to model outcomes.  Based on my research, I see that modeling a
variable that is binary can be done with logistic regression.  I have
created a "model" to the best of my ability based on theory and statistics,
but wonder on its application.  I know (think) that models can be created
for a variety of reasons, but to put one into action, is there a
statistically accepted cutoff?  Is it the various r-square measures?
Predicted group membership success?  Statistically significant variables?  I
know all of this matters, but to make decisions based on a model, it seems
that there should be some criterion that can be used.  Not to bring in
multiple regression, but an r-square of .05 might not be applicable in a
practical sense.



Anyway, I know this may be off topic, but I figure SPSS users have war
stories/experience that they can share on the matter.



Many thanks in advance.



Brock

[Brock-15]

Hi Jason,

Thanks for your willingness to respond.  My model could have some serious
implications if the outcomes do not occur.  I guess I am getting hung up
on the concept of what a fundamentally sound model is.  I have read in
various texts that the various R-square measures will not be as high as if
you were doing a Linear Multiple Regression.  In short, that simply
comparing an R-square from a Logistic to a MR is no comparable because the
Logistic will be lower.  Therefore, sometimes people will throw away
results of a Logistic model because it may only explain .44% of the
variance.

However, that leaves me to evaluate a model on classification.  From your
experience, when client campaigns were successful, were there any traits
of the model that were common throughout?  Is there a level of (for lack
of a better term) inaccuracy in which you would simply tell your client we
can not act upon this information?

Many thanks again.  I have been doing countless searches for information
on the topic.  I know sometimes its a judgement call, but it appears as if
many of the texts I read simply talk about the estimated coefficients,
etc, and never discuss if it would be practical to put this model into
practice.

I appreciate your time,

Brock