validating a logistic regression model

[J P-6]

Dear Colleagues,

I am attempting to learn  how to valide a logististic regression model. I've been reading about bootstrapping and cross validation, etc. But have found no instruction on how to actually conduct the anaysis and interpret the results. Any references on this subject or advice on how to perform this with SPSS is greatly appreciated.

Here is an example of what I am talking about....http://symptomresearch.nih.gov/chapter_8/sec7/cess7pg14.htm

Thanks!
John




====================To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Ornelas, Fermin-2]

There is a book on data mining wrote by Ruth Phar (not sure about the last name spelling). Have the book at home. She should give you some sas code and ideas on how to validate a logistic regression model applied to the credit card industry

F. Ornelas

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of J P
Sent: Wednesday, December 10, 2008 1:33 PM
To: [hidden email]
Subject: validating a logistic regression model

Dear Colleagues,

I am attempting to learn  how to valide a logististic regression model. I've been reading about bootstrapping and cross validation, etc. But have found no instruction on how to actually conduct the anaysis and interpret the results. Any references on this subject or advice on how to perform this with SPSS is greatly appreciated.

Here is an example of what I am talking about....http://symptomresearch.nih.gov/chapter_8/sec7/cess7pg14.htm

Thanks!
John




=======
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR CONFIDENTIAL information and is intended only for the use of the specific individual(s) to whom it is addressed.  It may contain information that is privileged and confidential under state and federal law.  This information may be used or disclosed only in accordance with law, and you may be subject to penalties under law for improper use or further disclosure of the information in this e-mail and its attachments. If you have received this e-mail in error, please immediately notify the person named above by reply e-mail, and then delete the original e-mail.  Thank you.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Hector Maletta]

Whatever you use, please do not use the percentage of correctly predicted
individual cases. In my opinion it means little or nothing (even if you
choose a "correct" cutoff point, which is itself a difficult and --from some
viewpoints-- an unsolvable problem). Probability prediction by logistic
regression is not predicated of individuals but of populations or groups or
similar individuals: any individual outcome is compatible with the
prediction. For instance, if you predict a 90% probability that I (or more
exactly, people with my values in the chosen predictors) would die within a
year, my eventual survival for another 45 years is perfectly compatible with
that prediction. What you were actually predicting is that out of a large
number of people like me, 90 out of every 100 die within one year. I just
happened to be in the lucky 10% living longer. Even the estimated 90%
probability is itself subjet to estimation error: the "true" population
probability might be higher or lower, with certain probability of error (you
may have, say, a 95% chance that the true probability is between 0.85 and
0.95, and 5% chance that is it either lower or higher. In other words, the
true probability might be much lower.
The probability (observed or predicted), whatever its value happens to be,
is an attribute of the group, not an attribute of each subject. This is, of
course, the frequentist interpretation of probability, but it is arguably
the only consistent one. Individual outcomes of random variables are
strictly indeterminate: it is the group aggregate outcome which is subject
to the prediction.
With these caveats in mind, you may turn for example to Hosmer and
Lemeshow's book, Applied Logistic Regression for detailed information about
significance tests and goodness of fit tests for logistic regression, and
about applying a logistic regression solution (obtained from one dataset) to
a second dataset with a validation purpose.
Hope this helps.

Hector
-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of J P
Sent: 10 December 2008 18:33
To: [hidden email]
Subject: validating a logistic regression model

Dear Colleagues,

I am attempting to learn  how to valide a logististic regression model. I've
been reading about bootstrapping and cross validation, etc. But have found
no instruction on how to actually conduct the anaysis and interpret the
results. Any references on this subject or advice on how to perform this
with SPSS is greatly appreciated.

Here is an example of what I am talking
about....http://symptomresearch.nih.gov/chapter_8/sec7/cess7pg14.htm

Thanks!
John





To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Whanger, J. Mr. CTR]

John,

Before proceeding, you may want to consider the logic underlying the notion of cross-validation. If the goal is to most accurately estimate population parameters that can later be used for probabilistic prediction, then a large sample that is representative of the population is the solution rather than reducing statistical power via sample splitting or bootstrapping.  When the sample size is large enough, bootstrapped parameter estimates will be roughly equivalent to non-bootstrapped.

However, because sample splitting cross-validation & bootstrapping are considered "best practice" you will likely need to be prepared to defend why a larger sample size provides the better estimate of population parameters relative to the other two methods.

Regards,

Jim


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of J P
Sent: Wednesday, December 10, 2008 3:33 PM
To: [hidden email]
Subject: validating a logistic regression model

Dear Colleagues,

I am attempting to learn  how to valide a logististic regression model. I've been reading about bootstrapping and cross validation, etc. But have found no instruction on how to actually conduct the anaysis and interpret the results. Any references on this subject or advice on how to perform this with SPSS is greatly appreciated.

Here is an example of what I am talking about....http://symptomresearch.nih.gov/chapter_8/sec7/cess7pg14.htm

Thanks!
John




=======
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[SR Millis-3]

Here's an excellent book on the topic:

Steyerberg, E. (2009). Clinical prediction models: A practical approach to development, validation, and updating. New York: Springer.


Scott R Millis, PhD, MEd, ABPP (CN,CL,RP), CStat
Professor & Director of Research
Dept of Physical Medicine & Rehabilitation
Wayne State University School of Medicine
261 Mack Blvd
Detroit, MI 48201
Email:  [hidden email]
Tel: [phone]
Fax: [phone]


--- On Wed, 12/10/08, Ornelas, Fermin <[hidden email]> wrote:

> From: Ornelas, Fermin <[hidden email]>
> Subject: Re: validating a logistic regression model
> To: [hidden email]
> Date: Wednesday, December 10, 2008, 4:02 PM
> There is a book on data mining wrote by Ruth Phar (not sure
> about the last name spelling). Have the book at home. She
> should give you some sas code and ideas on how to validate a
> logistic regression model applied to the credit card
> industry
>
> F. Ornelas
>
> -----Original Message-----
> From: SPSSX(r) Discussion [mailto:[hidden email]]
> On Behalf Of J P
> Sent: Wednesday, December 10, 2008 1:33 PM
> To: [hidden email]
> Subject: validating a logistic regression model
>
> Dear Colleagues,
>
> I am attempting to learn  how to valide a logististic
> regression model. I've been reading about bootstrapping
> and cross validation, etc. But have found no instruction on
> how to actually conduct the anaysis and interpret the
> results. Any references on this subject or advice on how to
> perform this with SPSS is greatly appreciated.
>
> Here is an example of what I am talking
> about....http://symptomresearch.nih.gov/chapter_8/sec7/cess7pg14.htm
>
> Thanks!
> John
>
>
>
>
> =======
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body
> text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the
> command
> INFO REFCARD
>
>
> NOTICE: This e-mail (and any attachments) may contain
> PRIVILEGED OR CONFIDENTIAL information and is intended only
> for the use of the specific individual(s) to whom it is
> addressed.  It may contain information that is privileged
> and confidential under state and federal law.  This
> information may be used or disclosed only in accordance with
> law, and you may be subject to penalties under law for
> improper use or further disclosure of the information in
> this e-mail and its attachments. If you have received this
> e-mail in error, please immediately notify the person named
> above by reply e-mail, and then delete the original e-mail.
> Thank you.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body
> text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the
> command
> INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Johnny Amora]

Can you please email the title of the book and authors?


Johnny
--- On Thu, 12/11/08, Ornelas, Fermin <[hidden email]> wrote:

From: Ornelas, Fermin <[hidden email]>
Subject: Re: validating a logistic regression model
To: [hidden email]
Date: Thursday, 11 December, 2008, 5:02 AM

There is a book on data mining wrote by Ruth Phar (not sure about the last name
spelling). Have the book at home. She should give you some sas code and ideas on
how to validate a logistic regression model applied to the credit card industry

F. Ornelas

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of J P
Sent: Wednesday, December 10, 2008 1:33 PM
To: [hidden email]
Subject: validating a logistic regression model

Dear Colleagues,

I am attempting to learn  how to valide a logististic regression model.
I've been reading about bootstrapping and cross validation, etc. But have
found no instruction on how to actually conduct the anaysis and interpret the
results. Any references on this subject or advice on how to perform this with
SPSS is greatly appreciated.

Here is an example of what I am talking
about....http://symptomresearch.nih.gov/chapter_8/sec7/cess7pg14.htm

Thanks!
John




=======
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR
CONFIDENTIAL information and is intended only for the use of the specific
individual(s) to whom it is addressed.  It may contain information that is
privileged and confidential under state and federal law.  This information may
be used or disclosed only in accordance with law, and you may be subject to
penalties under law for improper use or further disclosure of the information in
this e-mail and its attachments. If you have received this e-mail in error,
please immediately notify the person named above by reply e-mail, and then
delete the original e-mail.  Thank you.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD



      Importing contacts has never been easier. Bring your friends over to Yahoo! Mail today! http://www.trueswitch.com/yahoo-ph

====================To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Kruger, Michael]

Here's some syntax that I recently successfully used for bootstrapping
logistic regression coefficients.
It's syntax that I modified from the SPSS Programming and Data
Management Guide (4th ed.), Ch. 9, p155-164.


***Modify the following CD command to specify the Samples folder
   in the directory in which SPSS is installed on your computer,
   using the conventions for your operating system.

CD '/SPSS/Samples'.


***oms_bootstrapping.sps***.

PRESERVE.
SET TVARS NAMES.

***first OMS command just suppresses Viewer output***.
OMS /DESTINATION VIEWER=YES.

DATASET DECLARE bootstrap_example.

***select regression coefficients tables and write to data file***.
***Note that DIMNAMES values vary based on output language***.
***/COLUMNS SEQUENCE=[R2 C1] will achieve the same result in all
languages***.

OMS /SELECT TABLES
    /IF COMMANDS=['Logistic Regression'] SUBTYPES = ['Variables in the
Equation']
   /DESTINATION FORMAT=SAV OUTFILE='bootstrap_example'
   /COLUMNS DIMNAMES=['Variables'  'Statistics']
   /TAG='logeg_coeff'.

***define a macro to draw samples with replacement and run Regression
commands***.
DEFINE  regression_bootstrap (samples=!TOKENS(1)
                                           /depvar=!TOKENS(1)
                                           /indvars=!CMDEND)

COMPUTE dummyvar=1.
AGGREGATE
  /OUTFILE = * MODE = ADDVARIABLES
  /BREAK=dummyvar
  /filesize=N.
!DO !other=1 !TO !samples
SET SEED RANDOM.
WEIGHT OFF.
FILTER OFF.
DO IF $casenum=1.
- COMPUTE #samplesize=filesize.
- COMPUTE #filesize=filesize.
END IF.
DO IF (#samplesize>0 and #filesize>0).
- COMPUTE sampleWeight=rv.binom(#samplesize, 1/#filesize).
- COMPUTE #samplesize=#samplesize-sampleWeight.
- COMPUTE #filesize=#filesize-1.
ELSE.
- COMPUTE sampleWeight=0.
END IF.
WEIGHT BY sampleWeight.
FILTER BY sampleWeight.

LOGISTIC REGRESSION svlac
 /METHOD = ENTER  age gest_age racecat2 para nullip_stat gender marstat2
             /CONTRAST (racecat2)=Deviation(1)  /CONTRAST
(nullip_stat)=Indicator(1) /CONTRAST (gender)=Indicator  /CONTRAST
(marstat2)=Indicator(1)
  /PRINT = DEFAULT
  /CRITERIA = PIN(.05) POUT(.10) ITERATE(20) CUT(.5).
!DOEND
!ENDDEFINE.

***Call the macro, and specify number of samples, dependent variable,
and independent variables***.
regression_bootstrap
   samples=10
   depvar=svlac
   indvars=age gest_age racecat2 para nullip_stat gender marstat2.

OMSEND.

DATASET ACTIVATE bootstrap_example.

RESTORE.


Michael Kruger
"A True Prince"
Statistical Analyst
C.S. Mott Center
Dept. of OB/GYN
Wayne State University School of Medicine
(313)-577-1794



-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
SR Millis
Sent: Wednesday, December 10, 2008 4:59 PM
To: [hidden email]
Subject: Re: validating a logistic regression model


Here's an excellent book on the topic:

Steyerberg, E. (2009). Clinical prediction models: A practical approach
to development, validation, and updating. New York: Springer.


Scott R Millis, PhD, MEd, ABPP (CN,CL,RP), CStat
Professor & Director of Research
Dept of Physical Medicine & Rehabilitation
Wayne State University School of Medicine
261 Mack Blvd
Detroit, MI 48201
Email:  [hidden email]
Tel: [phone]
Fax: [phone]


--- On Wed, 12/10/08, Ornelas, Fermin <[hidden email]> wrote:

> From: Ornelas, Fermin <[hidden email]>
> Subject: Re: validating a logistic regression model
> To: [hidden email]
> Date: Wednesday, December 10, 2008, 4:02 PM
> There is a book on data mining wrote by Ruth Phar (not sure about the
> last name spelling). Have the book at home. She should give you some
> sas code and ideas on how to validate a logistic regression model
> applied to the credit card industry
>
> F. Ornelas
>
> -----Original Message-----
> From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf
> Of J P
> Sent: Wednesday, December 10, 2008 1:33 PM
> To: [hidden email]
> Subject: validating a logistic regression model
>
> Dear Colleagues,
>
> I am attempting to learn  how to valide a logististic regression
> model. I've been reading about bootstrapping and cross validation,
> etc. But have found no instruction on how to actually conduct the
> anaysis and interpret the results. Any references on this subject or
> advice on how to perform this with SPSS is greatly appreciated.
>
> Here is an example of what I am talking
> about....http://symptomresearch.nih.gov/chapter_8/sec7/cess7pg14.htm
>
> Thanks!
> John
>
>
>
>
> =======
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except
> the command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the
> command
> INFO REFCARD
>
>
> NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR
> CONFIDENTIAL information and is intended only for the use of the
> specific individual(s) to whom it is addressed.  It may contain
> information that is privileged and confidential under state and
> federal law.  This information may be used or disclosed only in
> accordance with law, and you may be subject to penalties under law for
> improper use or further disclosure of the information in
> this e-mail and its attachments. If you have received this
> e-mail in error, please immediately notify the person named
> above by reply e-mail, and then delete the original e-mail.
> Thank you.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except
> the command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the
> command
> INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command SIGNOFF SPSSX-L For a list
of commands to manage subscriptions, send the command INFO REFCARD

__________ Information from ESET NOD32 Antivirus, version of virus
signature database 3682 (20081210) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD