REGRESSION CROSS-VALIDATION

[Kathryn Gardner]

Dear list,
 
Can someone please tell me how to cross-validate a regression model? I'm currently using multinominal logistic regression and want to use 70% of my sample for the model, and then see how well the model can correctly classify the remain 30%. I  have read that in standard linear regression I can use the regression coefficients of the 70% of my sample to obtain predicted values for the remaining 30% of the sample (the validation sample). I'm not sure how to do this though. Can someone please advise or does anyone have some syntax I can use?
 
Thanks
Kathryn
_________________________________________________________________
Celeb spotting – Play CelebMashup and win cool prizes
https://www.celebmashup.com/index2.html

[Ornelas, Fermin]

In my own experience with binary logistic regression I have run the
validation data through the logistic expression and score the data by
creating some deciles table to see how well the model is capturing those
individuals identified as having the event (=1). The validation code
also includes the transformations you may have applied to your data
prior to the estimation. It is also important that your descriptive
statistics from your validation sample share some similarities with the
development sample so as to ensure an adequate performance compared to
the development sample. I actually graphed a lift curve showing the
model's performance. Hope this comment is helpful for your modeling
exercise.

Fermin Ornelas, Ph.D.
Management Analyst III, AZ DES
1789 W. Jefferson Street
Phoenix, AZ 85032
Tel: [phone]
E-mail: [hidden email]

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Kathryn Gardner
Sent: Monday, August 06, 2007 8:53 AM
To: [hidden email]
Subject: REGRESSION CROSS-VALIDATION

Dear list,

Can someone please tell me how to cross-validate a regression model? I'm
currently using multinominal logistic regression and want to use 70% of
my sample for the model, and then see how well the model can correctly
classify the remain 30%. I  have read that in standard linear regression
I can use the regression coefficients of the 70% of my sample to obtain
predicted values for the remaining 30% of the sample (the validation
sample). I'm not sure how to do this though. Can someone please advise
or does anyone have some syntax I can use?

Thanks
Kathryn
_________________________________________________________________
Celeb spotting - Play CelebMashup and win cool prizes
https://www.celebmashup.com/index2.html

NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR
CONFIDENTIAL information and is intended only for the use of the
specific individual(s) to whom it is addressed.  It may contain
information that is privileged and confidential under state and federal
law.  This information may be used or disclosed only in accordance with
law, and you may be subject to penalties under law for improper use or
further disclosure of the information in this e-mail and its
attachments. If you have received this e-mail in error, please
immediately notify the person named above by reply e-mail, and then
delete the original e-mail.  Thank you.

[Kathryn Gardner]

Thanks for the advice. I'll look into this further.
 
I have been trying to work out how to run a "leave-one-out" classification in logistic regression (Discriminant Function Analysis has an option for this). If anyone has any syntax or knows how to do this i'd be really greatful. The only way I can see of doing it is to manually de-select each case and re-run the model (a long tedious process with a high n).
 
Kathryn> Date: Mon, 6 Aug 2007 09:45:47 -0700> From: [hidden email]> Subject: Re: REGRESSION CROSS-VALIDATION> To: [hidden email]> > In my own experience with binary logistic regression I have run the> validation data through the logistic expression and score the data by> creating some deciles table to see how well the model is capturing those> individuals identified as having the event (=1). The validation code> also includes the transformations you may have applied to your data> prior to the estimation. It is also important that your descriptive> statistics from your validation sample share some similarities with the> development sample so as to ensure an adequate performance compared to> the development sample. I actually graphed a lift curve showing the> model's performance. Hope this comment is helpful for your modeling> exercise.> > Fermin Ornelas, Ph.D.> Management Analyst III, AZ DES> 1789 W. Jefferson Street> Phoenix, AZ 85032> Tel: [phone]> E-mail: [hidden email]> > -----Original Message-----> From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of> Kathryn Gardner> Sent: Monday, August 06, 2007 8:53 AM> To: [hidden email]> Subject: REGRESSION CROSS-VALIDATION> > Dear list,> > Can someone please tell me how to cross-validate a regression model? I'm> currently using multinominal logistic regression and want to use 70% of> my sample for the model, and then see how well the model can correctly> classify the remain 30%. I have read that in standard linear regression> I can use the regression coefficients of the 70% of my sample to obtain> predicted values for the remaining 30% of the sample (the validation> sample). I'm not sure how to do this though. Can someone please advise> or does anyone have some syntax I can use?> > Thanks> Kathryn> _________________________________________________________________> Celeb spotting - Play CelebMashup and win cool prizes> https://www.celebmashup.com/index2.html> > NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR> CONFIDENTIAL information and is intended only for the use of the> specific individual(s) to whom it is addressed. It may contain> information that is privileged and confidential under state and federal> law. This information may be used or disclosed only in accordance with> law, and you may be subject to penalties under law for improper use or> further disclosure of the information in this e-mail and its> attachments. If you have received this e-mail in error, please> immediately notify the person named above by reply e-mail, and then> delete the original e-mail. Thank you.
_________________________________________________________________
The next generation of MSN Hotmail has arrived - Windows Live Hotmail
http://www.newhotmail.co.uk