Combining multiple ROC data sets
Locked
 
|
CONTENTS DELETED
The author has deleted this message.
|
Re: Combining multiple ROC data sets
|
|
As I understand it, ROC curves are typically based on two variables -
an observed state variable and predictions from some model. The observed state variable could be a binary variable indicating disease yes/no, attrition yes/no, and the like. As the researcher, you would view one of the two conditions as the "positive" condition, which would enable calculation of sensitivity and specificity for different cutoff values. The predictions would come from a model such as logistic regression. The ROC curve and its related AUC value could be used in model comparison, since you could obtain predictions from 1-variable models or a 6-variable model. Tony Babinec [hidden email] -----Original Message----- From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of mseth Sent: Friday, November 05, 2010 8:18 AM To: [hidden email] Subject: Combining multiple ROC data sets Hi I have a fairly simple data set with 6 variables (5 continuous and 1 categorical) and about 200 entries.I have produced ROC curves for each individual variable and they each have an AUC in the region of 0.75-0.85. Is there any way I can use SPSS to see if combining variables can increase the ROC? Thanks for any/all help Mayank -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Combining-multiple-ROC-data-se ts-tp3251795p3251795.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
|
In reply to this post by mseth
Time is really against me right now, but the VERY short answer is yes. One approach would be to fit a binary logistic regression (BLR) model treating all 6 variables as predictors and the actual state variable as the dependent variable. Save the predicted values from the BLR, and then use those values as the "TEST" variable in the ROC analysis.
Ryan
On Fri, Nov 5, 2010 at 9:18 AM, mseth <[hidden email]> wrote: Hi |
Re: Combining multiple ROC data sets
Administrator
|
Just in case it is not clear to the OP, Ryan is referring to the "predicted probabilities" from the binary logistic regression model.
HTH.
--
Bruce Weaver bweaver@lakeheadu.ca http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." PLEASE NOTE THE FOLLOWING: 1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. 2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/). |
|
|
Hi all,
The code I present below does the following:
(1) generates data to be used in the logistic regression
(2) runs the logistic regression on the simulated data
(3) runs the ROC analysis using the predicted probabilities obtained from the logistic regression
Hope this helps in some way.
Ryan
--
*Generate data.
set seed 98765432.
new file. inp pro.
loop ID= 1 to 10000.
comp x1 = rv.normal(0,1). comp x2 = rv.normal(1,2). comp x3 = rv.normal(1,1.5). comp b0 = -1.5. comp b1 = 0.9. comp b2 = 1.2. comp b3 = 0.6 comp eta = b0 + b1*x1 + b2*x2 + b3*x3.
comp prob = exp(eta) / (1+ exp(eta)). comp y = rv.bernoulli(prob).
end case.
end loop. end file. end inp pro. exe. Delete variables b0 b1 b2 b3 eta prob.
*Run the logistic regression and save the predicted probabilities.
LOGISTIC REGRESSION VARIABLES y
/METHOD=ENTER x1 x2 x3 /SAVE=PRED /CRITERIA=PIN(.05) POUT(.10) ITERATE(20) CUT(.5). *Run the ROC analysis using the predicted probabilities as the "Test Variable".
ROC PRE_1 BY y (1)
/PLOT=CURVE(REFERENCE) /PRINT=SE COORDINATES /CRITERIA=CUTOFF(INCLUDE) TESTPOS(LARGE) DISTRIBUTION(FREE) CI(95) /MISSING=EXCLUDE. On Fri, Nov 5, 2010 at 11:19 AM, Bruce Weaver <[hidden email]> wrote:
Just in case it is not clear to the OP, Ryan is referring to the "predicted |
|
|
In reply to this post by Ryan
Hello Ryan,
I' ve found your answer here and have a question, because I have a similar problem. I have got 4 predictors in a ROC analysis. I want to test if a combined predictor would be better than the four seperate predictors. If my understanding of your answer is correct, I must save the predicted values from the binary logistic regression. How can I do this? Afterwards, I must use those values as the "Test" variable in the ROC analysis. Have you got any reference for doing so? Is it correct that one can also use the Bs from the BLR to weight the predictors and compute a ROC analysis with this function? You really would help me a lot with a reference. best wishes, JR |
|
|
JR,
I assume you did not see my subsequent post demonstrating how to do it. Here you go: http://www.listserv.uga.edu/cgi-bin/wa?A2=ind1011&L=spssx-l&P=R13051 Hopefully someone else can respond to you about your particular questions. Time is against me today. Ryan On Mon, Nov 22, 2010 at 5:50 AM, JR <[hidden email]> wrote: > Hello Ryan, > > I' ve found your answer here and have a question, because I have a similar > problem. I have got 4 predictors in a ROC analysis. I want to test if a > combined predictor would be better than the four seperate predictors. If my > understanding of your answer is correct, I must save the predicted values > from the binary logistic regression. How can I do this? Afterwards, I must > use those values as the "Test" variable in the ROC analysis. Have you got > any reference for doing so? Is it correct that one can also use the Bs from > the BLR to weight the predictors and compute a ROC analysis with this > function? > > You really would help me a lot with a reference. > > best wishes, > JR > -- > View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Combining-multiple-ROC-data-sets-tp3251795p3275180.html > Sent from the SPSSX Discussion mailing list archive at Nabble.com. > > ===================== > To manage your subscription to SPSSX-L, send a message to > [hidden email] (not to SPSSX-L), with no body text except the > command. To leave the list, send the command > SIGNOFF SPSSX-L > For a list of commands to manage subscriptions, send the command > INFO REFCARD > ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
|
Dear Ryan,
thank you a lot for your answer. Now, I understand the principle. I have a lot of time pressure to find a reference describing that procedure. Do you remember in which book you read about? Best wishes, JR |
«
Return to SPSSX Discussion
|
1 view|%1 views
| Free forum by Nabble | Edit this page |