A tree based lack of fit test

[Cathy Mazzotta]

 

 Hi everyone,

 I am working on my dissertation and will be doing a multivariate logistical
 regression for my analysis.

 I have continuous co-variates and a dichotomous dependent variable.

 I came across the following reference:
 Moons, E., Aerts, M., & Wets, G. A tree based lack of fit test for multiple
 logistic regression.   The authors compare Tree based lack of fit to Hosmer and Lemeshow.

 Is this an accepted alternative to H & L?  I would appreciate any thoughts
 the you all may have.

 

Thank you so much

 

C. Mazzotta

Doctoral Candidate

UB

Buffalo, NY

[Mike]

A few points:

 

(1)  I believe that you are referring to the following reference:

http://onlinelibrary.wiley.com/doi/10.1002/sim.1750/abstract

It's a good idea to provide a complete reference, especially if

one wants to know if it is highly cited or not.

 

(2) In scholar.google.com, there are only 5 sources citing this

article and only one appears to be journal article (Theetan et al 2007).

From their presentation, their justification does not seem to be

very convincing to me but what do I know.  I note that both the

Moons et al paper and the Theetan et al paper come from institutions

in Belgium and perhaps this is a procedure that has some local

preference (a wide acceptance would be indicated by a larger

number of citation and from institutions in different locations).

 

(3)  If you have read the Moons et al paper, early on they state

that there are various nonparametric analyses that can be used to

assess fit in logistic regression when one has continuous predictors.

Only the Hosmer & Lemeshow test makes use of the Pearson

chi-square while the nonparametric tests don't -- Moons et al

propose a test that also makes use of the Chi-square which seems

to be their main argument for their test but both produce approximate

results.  The Tree-based (TB) approach appears to behave better

than the Hosmer & Lemeshow test (e.g., better power, etc.) but

one will have to spend some time determining whether that is the

case for one's dataset.

(4)  Moons et al state at the end of their article that an S-plus program

is available from the authors but you have to contact them as they

don't provide a website address to access the S-plus code.  This raises

the question of whether one can only do their analyses in S-plus (do you

have access to it?) or can it be translated into SPSS?  Translating

it into SPSS and testing it could be a time-consuming process unless

you have someone really skilled doing the work -- I hear that David

Marso is good at this, if you have the money. ;-)

 

So, you have to ask yourself:  does the Moons et al TB procedure

answer the questions you want answered better than other procedures

(e.g., the Hosmer & Lemeshow test or one of the other nonparametric

procedures)?  If so, then you might want to invest the time and effort

in setting up the TB procedure in whatever fashion your resources allow.

You might want to ask your dissertation advisor about this and/or see

if you can find someone who has actually used this procedure and what

ask them if it is really what you need.  However, it seems to me that not

many people may have used this procedure, so finding an appropriate

consultant might be difficult.

 

It would be interesting to hear what others have to say about this.

 

-Mike Palij

New York University

[hidden email]

 

 

----- Original Message -----

From: [hidden email]

To: [hidden email]

Sent: Monday, June 17, 2013 11:03 AM

Subject: A tree based lack of fit test

 

 Hi everyone,

 I am working on my dissertation and will be doing a multivariate logistical
 regression for my analysis.

 I have continuous co-variates and a dichotomous dependent variable.

 I came across the following reference:
 Moons, E., Aerts, M., & Wets, G. A tree based lack of fit test for multiple
 logistic regression.   The authors compare Tree based lack of fit to Hosmer and Lemeshow.

 Is this an accepted alternative to H & L?  I would appreciate any thoughts
 the you all may have.

 

Thank you so much

 

C. Mazzotta

Doctoral Candidate

UB

Buffalo, NY

[Andy W]

Accepted is perhaps not the best way to frame it. It has only been cited by 5 people, but it is in a good journal and you should read it and make that judgement for yourself if it offers obvious improvements over the HL test. I provide various other references to logistic regression diagnostics on this question on the CrossValidated forum, http://stats.stackexchange.com/a/45099/1036. It is an area of active research.

[Bruce Weaver]

In addition to what Mike and Andy have said, see Frank Harrell's post in this thread on the R help list:

  http://r.789695.n4.nabble.com/Hosmer-Lemeshow-test-td4665115.html

Harrell is the author of the well-known book, Regression Modeling Strategies.  You can probably find other comments he has made in the Google Groups Medstats group, or on StackExchange / Cross Validated.

HTH.

Cathy Mazzotta wrote

Hi everyone,

 I am working on my dissertation and will be doing a multivariate logistical
 regression for my analysis.
 I have continuous co-variates and a dichotomous dependent variable.
 I came across the following reference:
 Moons, E., Aerts, M., & Wets, G. A tree based lack of fit test for multiple
 logistic regression.   The authors compare Tree based lack of fit to
Hosmer and Lemeshow.
 Is this an accepted alternative to H & L?  I would appreciate any thoughts
 the you all may have.

Thank you so much

C. Mazzotta
Doctoral Candidate
UB
Buffalo, NY

[Patrick Estes]

Greetings,
Thank you for your message. I will be out of the office with limited internet access and will return on Tuesday, May 28th. I will reply to your email as soon as possible when I return if you seek a reply.
If you require immediate assistance, please contact E.J. Keeley, Director of Institutional Assessment & Research, at [hidden email]<mailto:[hidden email]> or x2223.

Thanks,

-Pat

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD