Re: Multinomial Logistic Regression - Category Size

Posted by Bruce Weaver on Nov 30, 2010; 9:55pm
URL: http://spssx-discussion.165.s1.nabble.com/Multinomial-Logistic-Regression-Category-Size-tp3286013p3286844.html

s-volk wrote

Dear all,

I ran a multinomial logistic regression analysis with one continuous independent variable. I have a sample size of 68 subjects (psychological experiment) which end up split into 5 categories ranging in size from 5 to 24 (dependent variable). The MLR-model has a Nagelkerke R2 of 0.27 and Model Fit χ2=19.71, p<0.01.

Now here is the problem: A reviewer complains that my results may be sample specific because one of the 5 categories of the dependent variable consists of only 5 observations (subjects), i.e. sh/e argues that very few participants (five) are responsible for the observed effects. Is this valid argument? I thought that if the overall model is significant, I can conclude that there is a significant relationship between the dependent and independent variable for all categories of the dependent variable? That is, the calculations for the overall model are based on all observations (68) and not only on the observations in specific categories (e.g., 5)?

I was wondering if someone could provide me with or point me to some arguments for reviewers (ideally including some references)?

Many thanks in advance,
Stefan

I'll have a kick at this one, more to get some discussion going than to provide any definitive answers.  ;-)

I suppose the comment about it being "sample specific" translates to "will not generalize well to other samples".  

Just thinking out loud here, so forgive me if it ends up being twaddle.  What if you ran the model again, but without the 5 potentially problematic cases.  If the predicted probabilities from the two models were very similar for the other N-5 cases, this might reassure the reviewer that the omitted 5 are not overly influential.  On the other hand, if the predicted probabilities differ a fair bit, that would confirm the reviewer's fears.

Another possibility--could the outcome category that the 5 problem are in reasonably be merged with one of the other categories?  Again, if the predicted probabilities from this model didn't differ substantially from those obtained with the original model, you could argue that the 5 cases are not very influential.

Perhaps someone else will have a better idea--remember, I was just trying to prime the pump here!

HTH.