Re: Bayesian Information Criterion & Checking for Normality

Posted by David Greenberg on Apr 16, 2013; 6:51pm
URL: http://spssx-discussion.165.s1.nabble.com/Bayesian-Information-Criterion-Checking-for-Normality-tp5719504p5719505.html

My intuition is that with different dependent variables, a comparison of BIC values would be meaningless. David Greenberg, Sociology Department, New York University

On Tue, Apr 16, 2013 at 2:37 PM, Kornbrot, Diana <[hidden email]> wrote:

Dear Colleagues,

Some time back I posted following query
Can I use BIC to compare 3 different transformations of a dependent variable, here accuracy of time estimation?
I am examining ‘accuracy’ as function of 1 between and 2 repeated measures factors using SPSS ‘MIXED’.

True value is A, obtained participant estimate is X. Possible accuracy measures are:
X-A         symmetric about accurate value =0, BIC astronomical ~2000
X/A         NOT symmetric  about accurate  value=1, BIC =57
Ln(X/A)  symmetric about accurate value =0, BIC = 11

Since df are all the same any improvement in BIC show model superiority. Want to strongly claim that ln(X/A) is ‘best’ measure of accuracy in this situation. The DATA should decide best measure

Because X/A is not very different from 1, the effect sizes &  p-values are quite similar for the ln(X/A) and X/A  analyses (cf expanding series for X/A about 1).

Realize now that I am making non-standard use of BIC as a test of normality.
Is this legitimate?

Several people said very firmly NO. BIC is ONLY for comparing nested models
Thanks Ric Ulric, Jo Pemberton and others. YOU ARE RIGHT , sorry for being stupid over this
BIC & other information criteria re useless for this purpose because they are NOT invariant under changes of scale.

The best way to check for normality is to SAVE the residuals & then check residual for normality.
Old ways are BEST!!
IN my example, ln(x/a) residuals have ns Kolmogorov-Smirnov, where x/a and x-a residuals had highly significant K-S. This solves my problem & hopefully is of some use to others. Shapiro-Wilks was significantly non-normal for all models, but with 390 observations, power was high. By eye deviation from normality was small but detectable in detrended Q-Q plot (max deviation 1 sd). For my particular problem, kurtosis was significantly > for normal, there is a reason for this

Thanks for everyone’s help

Best

Diana

---

Emeritus Professor Diana Kornbrot
email:  d.e.kornbrot@...    
 web:    http://dianakornbrot.wordpress.com/
Work
Department of Psychology
School of Life and Medical Sciences
University of Hertfordshire
College Lane, Hatfield, Hertfordshire AL10 9AB, UK
voice:   <a href="tel:%2B44%20%280%29%20170%20728%204626" value="[phone]" target="_blank">[phone]
Home
19 Elmhurst Avenue
London N2 0LT, UK
voice:   <a href="tel:%2B44%20%280%29%20208%20%C2%A0444%202081" value="[phone]" target="_blank">[phone]
mobile: <a href="tel:%2B44%20%280%29%20740%20318%201612" value="[phone]" target="_blank">[phone]