Bayesian Information Criterion & Checking for Normality

Posted by Kornbrot, Diana on Apr 16, 2013; 6:37pm
URL: http://spssx-discussion.165.s1.nabble.com/Bayesian-Information-Criterion-Checking-for-Normality-tp5719504.html

Dear Colleagues,

Some time back I posted following query
Can I use BIC to compare 3 different transformations of a dependent variable, here accuracy of time estimation?
I am examining ‘accuracy’ as function of 1 between and 2 repeated measures factors using SPSS ‘MIXED’.

True value is A, obtained participant estimate is X. Possible accuracy measures are:
X-A         symmetric about accurate value =0, BIC astronomical ~2000
X/A         NOT symmetric  about accurate  value=1, BIC =57
Ln(X/A)  symmetric about accurate value =0, BIC = 11

Since df are all the same any improvement in BIC show model superiority. Want to strongly claim that ln(X/A) is ‘best’ measure of accuracy in this situation. The DATA should decide best measure

Because X/A is not very different from 1, the effect sizes &  p-values are quite similar for the ln(X/A) and X/A  analyses (cf expanding series for X/A about 1).

Realize now that I am making non-standard use of BIC as a test of normality.
Is this legitimate?

Several people said very firmly NO. BIC is ONLY for comparing nested models
Thanks Ric Ulric, Jo Pemberton and others. YOU ARE RIGHT , sorry for being stupid over this
BIC & other information criteria re useless for this purpose because they are NOT invariant under changes of scale.

The best way to check for normality is to SAVE the residuals & then check residual for normality.
Old ways are BEST!!
IN my example, ln(x/a) residuals have ns Kolmogorov-Smirnov, where x/a and x-a residuals had highly significant K-S. This solves my problem & hopefully is of some use to others. Shapiro-Wilks was significantly non-normal for all models, but with 390 observations, power was high. By eye deviation from normality was small but detectable in detrended Q-Q plot (max deviation 1 sd). For my particular problem, kurtosis was significantly > for normal, there is a reason for this

Thanks for everyone’s help

Best

Diana

---

Emeritus Professor Diana Kornbrot
email:  d.e.kornbrot@...    
 web:    http://dianakornbrot.wordpress.com/
Work
Department of Psychology
School of Life and Medical Sciences
University of Hertfordshire
College Lane, Hatfield, Hertfordshire AL10 9AB, UK
voice:   [phone]
Home
19 Elmhurst Avenue
London N2 0LT, UK
voice:   [phone]
mobile: [phone]