Bayesian Information Criterion & Checking for Normality
Locked
Bayesian Information Criterion & Checking for Normality
 
|
Some time back I posted following query Can I use BIC to compare 3 different transformations of a dependent variable, here accuracy of time estimation? I am examining ‘accuracy’ as function of 1 between and 2 repeated measures factors using SPSS ‘MIXED’. True value is A, obtained participant estimate is X. Possible accuracy measures are: X-A symmetric about accurate value =0, BIC astronomical ~2000 X/A NOT symmetric about accurate value=1, BIC =57 Ln(X/A) symmetric about accurate value =0, BIC = 11 Since df are all the same any improvement in BIC show model superiority. Want to strongly claim that ln(X/A) is ‘best’ measure of accuracy in this situation. The DATA should decide best measure Because X/A is not very different from 1, the effect sizes & p-values are quite similar for the ln(X/A) and X/A analyses (cf expanding series for X/A about 1). Realize now that I am making non-standard use of BIC as a test of normality. Is this legitimate? Several people said very firmly NO. BIC is ONLY for comparing nested models Thanks Ric Ulric, Jo Pemberton and others. YOU ARE RIGHT , sorry for being stupid over this BIC & other information criteria re useless for this purpose because they are NOT invariant under changes of scale. The best way to check for normality is to SAVE the residuals & then check residual for normality. Old ways are BEST!! IN my example, ln(x/a) residuals have ns Kolmogorov-Smirnov, where x/a and x-a residuals had highly significant K-S. This solves my problem & hopefully is of some use to others. Shapiro-Wilks was significantly non-normal for all models, but with 390 observations, power was high. By eye deviation from normality was small but detectable in detrended Q-Q plot (max deviation 1 sd). For my particular problem, kurtosis was significantly > for normal, there is a reason for this Thanks for everyone’s help Best Diana Emeritus Professor Diana Kornbrot email: d.e.kornbrot@... web: http://dianakornbrot.wordpress.com/ Work Department of Psychology School of Life and Medical Sciences University of Hertfordshire College Lane, Hatfield, Hertfordshire AL10 9AB, UK voice: +44 (0) 170 728 4626 Home 19 Elmhurst Avenue London N2 0LT, UK voice: +44 (0) 208 444 2081 mobile: +44 (0) 740 318 1612 |
Re: Bayesian Information Criterion & Checking for Normality
|
|
My intuition is that with different dependent variables, a comparison of BIC values would be meaningless. David Greenberg, Sociology Department, New York University On Tue, Apr 16, 2013 at 2:37 PM, Kornbrot, Diana <[hidden email]> wrote:
|
|
|
In reply to this post by Kornbrot, Diana
I just came across this post. It is true that one cannot compare statistical models using the BIC when the response variable is not the same. Having said that, the BIC, and AIC for that matter, may be used for comparison of non-nested models where the response variable is the same. It is also possible to perform a statistical test, called the "Vuong test," to compare non-nested models (again assuming the response variable is the same), which may or may not include an AIC or BIC correction.
Ryan On Tue, Apr 16, 2013 at 2:37 PM, Kornbrot, Diana <[hidden email]> wrote:
|
|
|
Needless to say, or perhaps not needless to say, comparisons of non-nested models via the AIC, BIC, and/or the Vuong statistic require that the same data be used; that is, the dependent variable values must be exactly the same, and the same valid cases used to fit one model must also be used to fit the other model.
Ryan On Fri, Apr 19, 2013 at 9:20 PM, R B <[hidden email]> wrote:
|
«
Return to SPSSX Discussion
|
1 view|%1 views
| Free forum by Nabble | Edit this page |