All, A recent question about computing an ICC reminded me of a simulation experiment I conducted a while back to assess whether or not one could directly obtain 95% confidence limits about an ICC for a specific situation from the MIXED procedure. Here is the situation in which one can directly obtain 95% confidence limits:
An inter-rater reliability study is carried out under the following conditions: (1) exactly two raters are used [treated as a fixed effect], (2) subjects are sampled randomly from the population of interest, (3) both raters evaluate each subject, and (4) ratings are ordered categories.
Below my name is the simulation experiment. Note that the first MIXED code provides the ICC and 95% confidence limits in the "Estimates of Covariance Parameters" Table. The variance components [estimated from the second MIXED code] entered into the equation in the COMPUTE statement provide confirmation that specifying a first-order autoregressive residual covariance structure in the first MIXED code does in fact produce the ICC.
Hope this is of interest to others. Ryan -- *Generate data for MIXED model to estimate ICC and 95% Confidence Limits. set seed 65923454. new file. inp pro. compute subject=-99. compute rater = -99. compute x1 = -99. compute x2 = -99. compute e1 = -99. compute e2 = -99. compute sigma = 1. compute rho = 0.70. compute a11 = 1. compute a21 = rho. compute a22 = sqrt(1 - rho**2). leave subject to a22. loop subject= 1 to 1000. compute x1 = rv.normal(0,1).
compute x2 = rv.normal(0,1). compute e1 = sigma * a11*x1. compute e2 = sigma * (a21*x1 + a22*x2). loop rater = 1 to 2. compute y = 1.5 + 2.4*(rater=1) + e1*(rater=1) + e2*(rater=2). end case. end loop. end loop. end file. end inp pro. exe. delete variables x1 x2 sigma rho a11 a21 a22 e1 e2. MIXED y BY rater /FIXED=rater | SSTYPE(3) /METHOD=REML /PRINT= TESTCOV /REPEATED=rater | SUBJECT(subject) COVTYPE(AR1). MIXED y BY rater
/FIXED=rater | SSTYPE(3) /METHOD=REML /PRINT=G /RANDOM=intercept | SUBJECT(subject). COMPUTE ICC=0.7288263600198803 / (0.7288263600198803 + 0.3157454299761715). EXECUTE. |
Administrator
|
Hi Ryan. You could provide further confirmation as follows.
* To provide further confirmation, restructure the file * and use RELIABILITY to compute the ICC and 95% CI. FORMATS subject rater (f5.0). CASESTOVARS /ID=subject /INDEX=rater /FIXED = ICC /GROUPBY=VARIABLE. RELIABILITY /VARIABLES=y.1 y.2 /SCALE('ALL VARIABLES') ALL /MODEL=ALPHA /ICC=MODEL(MIXED) TYPE(CONSISTENCY) CIN=95 TESTVAL=0. Look at the output for the Single-Measures ICC.
--
Bruce Weaver bweaver@lakeheadu.ca http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." PLEASE NOTE THE FOLLOWING: 1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. 2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/). |
Free forum by Nabble | Edit this page |