We have a repeated measures design for 15 subjects. Subjects undergo the same testing procedure on 3 different occasions. We want to know how reproducible are the test scores across all 3 sessions.
I've read references to the traditional ANOVA method of calculating the ICC (Shrout & Fleiss 1999; McGraw and Wong 1996). Where a large ICC reflects the test-retest or relability/sameness between 3 test sessions.
I've also seen calculations from linear mixed models such as:
ICC = var_between_subjects/(var_between_subjects + var_residual). Here, the ICC is the proportion of variance attributed to var_between relative to the total variance. Although in this case, is it testing how reliable the scores are over the 3 testing sessions? It seems a high ICC value in this case is determining how different the testing scores are among the 3 testing sessions. The inverse of the traditional ANOVA ICC. Is this a correct interpretation?
I ask because we have a time-varying covariate we would also like to include in the analysis. Based on the example in the following link:
http://www.ats.ucla.edu/stat/spss/faq/repeat_covar.htmIt is suggested we can only use a linear mixed model if we want to include a time-varying covariate. If so, is the above calculation correct for the ICC? Or is there more to it. Is the interpretation of the result the same in comparison to the ICC from the traditional ANOVA model?
Also, is there a way to distinguish between absolute agreement and consistency when calculating ICC from a linear mixed model? How does the selection of the covariance structure for the repeated measures factor into calculating the ICC?
Any thoughts, ideas, suggested papers/citations, corrections, etc. would help a great deal.
Thanks in advance
h.
Locked
