Why is 95% CI for ICC so large?

[J McClure]

Hi,
I have run ICC's with a 95% CI to compare agreement self reported rating
by the patient and the doctor's rating by another characteristic. For
example, I divided the sample between the patient's who had major
depression and those without major depression and determined the
agreement of the MD's with patients with major depression and separately
agreement with patient's without major depression.
The ICC for those without depression (n=153) was 0.43 (95%CI 0.23, 0.52
) and for those with depression (n=224) it was 0.43 (CI 0.04, 0.66).
Why is the CI for the second group so much wider than for the first? The
ICC syntax I am using is   /ICC=model (random) type(absolute).
Thanks for any help,
Jan

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[David Marso]

Impossible to say without your specific data!
Try to locate the ICC algorithms on the SPSS site and study the elements in the CI's.
I always have terrible luck locating anything there besides marketing fluff.  I am registered but it doesn't seem to recognize my password.  Then it demands that I accept cookies from spss.com ( I checked my cookies and there is one for spss.com ).   So, AFAIAC the site is a PITA, and too much hassle for me to bother at the moment!
So...I leave that hassle to you!  Probably has something to do with Big Blue?
-------
OTOH:  My intuitions?
The non-depressed group has a consistent moderate agreement, the depressed group has a mix of very close agreement and considerable disparity.
Since we don't know your data or the structural specifics it is really not possible to answer your question.
"ICC syntax I am using is   /ICC=model (random) type(absolute)."
That is really *not* enough to guess what is going on.
Means, Covariances, SDs? SPECIFIC syntax???
HTH, David
 

J McClure wrote

Hi,
I have run ICC's with a 95% CI to compare agreement self reported rating
by the patient and the doctor's rating by another characteristic. For
example, I divided the sample between the patient's who had major
depression and those without major depression and determined the
agreement of the MD's with patients with major depression and separately
agreement with patient's without major depression.
The ICC for those without depression (n=153) was 0.43 (95%CI 0.23, 0.52
) and for those with depression (n=224) it was 0.43 (CI 0.04, 0.66).
Why is the CI for the second group so much wider than for the first? The
ICC syntax I am using is   /ICC=model (random) type(absolute).
Thanks for any help,
Jan

=====================
To manage your subscription to SPSSX-L, send a message to
[e-mail] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Rich Ulrich]

The main difference between a simple ICC and a Pearson R is that
the ICC *assumes*  that the means are the same.  The result is a
smaller value for the ICC than for the Pearson when the means differ.

I expect that your ICC for the depressed group has a wider CI
because of an underlying mean difference, since that is the only
potential difference I can think of, to account for the discrepancy.
The N's are actually larger for the depressed group, which would
tend to give a smaller CI.  Otherwise -- Are you sure that all the
data were included in the analysis, nothing Missing?

With two raters, I prefer to look at SPSS's paired t-test between raters.

That shows me both the mean difference and the direct correlation.



--
Rich Ulrich

----------------------------------------

> Date: Wed, 9 Mar 2011 20:53:52 -0800
> From: [hidden email]
> Subject: Why is 95% CI for ICC so large?
> To: [hidden email]
>
> Hi,
> I have run ICC's with a 95% CI to compare agreement self reported rating
> by the patient and the doctor's rating by another characteristic. For
> example, I divided the sample between the patient's who had major
> depression and those without major depression and determined the
> agreement of the MD's with patients with major depression and separately
> agreement with patient's without major depression.
> The ICC for those without depression (n=153) was 0.43 (95%CI 0.23, 0.52
> ) and for those with depression (n=224) it was 0.43 (CI 0.04, 0.66).
> Why is the CI for the second group so much wider than for the first? The
> ICC syntax I am using is /ICC=model (random) type(absolute).
> Thanks for any help,
> Jan

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[J McClure]

Thanks RIch and David for the help!
Jan

On 3/10/2011 3:28 PM, Rich Ulrich wrote:

> The main difference between a simple ICC and a Pearson R is that
> the ICC *assumes*  that the means are the same.  The result is a
> smaller value for the ICC than for the Pearson when the means differ.
>
> I expect that your ICC for the depressed group has a wider CI
> because of an underlying mean difference, since that is the only
> potential difference I can think of, to account for the discrepancy.
> The N's are actually larger for the depressed group, which would
> tend to give a smaller CI.  Otherwise -- Are you sure that all the
> data were included in the analysis, nothing Missing?
>
> With two raters, I prefer to look at SPSS's paired t-test between raters.
>
> That shows me both the mean difference and the direct correlation.
>
>
>
> --
> Rich Ulrich
>
> ----------------------------------------
>> Date: Wed, 9 Mar 2011 20:53:52 -0800
>> From: [hidden email]
>> Subject: Why is 95% CI for ICC so large?
>> To: [hidden email]
>>
>> Hi,
>> I have run ICC's with a 95% CI to compare agreement self reported rating
>> by the patient and the doctor's rating by another characteristic. For
>> example, I divided the sample between the patient's who had major
>> depression and those without major depression and determined the
>> agreement of the MD's with patients with major depression and separately
>> agreement with patient's without major depression.
>> The ICC for those without depression (n=153) was 0.43 (95%CI 0.23, 0.52
>> ) and for those with depression (n=224) it was 0.43 (CI 0.04, 0.66).
>> Why is the CI for the second group so much wider than for the first? The
>> ICC syntax I am using is /ICC=model (random) type(absolute).
>> Thanks for any help,
>> Jan
>
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD