Building a hybrid regression model incorporating intraclass correlation

[Vaughan, Christopher]

Hello Everyone,

 

We am trying to tackle an issue related to developing a hybrid regression formula that combines both intra-class (i.e., correlating the same variable) and inter-class coefficients (e.g., Pearson) in the same regression model.  The purpose is to develop a regression line to predict Test Score at Time 2, using Test Score at Time 1 along with other demographic factors.   Our thought is that it is appropriate to use intra-class correlation coefficients in the model for the Time 1 Test Score (correlating the same type of variable), and when using various demographic factors (age, sex) variables in the prediction equation, the appropriate statistical method would be inter-class correlation as they are different variable types than the outcome variable (Test Score at Time 2). It is my understanding that the SPSS statistical models only use the Pearson - inter-class - method of relating predictor variables to outcome variables.

 

So here is where the question/ issue arises. The ultimate predictor variable, Test score at Time 1, is in fact an intra-class variable as it is the same type as the outcome variable, but we believe the SPSS regression methodology treats Test score at Time 1 statistically as an inter-class variable and calculates its correlation in the multivariate equation as inter-class. As a result, the prediction equation of the Outcome variable is using a lower test-retest coefficient and therefore generating a larger confidence band around the Test Score at Time 2 variable. It seems to us that this is not the appropriate statistical method.

 

First, is our argument appropriate/ cogent?

 

Second, if so, how do we correct the regression equation to be "hybrid" allowing intra-class correlation for the Test score at Time 1 variable, and inter-class for the other predictor variables?

 

Any and all of your thoughts would be much appreciated.

 

Thanks.

 

 

Christopher G. Vaughan, PsyD

Pediatric Neuropsychologist

Division of Pediatric Neuropsychology, Children's National Medical Center

Assistant Professor, Depts of Psychiatry and Behavioral Sciences & Pediatrics

The George Washington University School of Medicine

Safe Concussion Outcome, Recovery and Education (SCORE) Clinic: [phone]

Neuropsychology Dept: [phone]

Fax:    [phone]

 

[Garry Gelade]

Dear Christopher

 

The term ‘intra-class correlation’ refers to grouped data and is a measure of the proportion of variance attributable to group membership.  It has nothing to do with the fact that the time1 test score is the” same type of variable” as the time2 test score.

 

For the analysis you are proposing, you can treat the time1 score in just the same way as age or gender, it is just another predictor of the time2 score.

 

Garry Gelade

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Vaughan, Christopher
Sent: 16 September 2011 14:11
To: [hidden email]
Subject: Building a hybrid regression model incorporating intraclass correlation

 

Hello Everyone,

 

We am trying to tackle an issue related to developing a hybrid regression formula that combines both intra-class (i.e., correlating the same variable) and inter-class coefficients (e.g., Pearson) in the same regression model.  The purpose is to develop a regression line to predict Test Score at Time 2, using Test Score at Time 1 along with other demographic factors.   Our thought is that it is appropriate to use intra-class correlation coefficients in the model for the Time 1 Test Score (correlating the same type of variable), and when using various demographic factors (age, sex) variables in the prediction equation, the appropriate statistical method would be inter-class correlation as they are different variable types than the outcome variable (Test Score at Time 2). It is my understanding that the SPSS statistical models only use the Pearson - inter-class - method of relating predictor variables to outcome variables.

 

So here is where the question/ issue arises. The ultimate predictor variable, Test score at Time 1, is in fact an intra-class variable as it is the same type as the outcome variable, but we believe the SPSS regression methodology treats Test score at Time 1 statistically as an inter-class variable and calculates its correlation in the multivariate equation as inter-class. As a result, the prediction equation of the Outcome variable is using a lower test-retest coefficient and therefore generating a larger confidence band around the Test Score at Time 2 variable. It seems to us that this is not the appropriate statistical method.

 

First, is our argument appropriate/ cogent?

 

Second, if so, how do we correct the regression equation to be "hybrid" allowing intra-class correlation for the Test score at Time 1 variable, and inter-class for the other predictor variables?

 

Any and all of your thoughts would be much appreciated.

 

Thanks.

 

 

Christopher G. Vaughan, PsyD

Pediatric Neuropsychologist

Division of Pediatric Neuropsychology, Children's National Medical Center

Assistant Professor, Depts of Psychiatry and Behavioral Sciences & Pediatrics

The George Washington University School of Medicine

Safe Concussion Outcome, Recovery and Education (SCORE) Clinic: [phone]

Neuropsychology Dept: [phone]

Fax:    [phone]

 

[Bruce Weaver]

It seems to me that there are (at least) two different uses of the ICC.  

1. It is used as a reliability coefficient (e.g., see Health Measurement Scales by Streiner & Norman).  
2. It is used in the way Garry describes, where it is often called the intra-cluster correlation.  

For the first use, one should have the same variable measured twice (in the same units) IMO.

Cheers!
Bruce

Garry Gelade wrote

Dear Christopher



The term 'intra-class correlation' refers to grouped data and is a measure
of the proportion of variance attributable to group membership.  It has
nothing to do with the fact that the time1 test score is the" same type of
variable" as the time2 test score.



For the analysis you are proposing, you can treat the time1 score in just
the same way as age or gender, it is just another predictor of the time2
score.



Garry Gelade



From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Vaughan, Christopher
Sent: 16 September 2011 14:11
To: [hidden email]
Subject: Building a hybrid regression model incorporating intraclass
correlation



Hello Everyone,



We am trying to tackle an issue related to developing a hybrid regression
formula that combines both intra-class (i.e., correlating the same variable)
and inter-class coefficients (e.g., Pearson) in the same regression model.
The purpose is to develop a regression line to predict Test Score at Time 2,
using Test Score at Time 1 along with other demographic factors.   Our
thought is that it is appropriate to use intra-class correlation
coefficients in the model for the Time 1 Test Score (correlating the same
type of variable), and when using various demographic factors (age, sex)
variables in the prediction equation, the appropriate statistical method
would be inter-class correlation as they are different variable types than
the outcome variable (Test Score at Time 2). It is my understanding that the
SPSS statistical models only use the Pearson - inter-class - method of
relating predictor variables to outcome variables.



So here is where the question/ issue arises. The ultimate predictor
variable, Test score at Time 1, is in fact an intra-class variable as it is
the same type as the outcome variable, but we believe the SPSS regression
methodology treats Test score at Time 1 statistically as an inter-class
variable and calculates its correlation in the multivariate equation as
inter-class. As a result, the prediction equation of the Outcome variable is
using a lower test-retest coefficient and therefore generating a larger
confidence band around the Test Score at Time 2 variable. It seems to us
that this is not the appropriate statistical method.



First, is our argument appropriate/ cogent?



Second, if so, how do we correct the regression equation to be "hybrid"
allowing intra-class correlation for the Test score at Time 1 variable, and
inter-class for the other predictor variables?



Any and all of your thoughts would be much appreciated.



Thanks.





Christopher G. Vaughan, PsyD

Pediatric Neuropsychologist

Division of Pediatric Neuropsychology, Children's National Medical Center

Assistant Professor, Depts of Psychiatry and Behavioral Sciences &
Pediatrics

The George Washington University School of Medicine

Safe Concussion Outcome, Recovery and Education (SCORE) Clinic: (202)
476-2429

Neuropsychology Dept: [phone]

Fax:    [phone]

[Vaughan, Christopher]

Great point of clarification Bruce, and thank you.

Yes, the intention is to build a prediction model in order to guide
decisions about reliable change.  It is in fact the exact same variable
that we are measuring twice in the same person that we want to assess.
It seems that in this situation, the ICC is more appropriate measure of
association (rather than a Pearson r) but it is not typically used in a
regression model.  Does this make sense, and if so, how do we build a
model that would include it?

Thanks,
CV


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Bruce Weaver
Sent: Friday, September 16, 2011 12:06 PM
To: [hidden email]
Subject: Re: Building a hybrid regression model incorporating intraclass
correlation

It seems to me that there are (at least) two different uses of the ICC.

1. It is used as a reliability coefficient (e.g., see Health Measurement
Scales by Streiner & Norman).
2. It is used in the way Garry describes, where it is often called the
intra-cluster correlation.

For the first use, one should have the same variable measured twice (in
the
same units) IMO.

Cheers!
Bruce



Garry Gelade wrote:
>
> Dear Christopher
>
>
>
> The term 'intra-class correlation' refers to grouped data and is a
measure
> of the proportion of variance attributable to group membership.  It
has
> nothing to do with the fact that the time1 test score is the" same
type of
> variable" as the time2 test score.
>
>
>
> For the analysis you are proposing, you can treat the time1 score in
just
> the same way as age or gender, it is just another predictor of the
time2
> score.
>
>
>
> Garry Gelade
>
>
>
> From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf
Of

> Vaughan, Christopher
> Sent: 16 September 2011 14:11
> To: [hidden email]
> Subject: Building a hybrid regression model incorporating intraclass
> correlation
>
>
>
> Hello Everyone,
>
>
>
> We am trying to tackle an issue related to developing a hybrid

regression
> formula that combines both intra-class (i.e., correlating the same
> variable)
> and inter-class coefficients (e.g., Pearson) in the same regression
model.
> The purpose is to develop a regression line to predict Test Score at
Time
> 2,
> using Test Score at Time 1 along with other demographic factors.   Our
> thought is that it is appropriate to use intra-class correlation
> coefficients in the model for the Time 1 Test Score (correlating the
same
> type of variable), and when using various demographic factors (age,
sex)
> variables in the prediction equation, the appropriate statistical
method
> would be inter-class correlation as they are different variable types
than
> the outcome variable (Test Score at Time 2). It is my understanding
that
> the
> SPSS statistical models only use the Pearson - inter-class - method of
> relating predictor variables to outcome variables.
>
>
>
> So here is where the question/ issue arises. The ultimate predictor
> variable, Test score at Time 1, is in fact an intra-class variable as
it
> is
> the same type as the outcome variable, but we believe the SPSS
regression
> methodology treats Test score at Time 1 statistically as an
inter-class
> variable and calculates its correlation in the multivariate equation
as
> inter-class. As a result, the prediction equation of the Outcome
variable
> is
> using a lower test-retest coefficient and therefore generating a
larger
> confidence band around the Test Score at Time 2 variable. It seems to
us
> that this is not the appropriate statistical method.
>
>
>
> First, is our argument appropriate/ cogent?
>
>
>
> Second, if so, how do we correct the regression equation to be
"hybrid"
> allowing intra-class correlation for the Test score at Time 1
variable,

> and
> inter-class for the other predictor variables?
>
>
>
> Any and all of your thoughts would be much appreciated.
>
>
>
> Thanks.
>
>
>
>
>
> Christopher G. Vaughan, PsyD
>
> Pediatric Neuropsychologist
>
> Division of Pediatric Neuropsychology, Children's National Medical

Center

>
> Assistant Professor, Depts of Psychiatry and Behavioral Sciences &
> Pediatrics
>
> The George Washington University School of Medicine
>
> Safe Concussion Outcome, Recovery and Education (SCORE) Clinic: (202)
> 476-2429
>
> Neuropsychology Dept: [phone]
>
> Fax:    [phone]
>

-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context:
http://spssx-discussion.1045642.n5.nabble.com/Building-a-hybrid-regressi
on-model-incorporating-intraclass-correlation-tp4810791p4811154.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Rich Ulrich]

No, I don't see how it makes any particular sense.  You don't offer
a rationale, and I don't see one, for computing covariances around
the pooled mean. Especially if you are expecting that mean to
have changed over time.

Technically, the difference between inter- and intra-class is
that you use the separate means when you computer an interclass r,
and you use a pooled estimate of means for an intraclass r.   There
are several versions of ICCs because there are several distinct versions
of what you may elect to pool, for more complex designs.

I remember computing an ICC 40 years ago by duplicating a set
of X,Y data, and reversing the order for the second set (to Y, X),
so that a simple Pearson would estimate the intraclass r by
forcing the means to be the pooled value. 

--
Rich Ulrich

> Date: Fri, 16 Sep 2011 12:20:19 -0400

> From: [hidden email]
> Subject: Re: Building a hybrid regression model incorporating intraclass correlation
> To: [hidden email]
>
> Great point of clarification Bruce, and thank you.
>
> Yes, the intention is to build a prediction model in order to guide
> decisions about reliable change. It is in fact the exact same variable
> that we are measuring twice in the same person that we want to assess.
> It seems that in this situation, the ICC is more appropriate measure of
> association (rather than a Pearson r) but it is not typically used in a
> regression model. Does this make sense, and if so, how do we build a
> model that would include it?
>

[snip, previous]