Generalised Linear Mixed Models - analysing for change over time, not just group difference at any timepoint

[Carol Ma]

First up, I want to explain that I am a novice, so please go easy on me :) I am health behaviour researcher, and trying to upskill. The version of SPSS that I'm using is 22.

I'm attempting to analyse data from a 2 group randomised controlled trial (i.e. intervention vs control group, with 3 time points). A couple of years ago the statistician I was working with taught me how to do this in SPSS using the Generalised Linear Mixed Models function. Sadly, he has now passed away, so I can't ask him my questions.

I have run analyses on a few different datasets over the last several months, and some niggling questions are starting to come up in my head - I'm not confident anymore that the analysis is telling me what I thought it was. Being an RCT, I'm interested in whether the two groups (intervention group and control) are changing differently to one another, over the course of the intervention. I want it to be taking the groups' baseline values into account, when it analyses how they have changed across the intervention.

Briefly, here is how I'm running the analysis: In the data structure screen, I'm entering ID into the subjects canvas, and time into the repeated measures canvas. In the target, I'm setting it to the dependent variable (obviously!). In the fixed effects, I'm using time, group, and group x time (created by dragging group and time together into the 2-way box). In the random effects, I'm using ID. iN the build options menu, it is all the default settings. In the model options, I am setting it to provide estimates and pairwise contrasts. When I view the output, I am looking primarily at the group x time results.

Having seen outputs from multiple analyses using this method, I'm starting to get concerned that the outputs aren't actually telling me whether the two groups have changed differently from one another, but rather it is telling me simply whether the groups are different from one another at any given time point.

For example, by chance, for one outcomes, the control group started off considerably higher than the intervention at baseline. The control group tracked pretty similarly across all timepoints. In contrast, the intervention group increased a lot (based on the means) at the second and third time points (mid and end of intervention), so that their means at T2 and T3 were slightly higher than those of the control group. So, eyeballing it, the intervention has generally had the effect that we expected.

I was previously taught that to interpret the RCT results, the main p values of interest are those for timepoint 2 and timepoint 3, and to essentially disregard the p-value for timepoint 1.

However, for the particular example I give about, the timepoint 1 p value was near significant, whereas the timepoint 2 and 3 values are nowhere near significant (>0.9). To me, this is a red flag that the model isn't working in the way I believed it was - if it was "factoring in baseline values", the near significant p-value should be at T2, not T1, since the intervention group increased quite a lot from baseline, while the control group stayed essentially stable.

I'm starting to think that if I calculated the change scores, and then ran them in a repeated measures generalised linear mixed model (there would only then be two point points, instead of three) perhaps this would better answer my question - has the intervention group significantly changed across the intervention, relative to the control group? However, I'm worried that this is an incorrect way to do things, as a repeated measures analysis should be done using the raw values. Alternatively, is there another setting I should be using in the analysis so that the results are "anchored" to the baseline values? I hope this makes sense.

Thank you very much for any advice you are able to offer.

[Ryan]

Hello:

It isn't clear to me why you are fitting a generalized linear mixed model if your dependent variable is [presumably] conditionally normal. Unless told otherwise I will assume I am correct, and as a result I will also assume you should fit a linear mixed model via the MIXED procedure.

With that said, one approach typically employed with a standard RCT within a linear mixed modeling framework would be to incorporate (1) condition, (2) time, and (3) and condition-X-time as fixed effects, while allowing for within-subject temporally correlated errors by using an unstructured or perhaps more restrictive first-order autoregressive structure or compound symmetric structure. The fixed effect interaction term, condition-X-time, would be the omnibus test on whether one condition on average changed across time points significantly more than the other condition. The TEST sub-command could be used to perform post hoc/planned tests (e.g., diff between conditions from baseline to time 2, diff between conditions from baseline to time 3). 

It's also possible that there are conditional subject-specific intercepts while taking into account correlated errors via an autoregressive error structure, for example, which would necessitate incorporation of a random subject effect into the same model.

So, the following untested code might be a good starting point:

MIXED y BY condition time 
 /FIXED=condition time condition*time | SSTYPE(3)
 /METHOD=REML
 /PRINT=SOLUTION TESTCOV
 /RANDOM=intercept | SUBJECT(subjectID)
 /REPEATED=time | SUBJECT(subjectID) COVTYPE(AR1).

There are certainly other approaches/parameterizations that you may consider (e.g., treating baseline scores as a covariate).

Ryan

p.s., in the future it would be helpful if you (1) posted SPSS syntax with your questions, (2) described your dependent variable, and (3) gave us a name to call you...my preference :)

Sent from my iPhone

On May 29, 2015, at 6:24 PM, Carol Ma <[hidden email]> wrote:

First up, I want to explain that I am a novice, so please go easy on me :) I

am health behaviour researcher, and trying to upskill. The version of SPSS

that I'm using is 22.

I'm attempting to analyse data from a 2 group randomised controlled trial

(i.e. intervention vs control group, with 3 time points). A couple of years

ago the statistician I was working with taught me how to do this in SPSS

using the Generalised Linear Mixed Models function. Sadly, he has now passed

away, so I can't ask him my questions.

I have run analyses on a few different datasets over the last several

months, and some niggling questions are starting to come up in my head - I'm

not confident anymore that the analysis is telling me what I thought it was.

Being an RCT, I'm interested in whether the two groups (intervention group

and control) are /changing differently to one another/, over the course of

the intervention. I want it to be taking the groups' baseline values into

account, when it analyses how they have changed across the intervention.

Briefly, here is how I'm running the analysis: In the data structure screen,

I'm entering ID into the subjects canvas, and time into the repeated

measures canvas. In the target, I'm setting it to the dependent variable

(obviously!). In the fixed effects, I'm using time, group, and group x time

(created by dragging group and time together into the 2-way box). In the

random effects, I'm using ID. iN the build options menu, it is all the

default settings. In the model options, I am setting it to provide estimates

and pairwise contrasts. When I view the output, I am looking primarily at

the group x time results.

Having seen outputs from multiple analyses using this method, I'm starting

to get concerned that the outputs aren't actually telling me whether the two

groups have /changed differently from one another/, but rather it is telling

me simply whether the groups are different from one another at /any given

time point/.

For example, by chance, for one outcomes, the control group started off

considerably higher than the intervention at baseline. The control group

tracked pretty similarly across all timepoints. In contrast, the

intervention group increased a lot (based on the means) at the second and

third time points (mid and end of intervention), so that their means at T2

and T3 were slightly higher than those of the control group. So, eyeballing

it, the intervention has generally had the effect that we expected.

I was previously taught that to interpret the RCT results, the main p values

of interest are those for timepoint 2 and timepoint 3, and to essentially

disregard the p-value for timepoint 1.

However, for the particular example I give about, the timepoint 1 p value

was near significant, whereas the timepoint 2 and 3 values are nowhere near

significant (>0.9). To me, this is a red flag that the model isn't working

in the way I believed it was - if it was "factoring in baseline values", the

near significant p-value should be at T2, not T1, since the intervention

group increased quite a lot from baseline, while the control group stayed

essentially stable.

I'm starting to think that if I calculated the /change/ scores, and then ran

them in a repeated measures generalised linear mixed model (there would only

then be two point points, instead of three) perhaps this would better answer

my question - has the intervention group significantly changed across the

intervention, relative to the control group? However, I'm worried that this

is an incorrect way to do things, as a repeated measures analysis should be

done using the raw values. Alternatively, is there another setting I should

be using in the analysis so that the results are "anchored" to the baseline

values? I hope this makes sense.

Thank you very much for any advice you are able to offer.

--

View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Generalised-Linear-Mixed-Models-analysing-for-change-over-time-not-just-group-difference-at-any-timet-tp5729647.html

Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================

To manage your subscription to SPSSX-L, send a message to

[hidden email] (not to SPSSX-L), with no body text except the

command. To leave the list, send the command

SIGNOFF SPSSX-L

For a list of commands to manage subscriptions, send the command

INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Carol Ma]

Thanks Ryan,

I will give that a try. I used generalised linear mixed model as the variable is right skewed. However I thought generalised linear mixed model was also suitable for normally distributed variables?

OK, about to go and try the syntax you suggested. Thanks a lot for taking the time to reply and explain things to me - much appreciated.

Carol

[Bruce Weaver]

Notice that Ryan was talking about the dependent variable being conditionally normal.  In other words, the assumption is that the errors are normally distributed, not that the dependent variable is normally distributed.  

HTH.

Carol Ma wrote

Thanks Ryan,

I will give that a try. I used generalised linear mixed model as the variable is right skewed. However I thought generalised linear mixed model was also suitable for normally distributed variables?

OK, about to go and try the syntax you suggested. Thanks a lot for taking the time to reply and explain things to me - much appreciated.

Carol