Analysis of covariance: Different results using UNIANOVA or MIXED

[Frank Furter]

Dear Colleagues:

I am analyzing data from a crossover study using ANOVA or ANCOVA. The
dependent variable is the difference between baseline and end of treatment
for some physiological measure. With UNIANOVA my syntax looks like this:

UNIANOVA Outcome BY Period Sequence Product Subject
  /RANDOM=Subject
  /METHOD=SSTYPE(3)
  /INTERCEPT=INCLUDE
  /DESIGN=Sequence Period Product Subject(Sequence).

This gives me exactly the same results as

MIXED Outcome BY Period Sequence Product Subject
  /FIXED=Sequence Period Product | SSTYPE(3)
  /METHOD=REML
  /print=solution
  /REPEATED=Product | SUBJECT(Subject) COVTYPE(UN).

However, when I add the baseline value of the outcome measure as a
covariate, my syntax looks like this:

UNIANOVA Outcome BY Period Sequence Product Subject with Baseline
  /RANDOM=Subject
  /METHOD=SSTYPE(3)
  /INTERCEPT=INCLUDE
  /DESIGN=Sequence Period Product Subject(Sequence) Baseline.

vs.

MIXED Outcome BY Period Sequence Product Subject with Baseline
  /FIXED=Sequence Period Product Baseline | SSTYPE(3)
  /METHOD=REML
  /print=solution
  /REPEATED=Product | SUBJECT(Subject) COVTYPE(UN).

and the results of the two procedures are completely different.

What am I doing wrong? What is the difference between my two ANCOVA models?

Thank you,
Andreas



--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Kirill Orlov]

There is at least the difference in METHOD. MIXED uses REML or ML to estimate variance components. UNIANOVA/GLM computes them from expected mean squares (ANOVA method). With designes more or less complex, that may produce difference.

12.10.2017 20:08, AndreasV пишет:

Dear Colleagues:

I am analyzing data from a crossover study using ANOVA or ANCOVA. The
dependent variable is the difference between baseline and end of treatment
for some physiological measure. With UNIANOVA my syntax looks like this:

UNIANOVA Outcome BY Period Sequence Product Subject
  /RANDOM=Subject
  /METHOD=SSTYPE(3)
  /INTERCEPT=INCLUDE
  /DESIGN=Sequence Period Product Subject(Sequence).

This gives me exactly the same results as

MIXED Outcome BY Period Sequence Product Subject
  /FIXED=Sequence Period Product | SSTYPE(3)
  /METHOD=REML
  /print=solution
  /REPEATED=Product | SUBJECT(Subject) COVTYPE(UN).

However, when I add the baseline value of the outcome measure as a
covariate, my syntax looks like this:

UNIANOVA Outcome BY Period Sequence Product Subject with Baseline
  /RANDOM=Subject
  /METHOD=SSTYPE(3)
  /INTERCEPT=INCLUDE
  /DESIGN=Sequence Period Product Subject(Sequence) Baseline.

vs.

MIXED Outcome BY Period Sequence Product Subject with Baseline
  /FIXED=Sequence Period Product Baseline | SSTYPE(3)
  /METHOD=REML
  /print=solution
  /REPEATED=Product | SUBJECT(Subject) COVTYPE(UN).

and the results of the two procedures are completely different.

What am I doing wrong? What is the difference between my two ANCOVA models?

Thank you,
Andreas



--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Frank Furter]

Kirill Orlov wrote
> There is at least the difference in METHOD. MIXED uses REML or ML to
> estimate variance components. UNIANOVA/GLM computes them from expected
> mean squares (ANOVA method). With designes more or less complex, that
> may produce difference.

In this particular case, the results differ substantially depending on
whether I use the MIXED or the UNIANOVA procedure (and only if baseline is
added as a covariate). Apart from the fact that the exact method to be used
should be prespecified in a confirmatory investigation, which of the two
methods would be the more appropriate one for the analysis of a crossover
trial?




--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Rich Ulrich]

It bothers me to see Baseline as a covariate when it was used to compute each Outcome.

Please clarify some points.  

Two conditions?

How many Periods are there for each condition? 

Do data exist as, say, Baseline, A1, A2, B1, B2    for treatments A and B?  (and not where Baseline is the A1 period)

DO the widely different results include matching d.f.'s? 

Were there any warning messages as part of the output for either?

--

Rich Ulrich

---

From: SPSSX(r) Discussion <[hidden email]> on behalf of AndreasV <[hidden email]>
Sent: Thursday, October 12, 2017 1:08:17 PM
To: [hidden email]
Subject: Analysis of covariance: Different results using UNIANOVA or MIXED

 

Dear Colleagues:

I am analyzing data from a crossover study using ANOVA or ANCOVA. The
dependent variable is the difference between baseline and end of treatment
for some physiological measure. With UNIANOVA my syntax looks like this:

UNIANOVA Outcome BY Period Sequence Product Subject
  /RANDOM=Subject
  /METHOD=SSTYPE(3)
  /INTERCEPT=INCLUDE
  /DESIGN=Sequence Period Product Subject(Sequence).

This gives me exactly the same results as

MIXED Outcome BY Period Sequence Product Subject
  /FIXED=Sequence Period Product | SSTYPE(3)
  /METHOD=REML
  /print=solution
  /REPEATED=Product | SUBJECT(Subject) COVTYPE(UN).

However, when I add the baseline value of the outcome measure as a
covariate, my syntax looks like this:

UNIANOVA Outcome BY Period Sequence Product Subject with Baseline
  /RANDOM=Subject
  /METHOD=SSTYPE(3)
  /INTERCEPT=INCLUDE
  /DESIGN=Sequence Period Product Subject(Sequence) Baseline.

vs.

MIXED Outcome BY Period Sequence Product Subject with Baseline
  /FIXED=Sequence Period Product Baseline | SSTYPE(3)
  /METHOD=REML
  /print=solution
  /REPEATED=Product | SUBJECT(Subject) COVTYPE(UN).

and the results of the two procedures are completely different.

What am I doing wrong? What is the difference between my two ANCOVA models?

Thank you,
Andreas



--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Frank Furter]

The study was a simple two-treatment two-period crossover design with
(baseline A -> drug A) followed by (baseline B -> drug B) or vice versa in
randomized order (i.e., two sewuences). There is a regulatory guideline that
needs to be followed and which specifies the following requirements:

"When the primary analysis is based on a continuous outcome there is
commonly the choice of whether to use the raw outcome variable or the change
from baseline as the primary endpoint. Whichever of these endpoints is
chosen, the baseline value should be included as a covariate in the primary
analysis. The use of change from baseline with adjustment for baseline is
generally more precise than change of baseline without adjustment. Note that
when the baseline is included as a covariate in a standard linear model, the
estimated treatment effects are identical for both ‘change from baseline’
(on an additive scale) and the ‘raw outcome’ analysis. Consequently if the
appropriate adjustment is done, then the choice of endpoint becomes solely
an issue of interpretability."

I first did the analysis using treatment as a factor and change from
baseline as the dependent variable, and the results for UNIANOVA and MIXED
were identical. Then I added the baseline value as a covariate using the
syntax above. Now the d.f.s for all main effects were different between
UNIANOVA and MIXED, and so were the F and p-values.

Andreas



--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

Andreas, providing a small sample dataset (e.g., via DATA LIST) that
illustrates the problem will increase the likelihood that someone will help
you work out what is going on.  

Cheers,
Bruce


AndreasV wrote

> The study was a simple two-treatment two-period crossover design with
> (baseline A -> drug A) followed by (baseline B -> drug B) or vice versa in
> randomized order (i.e., two sewuences). There is a regulatory guideline
> that
> needs to be followed and which specifies the following requirements:
>
> "When the primary analysis is based on a continuous outcome there is
> commonly the choice of whether to use the raw outcome variable or the
> change
> from baseline as the primary endpoint. Whichever of these endpoints is
> chosen, the baseline value should be included as a covariate in the
> primary
> analysis. The use of change from baseline with adjustment for baseline is
> generally more precise than change of baseline without adjustment. Note
> that
> when the baseline is included as a covariate in a standard linear model,
> the
> estimated treatment effects are identical for both ‘change from baseline’
> (on an additive scale) and the ‘raw outcome’ analysis. Consequently if the
> appropriate adjustment is done, then the choice of endpoint becomes solely
> an issue of interpretability."
>
> I first did the analysis using treatment as a factor and change from
> baseline as the dependent variable, and the results for UNIANOVA and MIXED
> were identical. Then I added the baseline value as a covariate using the
> syntax above. Now the d.f.s for all main effects were different between
> UNIANOVA and MIXED, and so were the F and p-values.
>
> Andreas
>
>
>
> --
> Sent from: http://spssx-discussion.1045642.n5.nabble.com/
>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Rich Ulrich]

Thanks for the detail. Here is what I gather:

The difference in d.f. in the results says that the two models are different in hypotheses, for whatever reason.

I asked about "warnings" because I did not know what the choices were for periods, and whether

the ANOVA might have found a redundant column in its matrix.  From the note you provide on using

Baseline as covariate, I gather that you are presenting only two "times" to the analyses.  The "Period"

variable in the analyses is a dichotomy: it shows the 2nd vs. 4th data point, where the two Baselines

are 1 and 3.  That looks okay. 

For these programs: I don't know how you test for equality of regression coefficients.  Or for

no-differences at baseline.  Those are among the assumptions for Crossover.

SSType3 was never my default, but if you have equal Ns,that should make no difference.

What I can offer next is a question of whether dropping "Intercept" helps in the covariate model; and,

what options exist that describe what to do with covariates in "Repeated"?

--

Rich Ulrich

---

From: SPSSX(r) Discussion <[hidden email]> on behalf of AndreasV <[hidden email]>
Sent: Thursday, October 19, 2017 4:18:11 PM
To: [hidden email]
Subject: Re: Analysis of covariance: Different results using UNIANOVA or MIXED

 

The study was a simple two-treatment two-period crossover design with
(baseline A -> drug A) followed by (baseline B -> drug B) or vice versa in
randomized order (i.e., two sewuences). There is a regulatory guideline that
needs to be followed and which specifies the following requirements:

"When the primary analysis is based on a continuous outcome there is
commonly the choice of whether to use the raw outcome variable or the change
from baseline as the primary endpoint. Whichever of these endpoints is
chosen, the baseline value should be included as a covariate in the primary
analysis. The use of change from baseline with adjustment for baseline is
generally more precise than change of baseline without adjustment. Note that
when the baseline is included as a covariate in a standard linear model, the
estimated treatment effects are identical for both ‘change from baseline’
(on an additive scale) and the ‘raw outcome’ analysis. Consequently if the
appropriate adjustment is done, then the choice of endpoint becomes solely
an issue of interpretability."

I first did the analysis using treatment as a factor and change from
baseline as the dependent variable, and the results for UNIANOVA and MIXED
were identical. Then I added the baseline value as a covariate using the
syntax above. Now the d.f.s for all main effects were different between
UNIANOVA and MIXED, and so were the F and p-values.

Andreas



--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD