Re: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean

Posted by Ryan on Apr 29, 2016; 11:46am
URL: http://spssx-discussion.165.s1.nabble.com/2way-mixed-ANOVA-significant-interaction-but-main-effects-not-significant-What-does-this-mean-tp5732027p5732057.html

The classic Behrens-Fisher problem was to perform a t-test where the residual variances across groups varied. SPSS provides a test of the null of equal means between independent groups by employing the Satterthwaite correction to the degrees of freedom.

The model proposed by Bruce essentially applies that correction in the presence of a covariate. The model presented by Bruce does not loosen any of the assumptions of a covariate. Within the MIXED procedure. one could certainly evaluate the tenability of the assumptions of the covariate, while allowing for heterogeneous residual variances. For example, one could employ a MIXED model which incorporates the interaction between group and covariate and perform an LRT to see if the fit improves significantly, akin to Bartlett's test.

Ryan

On Thu, Apr 28, 2016 at 4:51 PM, Rich Ulrich <[hidden email]> wrote:

Clearly, it does not address the problem of unequal values for Pre, between groups.
That raises a problem of logical conclusions, even if the MIXED test works okay as a test.

Clearly, it will be inferior testing compared to testing a simple, natural transformation
of the measures, when that is what the data cry out for.

 - Those are the two big considerations that jump to my mind.

It would be nice to know what situations have this as the ideal approach; and to know what
other assumptions there are for this approach, and how serious it is to violate them.

--
Rich Ulrich

> Date: Wed, 27 Apr 2016 10:32:40 -0700
> From: [hidden email]
> Subject: Re: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean
> To: [hidden email]
>

> Another way to handle heterogeneity of variance is to estimate the model via

> MIXED, allowing for heterogeneous group variances. See below.
>
> * ANCOVA with DV = post, covariate = pre estimated in the usual way.
>
> UNIANOVA post BY grp WITH pre
> /PRINT = parameter
> /EMMEANS = table(grp)
> /DESIGN = pre grp
> .
>
> * Same model via MIXED.
>
> MIXED post BY grp WITH pre
> /FIXED=pre grp | SSTYPE(3)
> /PRINT=SOLUTION TESTCOV
> /EMMEANS = tables(grp)
> .
>
> * Now include /REPEATED sub-command to
> * allow for heterogenous group variances.
>
> MIXED post BY grp WITH pre
> /FIXED=pre grp | SSTYPE(3)
> /PRINT=SOLUTION TESTCOV
> /REPEATED=grp | SUBJECT(ID) COVTYPE(DIAG)
> /EMMEANS = tables(grp)
> .
>
> HTH.
>
>
> Rich Ulrich wrote
> > There is a ton of literature on the subject of analyzing change.
> > Unfortunately,
> > much of it is not very well informed. On a quick Google, I found that the
> > first
> > answer at the URL below (the Reply referencing Senn) gives a pretty good
> > overview,
> > plus references. (The Reply-er endorses using ANCOVA for controlled
> > studies.)
> >
> > http://stats.stackexchange.com/questions/3466/best-practice-when-analysing-pre-post-treatment-control-designs
> >
> > I will add:
> > On assumptions: It is nice to have (1) equal variances everywhere, (2)
> > equal means at Pre, and
> > (3) a shared regression line. Given those, it is hard to fault the
> > ANCOVA. In my experience,
> > "unequal variances" are sometimes fixed by suitable transformation of the
> > criterion. But the
> > failure of the assumptions is why (at least, in my off-hand thoughts here)
> > you might want to
> > analyze the Outcome while ignoring Pre, or analyze the simple change
> > score. Unequal-at-Pre
> > raises serious logical conundrums, at times, and
> > regression-not-to-the-shared-mean on top of
> > unequal-at-Pre puts you into statistical complication, and controversy.
> > The latter was the case
> > of analyzing long-term outcome for Headstart vs. other students -- where
> > Expected outcome with
> > no intervention, according to other experience, would be that the
> > lower-achieving target cases
> > should fall further and further behind.
> >
> > --
> > Rich Ulrich
> >
> > --- snip ---
>
>
>
>
>
> -----
> --
> Bruce Weaver
> [hidden email]
> http://sites.google.com/a/lakeheadu.ca/bweaver/
>
> "When all else fails, RTFM."
>
> NOTE: My Hotmail account is not monitored regularly.
> To send me an e-mail, please use the address shown above.
>
> --
> View this message in context: http://spssx-discussion.1045642.n5.nabble.com/2way-mixed-ANOVA-significant-interaction-but-main-effects-not-significant-What-does-this-mean-tp5732027p5732051.html
> Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD