Re: Simple Main Effects Pairwise Comparisons vs Univariate Tests

Posted by statshelp on Jun 06, 2014; 7:39pm
URL: http://spssx-discussion.165.s1.nabble.com/Simple-Main-Effects-Pairwise-Comparisons-vs-Univariate-Tests-tp5726323p5726379.html

Thank you all for your helpful feedback. I think the discussion has moved a bit away from what my original question was, but to answer some of your questions:

Yes, the design was a true experiment--participants were randomly assigned and do not differ significantly at pre-test.

Yes, I graphed the 9 means and the trend is as hypothesized--groups do not differ at pre-test and the experimental group appears to have a more dramatic increase at post-tests 1 and 2.

The control groups do not appear to be entirely parallel--one group was a treated control group and so it was expected that there may be some gain over time in that group relative to a control group who did not receive any interaction during the intervention phase.

Yes, I did a power analysis--unfortunately we were slightly below our target sample size.
I'm not opposed to the pre-test as a covariate. I only wonder how the results would be received. In virtually every publication on this topic I see ANOVAs presented with the interaction results followed by the pairwise comparisons.

My original question still remains though regarding how to report the results. The pairwise comparisons provide p values, and the univariate tests that are presented right after in the SPSS output provide F statistics with p values. I did run an ANCOVA with pre-test as a covariate and now both the pairwise comparisons between groups and the univariate tests for group are significant at both post-tests, but with different p values. In the other publications with this type of design with 3 groups, I see only F statistics reported when discussing group differences. Is this from the univariate test--even though that is not referring to the contrasts between only two groups?

Thanks!

On Fri, Jun 6, 2014 at 6:37 AM, GMAIL [via SPSSX Discussion] <[hidden email]> wrote:

There are several assumptions for the traditional ANCOVA, some of which that immediately come to mind include:

(1) homogeneity of regression slopes 

(2) independence of IV and covariate

(3) homogeneity of adjusted population variances

(4) linear relationship between covariate and DV

(5) Covariate is fixed and measured without error

(6) independence of observations

(7) normality 

Arguably, some are more important than others. Violations of key ANCOVA assumptions tend to arise when one is dealing with intact / nonrandomized groups, as discussed by another poster.

Weren't there more than 2 time points? I would move over to a linear mixed model which is far more flexible (eg, does not require: (a) sphericity/CS residual covariance, (b) deletion of cases listwise due to missing response data, and the list goes on and on).

Ryan

Sent from my iPhone

On Jun 5, 2014, at 12:09 PM, Rich Ulrich <[hidden email]> wrote:

Assumptions:  What is assumed for ANCOVA, along with homogeneity of slopes,
is the homogeneity of initial means.  Neither is apt to be a problem for designed
studies where the group membership was randomized. 

I thought about tossing in a comment about the need for the Pre means to be equal
(or you can't do valid inference), but I didn't give a thought to the slopes.  It is not
that the assumption does not matter: but I have seldom seen it violated, and I have
never seen it violated with equal means for Pre, and Pre as covariate -- except, maybe,
if that was an expected part of a weird outcome.  If the slopes are not readily *assumed*
to be equal, then I sort of expect that the "unequal slopes" might be a major aspect
of whatever is being concluded.  - Maybe I should leave further commentary on actually
using "unequal slopes" to people who have somehow found them useful, instead of
a symptom to be cured.  I can say more about the bad symptom.

When means of the covariate are unequal, there are usually other problems.
Covariance is problematic when the groups are Observational, especially when
they are created by the high/low dichotomy of some variable correlated with the
covariate.  It is problematic for inference even when there are *no* scaling problems
on the measures, such as basement/ ceiling effects.  (Combining those problems
will often create "unequal slopes".) 

It is easy to make mistakes with unequal Pre, even with homogeneous regressions.
It is not "fair", for instance, to conclude that the high-scoring group "did worse" when
the plots show that high and low groups both merely "regressed toward the mean" as
any statistician should expect.

--
Rich Ulrich

> Date: Thu, 5 Jun 2014 09:08:30 -0400

> From: [hidden email]
> Subject: Re: Simple Main Effects Pairwise Comparisons vs Univariate Tests
> To: [hidden email]
>
> Has somebody declared there is no longer any need to satisfy the homogeneity of slopes assumption before ANCOVA? I keep seeing presentations and publications where everything is “adjusted” for covariates like “pre” or “age” and there is no mention of the test to see if group slopes are NSD. How can one adjust means from a common regression line when subgroups may have completely opposing slopes?
>
> I did check with a former stats consultant faculty at U. Waterloo — now retired, and she was rather scathing about what she saw as cavalier application of ANCOVA for “adjustment” of group means. I have even seen one solution for situations where slopes are sig. different (in another forum): “just don’t call it ANCOVA” !
>
> I’d be interested in feedback, since some colleagues stuck my name on a ms and were surprised when I queried them about the pre-condition for ANCOVA. Am I olde fashioned or misinformed?
>
> Ian Martin
> Ian D. Martin, Ph.D.

---

If you reply to this email, your message will be added to the discussion below:

http://spssx-discussion.1045642.n5.nabble.com/Simple-Main-Effects-Pairwise-Comparisons-vs-Univariate-Tests-tp5726323p5726368.html

To unsubscribe from Simple Main Effects Pairwise Comparisons vs Univariate Tests, click here.
NAML