Re: One-Way Anova Repeated Measures: Main effect is significant, but post-hoc tests are not significant

Posted by Kornbrot, Diana on Jan 03, 2014; 1:05pm
URL: http://spssx-discussion.165.s1.nabble.com/One-Way-Anova-Repeated-Measures-Main-effect-is-significant-but-post-hoc-tests-are-not-significant-tp5723746p5723759.html

It is possible for a factor to be significant overalland no pair wise comparisons to be significant even with perfect sphericity.
Contrasts do not have to be pairwise. E.g. It amy be that mean of groups 1 and 2 combines is greater than mean groups 3,3,45 combined – but neither 1 or 2 is signficantly different form any of 3,4,5. There are many more possibilities than df.

If the study has a priori hypotheses [no mre than k-1] then these can be tested without corrections.
If the study is exploratory, aka fishing expedition, then corrections for multiples should always be made.

It is very important to look at effect sizes. If a post hoc omparison has large but ns effect then more power is needed. No conclusions can be drawn, but it is worth replicating with more observations.

In any event a priori power should ALWAYS be reported for ns tests, low power with ns means that study s not adding much to knowledge.

Note that current guidelines,e.g. Equator, and ‘respectable’ journals recommend that the data analysis plan is produced BEFORE the data are collected.
Conclusions based on post hoc unplanned tests, are dubious unless effect sizes are very large, and even then they should be the basis of further studies with panned tests of replication

Best

Diana


On 03/01/2014 05:43, "Ryan Black" <ryan.andrew.black@...> wrote:

JohnDoe,

Under certain conditions, it's perfectly acceptable to employ the Tukey HSD procedure as a follow-up procedure in testing all-possible pairwise comparisons in a repeated measures ANOVA. While it might take a little bit of effort, this procedure can be employed in SPSS for your design.

But let's step back for a moment. What is your understanding of the assumption of sphericity? How could a departure in sphericity affect the the sampling distribution of the ratio of mean squares? Are you familiar with the index, espilon? What is your estimate of epsilon?  What is your understanding of the rule(s) of thumb regarding epsilon? What are the corrections offered by SPSS? 

I really appreciated Bruce's question/comment about the "...nature of the repeated measures factor." It could have an impact not only on the post-hoc tests performed but the model as well (e.g., general linear model versus linear mixed model). For example, if your repeated measures variable is a measure of time, it's possible to observe a decay in correlations between residuals from observations more distant in time. This type of residual correlation structure could be modeled in the MIXED procedure. Sorry, but I can't resist plugging the MIXED procedure. :-)

Ryan


On Thu, Jan 2, 2014 at 12:13 PM, JohnDoe <roryboyle18@...> wrote:

Thanks,

As it was a repeated measures test, I was unable to select any post hocs
except for LSD, Bonferroni, and Sidak. I have four levels in my design



--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/One-Way-Anova-Repeated-Measures-Main-effect-is-significant-but-post-hoc-tests-are-not-significant-tp5723746p5723748.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
LISTSERV@... (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

---

Professor Diana Kornbrot
email: : d.e.kornbrot@...
web:    http://dianakornbrot.wordpress.com/
            http://go.herts.ac.uk/diana_kornbrot
Work
Department of Psychology
School of Life and Medical Sciences
University of Hertfordshire
College Lane, Hatfield, Hertfordshire AL10 9AB, UK
voice:   [phone]
Home
19 Elmhurst Avenue
London N2 0LT, UK
voice:   [phone]
mobile: [phone]