I think that I would have to modify this procedure, a couple of ways.
"The 3-way interaction was
significant, thus I did not interpret main effects or 2-way interaction and
I focused only on the significant 3-way interaction. In order to explain
this significant 3-way interaction, I ran pairwise comparison corrected with
the bonferonni procedure and I described comparisons that answered my
hypotheses."
1) How big are the various effects? If the N is big enough, it is easy for trivial effects to
be magnified to "significance" at the arbitrary level. "Be cautious" about interpreting
effects is a more cautious way of giving the conventional warning. You may hear
"do not interpret" from people who are looking at your analysis if they see that the high-
order interaction is about as big as the others, but they are skipping the statement of the
assumption, that these are all the same size. (Professors who have not done much data
analysis can also be guilty of this oversight.)
2) Before worrying about followup tests, look at the means. Is there an obvious story
there? Do you have "basement" or "ceiling" effects which would introduce interactions?
- those are the sort that are easiest to ignore, as potential artifacts of scaling. (Dichotomies
created from continuous scales introduce additional potential for artifact, but, like analyzing
ranks, these are apt to be too small to show up unless the N is very large.)
Paired tests are the right sort to run for follow-up, but I don't see that they should deliver
a good "answer" for your hypotheses, because interactions disrupt the narrative of the main
effects. I think it is awkward if your design puts the hypothesis into the interaction. What an
unexpected and strong interaction sometimes shows you is that your whole model should be
constructed differently. -- I learned that from a Usenet newsgroup poster, 15 or 20 years ago.
--
Rich Ulrich
Free forum by Nabble | Edit this page |