not significant interation effect but very significant post hoc

[znbaran]

Hi to Dear All,

l have some confusion how to explain the following situation:

In a mixed design l have one group (let say A) factor (btw subject factor) and two repeated factors (let say B and C). Since spss does not generate post hoc comparasions automatically, l use the EMMEANS and compare commands like at below.

 /EMMEANS=TABLES(A*B) COMPARE(A) ADJ(BONFERRONI)
 /EMMEANS=TABLES(A*B) COMPARE(B) ADJ(BONFERRONI)

 /EMMEANS=TABLES(B*C) COMPARE(B) ADJ(BONFERRONI)
 /EMMEANS=TABLES(B*C) COMPARE(C) ADJ(BONFERRONI)

the finding was that A and B main effects significant but A*B interaction is not. Just for curiosity, l made post hoc tests for insignificant A*B interaction effect and saw that some pairwise comparasions were significant with very small p values such as 0.000 even though to Bonferroni adjustments.

So what can be the possible explanation for this? OK, l should not do the post hocs for insignificant omnibus effect. However, how can be at there very significant (p=0.000) pairwise comparasions values in this situation?

Best wishes,


Zeynel

[Swank, Paul R]

How many levels to the repeated measures variables? If there is a large number of levels then a few pairwise comparisons may be significant in the absence of a significant omnibus interaction. Doing the posttest comparisons does take advantage of change if there are a lot of comparisons, and a significant pairwise test may be lost when combined with a lot of non-significant ones.

Dr. Paul R. Swank,
Professor and Director of Research
Children's Learning Institute
University of Texas Health Science Center-Houston

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of znbaran
Sent: Monday, April 18, 2011 3:49 PM
To: [hidden email]
Subject: not significant interation effect but very significant post hoc

Hi to Dear All,

l have some confusion how to explain the following situation:

In a mixed design l have one group (let say A) factor (btw subject factor)
and two repeated factors (let say B and C). Since spss does not generate
post hoc comparasions automatically, l use the EMMEANS and compare commands
like at below.

 /EMMEANS=TABLES(A*B) COMPARE(A) ADJ(BONFERRONI)
 /EMMEANS=TABLES(A*B) COMPARE(B) ADJ(BONFERRONI)

 /EMMEANS=TABLES(B*C) COMPARE(B) ADJ(BONFERRONI)
 /EMMEANS=TABLES(B*C) COMPARE(C) ADJ(BONFERRONI)

the finding was that A and B main effects significant but A*B interaction is
not. Just for curiosity, l made post hoc tests for insignificant A*B
interaction effect and saw that some pairwise comparasions were significant
with very small p values such as 0.000 even though to Bonferroni
adjustments.

So what can be the possible explanation for this? OK, l should not do the
post hocs for insignificant omnibus effect. However, how can be at there
very significant (p=0.000) pairwise comparasions values in this situation?

Best wishes,


Zeynel

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/not-significant-interation-effect-but-very-significant-post-hoc-tp4311755p4311755.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Rich Ulrich]

> l have some confusion how to explain the following situation:
>
> In a mixed design l have one group (let say A) factor (btw subject factor)
> and two repeated factors (let say B and C). Since spss does not generate
> post hoc comparasions automatically, l use the EMMEANS and compare commands
> like at below.
>
> /EMMEANS=TABLES(A*B) COMPARE(A) ADJ(BONFERRONI)
> /EMMEANS=TABLES(A*B) COMPARE(B) ADJ(BONFERRONI)
>
> /EMMEANS=TABLES(B*C) COMPARE(B) ADJ(BONFERRONI)
> /EMMEANS=TABLES(B*C) COMPARE(C) ADJ(BONFERRONI)
>
> the finding was that A and B main effects significant but A*B interaction is
> not. Just for curiosity, l made post hoc tests for insignificant A*B
> interaction effect and saw that some pairwise comparasions were significant
> with very small p values such as 0.000 even though to Bonferroni
> adjustments.
>
> So what can be the possible explanation for this? OK, l should not do the
> post hocs for insignificant omnibus effect. However, how can be at there
> very significant (p=0.000) pairwise comparasions values in this situation?

The technical explanation is probably what Paul suggests, that you have a
lot of d.f.  and the other effects were close enough to zero, so these were
swamped in the noise.

However, you have interactions of Group with a Repeated factor with many
levels.  In my experience, I've seen a lot of people analyze these wrong --
If your Repeats are not ordered in some fashion, then these comments are

not relevant.


In those data done wrong, the periods are ordered, so, at the very least,
the PIs usually *are*  interested in the linear trend, plus its interaction.
But they don't test it.

Or else, the expected change is all *immediate*; the proper testing may
be Period-1 versus <something>, followed by the linear trend on the rest.

--
Rich Ulrich



=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[znbaran]

Thank you for Answers

In fact, A and C have two levels, and B has three levels. :(

[Kornbrot, Diana]

Not convinced that this syntax will remove the main effects. It also appears to ignore the fact that B, C are repeated – so it won’t be using the right standard errors
To test: construct new repeated measures variables for all B combinations, eg b1-b2, b1-b3, etc. Then do new ANOVAs with A as btw and B diff as wthn for each B difference. Use .05/N as significance level where N is number of B pairs. Also for C – and if you are looking for 3-way interactions.....lost the will to live.
Or, revived now: label the B, C combinations D and procede as above. You will then know which, if any of the within comparisons are difference for different A groups
Best
Diana




On 19/04/2011 06:51, "Rich Ulrich" <rich-ulrich@...> wrote:

> l have some confusion how to explain the following situation:
>
> In a mixed design l have one group (let say A) factor (btw subject factor)
> and two repeated factors (let say B and C). Since spss does not generate
> post hoc comparasions automatically, l use the EMMEANS and compare commands
> like at below.
>
> /EMMEANS=TABLES(A*B) COMPARE(A) ADJ(BONFERRONI)
> /EMMEANS=TABLES(A*B) COMPARE(B) ADJ(BONFERRONI)
>
> /EMMEANS=TABLES(B*C) COMPARE(B) ADJ(BONFERRONI)
> /EMMEANS=TABLES(B*C) COMPARE(C) ADJ(BONFERRONI)
>
> the finding was that A and B main effects significant but A*B interaction is
> not. Just for curiosity, l made post hoc tests for insignificant A*B
> interaction effect and saw that some pairwise comparasions were significant
> with very small p values such as 0.000 even though to Bonferroni
> adjustments.
>
> So what can be the possible explanation for this? OK, l should not do the
> post hocs for insignificant omnibus effect. However, how can be at there
> very significant (p=0.000) pairwise comparasions values in this situation?

The technical explanation is probably what Paul suggests, that you have a
lot of d.f.  and the other effects were close enough to zero, so these were
swamped in the noise.

However, you have interactions of Group with a Repeated factor with many
levels.  In my experience, I've seen a lot of people analyze these wrong --
If your Repeats are not ordered in some fashion, then these comments are

not relevant.


In those data done wrong, the periods are ordered, so, at the very least,
the PIs usually *are*  interested in the linear trend, plus its interaction.
But they don't test it.

Or else, the expected change is all *immediate*; the proper testing may
be Period-1 versus <something>, followed by the linear trend on the rest.

--
Rich Ulrich



=====================
To manage your subscription to SPSSX-L, send a message to
LISTSERV@... (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

---

Professor Diana Kornbrot
email:  d.e.kornbrot@...    
web:    http://web.me.com/kornbrot/KornbrotHome.html
Work
School of Psychology
 University of Hertfordshire
 College Lane, Hatfield, Hertfordshire AL10 9AB, UK
 voice:   [phone]
   fax:     [phone]
Home
 19 Elmhurst Avenue
 London N2 0LT, UK
    voice:   [phone]
    mobile: [phone]
   fax:      [phone]

[Ryan]

Did you fit a linear mixed model? If yes, you should be able to
perform all post-hoc tests of interest employing the TEST subcommand.

Ryan

On Tue, Apr 19, 2011 at 2:18 AM, znbaran <[hidden email]> wrote:

> Thank you for Answers
>
> In fact, A and C have two levels, and B has three levels. :(
>
>
>
> --
> View this message in context: http://spssx-discussion.1045642.n5.nabble.com/not-significant-interation-effect-but-very-significant-post-hoc-tp4311755p4312575.html
> Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[znbaran]

Again thanks for all answers.

Then, Dear Diana

 l will implement your method at first via calculating difference scores at one level, and then do the repeated analysis again. But you are rigth for triple interaction effect this method make some time. But, l have not any other choice.

Best wishes,

Zeynel

[znbaran]

Dear Ryan

l did not do the linear mixed model on this data. Hence, l do not know how to do that. Yes, l have a mixed model and l entered all the data into a model chosen under the menu of repeated design. there is compare menu at there, l used firstly simple and repeated options to get the planned comparasions. you mean with "linear" word  as "polynominal selection" option at compare menu? This also gave some comparasions but not all comparasions.


Best,

Zeynel

[Ryan]

Now I see. By mixed model, I think you mean that the ANOVA you ran
included both "between-subjects" and "within-subjects" variables.
ANOVA falls under the rubric of general linear models. I was actually
referring to fitting a linear mixed model, which is not the same as a
general linear model, although it can yield similar results under
certain circumstances. Linear mixed models typically handle unbalanced
designs, hierarchical designs (e.g., students nested classroom),
repeated measures designs (particularly when there are more than two
repeats), and missing data better than general linear models. Linear
mixed models can be fit through the MIXED procedure in SPSS. It's also
possible to perform contrasts employing the TEST subcommand offered in
the MIXED procedure to answer specific research questions. Standard
errors derived from these TEST statements take into account the
specified variance-covariance matrix of the model. I don't have time
to elaborate further right now.

Ryan

On Wed, Apr 20, 2011 at 5:13 AM, znbaran <[hidden email]> wrote:

> Dear Ryan
>
> l did not do the linear mixed model on this data. Hence, l do not know how
> to do that. Yes, l have a mixed model and l entered all the data into a
> model chosen under the menu of repeated design. there is compare menu at
> there, l used firstly simple and repeated options to get the planned
> comparasions. you mean with "linear" word  as "polynominal selection" option
> at compare menu? This also gave some comparasions but not all comparasions.
>
>
> Best,
>
> Zeynel
>
> --
> View this message in context: http://spssx-discussion.1045642.n5.nabble.com/not-significant-interation-effect-but-very-significant-post-hoc-tp4311755p4315302.html
> Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD