ANOVA with Pos hoc comparisons

[Kuan]

Hi, I have run a one way analysis of variance using SPSS. The result of this
test was a significant difference among the three groups. The mean scores
and SDs for three groups were 3.74 (0.72) for A group, 3.26 (0.92) for B
group, & 3.25 (0.83) for C group. To determine where significant differences
lie, I performed a post hoc analysis. SPSS showed me that there was a
significant difference between A & B, however, no significant difference
between A & C.
But the mean difference between A & C is larger than the mean difference
between A & B. I wonder why there is no significant difference between A & C?

Any help would be greatly appreciated!

Kuan

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[David Marso]

Missing info: Sample sizes of each group!!!

Kuan wrote

Hi, I have run a one way analysis of variance using SPSS. The result of this
test was a significant difference among the three groups. The mean scores
and SDs for three groups were 3.74 (0.72) for A group, 3.26 (0.92) for B
group, & 3.25 (0.83) for C group. To determine where significant differences
lie, I performed a post hoc analysis. SPSS showed me that there was a
significant difference between A & B, however, no significant difference
between A & C.
But the mean difference between A & C is larger than the mean difference
between A & B. I wonder why there is no significant difference between A & C?

Any help would be greatly appreciated!

Kuan

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

Also missing:  Which multiple comparison method did you use?  

David Marso wrote

Missing info: Sample sizes of each group!!!

Kuan wrote

Hi, I have run a one way analysis of variance using SPSS. The result of this
test was a significant difference among the three groups. The mean scores
and SDs for three groups were 3.74 (0.72) for A group, 3.26 (0.92) for B
group, & 3.25 (0.83) for C group. To determine where significant differences
lie, I performed a post hoc analysis. SPSS showed me that there was a
significant difference between A & B, however, no significant difference
between A & C.
But the mean difference between A & C is larger than the mean difference
between A & B. I wonder why there is no significant difference between A & C?

Any help would be greatly appreciated!

Kuan

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Kuan]

Thanks for reply.

There are 163 for A group, 28 for B group & 6 group. I tried several
comparison methods, including, Tukey HSD, Bonferroni, and Scheffe. I got
same results.

Actually, I measured two DVs in the study. In addition to the one Ｉ　
mentioned in my previous message, I also have run a ANOVA for another DV. I
got a  significant difference among the three groups and got significant
differences between A and B and between A and C on post hoc comparisons.

Kuan

On Wed, 5 Oct 2011 06:05:32 -0700, Bruce Weaver <[hidden email]>
wrote:

>Also missing:  Which multiple comparison method did you use?
>
>
>David Marso wrote:
>>
>> Missing info: Sample sizes of each group!!!
>>
>> Kuan wrote:
>>>
>>> Hi, I have run a one way analysis of variance using SPSS. The result of
>>> this
>>> test was a significant difference among the three groups. The mean scores
>>> and SDs for three groups were 3.74 (0.72) for A group, 3.26 (0.92) for B
>>> group, & 3.25 (0.83) for C group. To determine where significant
>>> differences
>>> lie, I performed a post hoc analysis. SPSS showed me that there was a
>>> significant difference between A & B, however, no significant difference
>>> between A & C.
>>> But the mean difference between A & C is larger than the mean difference
>>> between A & B. I wonder why there is no significant difference between A
>>> & C?
>>>
>>> Any help would be greatly appreciated!
>>>
>>> Kuan
>>>
>>> =====================
>>> To manage your subscription to SPSSX-L, send a message to
>>> LISTSERV@.UGA (not to SPSSX-L), with no body text except the
>>> command. To leave the list, send the command
>>> SIGNOFF SPSSX-L
>>> For a list of commands to manage subscriptions, send the command
>>> INFO REFCARD
>>>
>>
>
>
>-----
>--
>Bruce Weaver
>[hidden email]
>http://sites.google.com/a/lakeheadu.ca/bweaver/
>
>"When all else fails, RTFM."
>
>NOTE: My Hotmail account is not monitored regularly.
>To send me an e-mail, please use the address shown above.
>
>--
>View this message in context:

http://spssx-discussion.1045642.n5.nabble.com/ANOVA-with-Pos-hoc-comparisons-tp4871444p4872596.html
>Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>
>=====================
>To manage your subscription to SPSSX-L, send a message to
>[hidden email] (not to SPSSX-L), with no body text except the
>command. To leave the list, send the command
>SIGNOFF SPSSX-L
>For a list of commands to manage subscriptions, send the command
>INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Rich Ulrich]

With several tests, you have several distinct cases to discuss.

As I recall the original rules for the post-hoc tests, *you* cannot
justify using Tukey's tests because they make a strong assumption
that the Ns are equal, or approximately so.  You Ns are not in
a range, say, of 20%.  Either Tukey test should make a simple
comparison of the observed means, using the pooled variance
 -- and assuming that each N is equal to the reciprocal mean of
the 3 Ns.

The rules for those two also say that you compare the most
extreme means; if they fail to reject, you stop testing.  You
never look at the tests for the means that are less extreme.

If SPSS is giving you "different" for the means ranked 1 and 2,
but not-different for 1 and 3, then it isn't following those rules.
I think I can forgive SPSS if it falls back to using the actual Ns,
but that should be documented, and it probably should be called
a "modified Tukey".  If SPSS also abandons the rule that the
comparisons start from the extreme ranks, and end when "not
different" occurs, then I think it should, at best, be called a
test that "uses the Tukey HSD ranges".  But it changes the rules
too much to keep the original name.   Are you sure you get those
same results for the Tukey test?

Using the actual Ns for two-group tests can easily yield the result
that you described for the group comparisons.  Using the actual
Ns is something that you might expect for Bonferroni test, or LSD
testing, but you should be sure to report clearly that this is what
is happening. 

Scheffe's test is not a post-hoc test in the same sense as others.
It can replace the preliminary test, or substitute for it.  It makes
a comparison between two groups; if that "rejects", then that
difference alone is large enough -- in the Sums of Squares for
their mean-difference -- to create a rejection for the overall test
on all groups.  That can be a reasonable result for those data.


--
Rich Ulrich

> Date: Thu, 6 Oct 2011 10:08:05 -0400

> From: [hidden email]
> Subject: Re: ANOVA with Pos hoc comparisons
> To: [hidden email]
>
> Thanks for reply.
>
> There are 163 for A group, 28 for B group & 6 group. I tried several
> comparison methods, including, Tukey HSD, Bonferroni, and Scheffe. I got
> same results.
>
> Actually, I measured two DVs in the study. In addition to the one Ｉ　
> mentioned in my previous message, I also have run a ANOVA for another DV. I
> got a significant difference among the three groups and got significant
> differences between A and B and between A and C on post hoc comparisons.
>
> Kuan
>
> On Wed, 5 Oct 2011 06:05:32 -0700, Bruce Weaver <[hidden email]>
> wrote:
>
> >Also missing: Which multiple comparison method did you use?
> >
> >
> >David Marso wrote:
> >>
> >> Missing info: Sample sizes of each group!!!
> >>
> >> Kuan wrote:
> >>>
> >>> Hi, I have run a one way analysis of variance using SPSS. The result of
> >>> this
> >>> test was a significant difference among the three groups. The mean scores
> >>> and SDs for three groups were 3.74 (0.72) for A group, 3.26 (0.92) for B
> >>> group, & 3.25 (0.83) for C group. To determine where significant
> >>> differences
> >>> lie, I performed a post hoc analysis. SPSS showed me that there was a
> >>> significant difference between A & B, however, no significant difference
> >>> between A & C.
> >>> But the mean difference between A & C is larger than the mean difference
> >>> between A & B. I wonder why there is no significant difference between A
> >>> & C?
> >>>

[snip]

[David Marso]

If one examines the algorithms used for Post Hoc Tests:
_Least Significant Difference (LSD), Bonferroni, and Sidak_
For the least significant difference, Bonferroni, and Sidak tests, only pairwise confidence intervals
are given.
The test statistic is Mean_i - Mean_j) > Q_ij * Range
Q_ij=SQRT(MSE*.5 * (1/n_i + 1/n_j))

_Unequal Sample Sizes and Unequal Variances_
"Two means are significantly different if  ABS(Mean_i - Mean_j) >= Q_ij * Range
where   Q_ij=SQRT (S2_i/n_i + S2_j/n_j)
and Range depends on the specific test being used, as listed below."
--
Throughout, some tests use Qh (based on harmonic n) others Q_ij :
In the case of the confidence intervals for mean differences.
Hence why I surmised something regarding the cell sample sizes.

Mockup of Kuan's data situation for illustration:
MATRIX DATA VARIABLES DEP GP
           /FORMAT FREE /FACTOR GP /CELLS 3 /CONTENTS (MEAN STDDEV N).
BEGIN DATA
3.74  1  .72   1  163   1
3.26  2  .92   2   28   2
3.25  3  .83   3    6   3
END DATA .
oneway DEP by GP
  / matrix in (*)
 / POSTHOC BONFERONNI LSD TUKEY .

Notice the Tukey provides a separate table for HSD using Qh.
HTH, David

Rich Ulrich-2 wrote

With several tests, you have several distinct cases to discuss.

As I recall the original rules for the post-hoc tests, *you* cannot
justify using Tukey's tests because they make a strong assumption
that the Ns are equal, or approximately so.  You Ns are not in
a range, say, of 20%.  Either Tukey test should make a simple
comparison of the observed means, using the pooled variance
 -- and assuming that each N is equal to the reciprocal mean of
the 3 Ns.

The rules for those two also say that you compare the most
extreme means; if they fail to reject, you stop testing.  You
never look at the tests for the means that are less extreme.

If SPSS is giving you "different" for the means ranked 1 and 2,
but not-different for 1 and 3, then it isn't following those rules.
I think I can forgive SPSS if it falls back to using the actual Ns,
but that should be documented, and it probably should be called
a "modified Tukey".  If SPSS also abandons the rule that the
comparisons start from the extreme ranks, and end when "not
different" occurs, then I think it should, at best, be called a
test that "uses the Tukey HSD ranges".  But it changes the rules
too much to keep the original name.   Are you sure you get those
same results for the Tukey test?

Using the actual Ns for two-group tests can easily yield the result
that you described for the group comparisons.  Using the actual
Ns is something that you might expect for Bonferroni test, or LSD
testing, but you should be sure to report clearly that this is what
is happening.  

Scheffe's test is not a post-hoc test in the same sense as others.
It can replace the preliminary test, or substitute for it.  It makes
a comparison between two groups; if that "rejects", then that
difference alone is large enough -- in the Sums of Squares for
their mean-difference -- to create a rejection for the overall test
on all groups.  That can be a reasonable result for those data.


--
Rich Ulrich


> Date: Thu, 6 Oct 2011 10:08:05 -0400
> From: [hidden email]
> Subject: Re: ANOVA with Pos hoc comparisons
> To: [hidden email]
>
> Thanks for reply.
>
> There are 163 for A group, 28 for B group & 6 group. I tried several
> comparison methods, including, Tukey HSD, Bonferroni, and Scheffe. I got
> same results.
>
> Actually, I measured two DVs in the study. In addition to the one Ｉ　
> mentioned in my previous message, I also have run a ANOVA for another DV. I
> got a  significant difference among the three groups and got significant
> differences between A and B and between A and C on post hoc comparisons.
>
> Kuan
>
> On Wed, 5 Oct 2011 06:05:32 -0700, Bruce Weaver <[hidden email]>
> wrote:
>
> >Also missing:  Which multiple comparison method did you use?
> >
> >
> >David Marso wrote:
> >>
> >> Missing info: Sample sizes of each group!!!
> >>
> >> Kuan wrote:
> >>>
> >>> Hi, I have run a one way analysis of variance using SPSS. The result of
> >>> this
> >>> test was a significant difference among the three groups. The mean scores
> >>> and SDs for three groups were 3.74 (0.72) for A group, 3.26 (0.92) for B
> >>> group, & 3.25 (0.83) for C group. To determine where significant
> >>> differences
> >>> lie, I performed a post hoc analysis. SPSS showed me that there was a
> >>> significant difference between A & B, however, no significant difference
> >>> between A & C.
> >>> But the mean difference between A & C is larger than the mean difference
> >>> between A & B. I wonder why there is no significant difference between A
> >>> & C?
> >>>
[snip]