significant Anova, but no significance by multiple comparisons
Locked
 
|
Hello Analysing with SPSS a dependent variable v1 by a factor f1 (4 groups) I receive a significant effect (F04.270; df=3; p=.006). In order to test which group is significantly different I perform a posthoc-test, because of
the homogeneity of variance and not equivalent group sizes I’ve chosen Scheffé. But there are no pair of groups significantly different! How come? Should I chose another posthoc-test because of the really different group sizes (Group 1: 108; 2: 10; 3: 27;
4: 11). Or is there need for a completely other analysis? Thanks for help. Tom |
Re: significant Anova, but no significance by multiple comparisons
|
|
Hi Thomas:
Scheffé is the champion of conservative tests. I agree that this particularity somewhat helps to protect against heteroskedasticity. Try Tamhane's T2. It deals with lack of HOV better than Scheffé, and the FWER is controlled less tightly (it uses Sidak adjustment of p values). HTH, Marta GG El 21/09/2012 15:18, Balmer, Thomas escribió:
|
Re: significant Anova, but no significance by multiple comparisons
|
|
Scheffé does guarantee at least one significant comparison in the presence of a significant omnibus test but it isn’t necessarily a pairwise difference. There is a more complex comparison that is likely significant. Dr. Paul R. Swank, Professor Health Promotion and Behavioral Sciences School of Public Health University of Texas Health Science Center Houston From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Marta García-Granero Hi Thomas:
|
|
|
Both the contrast analysis and Tamhane work fine – what about Kurskal-Wallis? What do I loose when performing this nonparametric test (which shows
me instead of T2 two pairwise significant differences). Von: SPSSX(r) Discussion [mailto:[hidden email]]
Im Auftrag von Swank, Paul R Scheffé does guarantee at least one significant comparison in the presence of a significant omnibus test but it isn’t necessarily a pairwise difference. There is a more complex comparison that is likely significant. Dr. Paul R. Swank, Professor
Health Promotion and Behavioral Sciences School of Public Health University of Texas Health Science Center Houston From: SPSSX(r) Discussion
[hidden email]
On Behalf Of Marta García-Granero Hi Thomas:
|
|
|
Thomas, Do not switch to an ordinal level test simply because you are unable to find post-hoc pairwise differences. Scheffe's test is known to be conservative. Marta's suggestion of considering other post-hoc tests is worthwhile--at the same time, I would not advocate a fishing expedition. Needless to say, Paul's point is an excellent one--so often people are under the impression that a significant ANOVA *guarantees* at least one significant *simple* pairwise comparison. There are other issues as well--small sample sizes in each cell or perhaps imbalanced sample sizes across cells...
Ryan On Fri, Sep 21, 2012 at 10:19 AM, Balmer, Thomas <[hidden email]> wrote:
|
Re: significant Anova, but no significance by multiple comparisons
|
|
Hi again:
El 21/09/2012 16:30, R B escribió:
Besides, Kruskal-Wallis test is also sensitive to heteroskedasticity (*), while Tamhane's T2 was SPECIFICALLY developed to deal with that problem. If you have some a priori hypotheses concerning groups of means (linear contrasts) then go ahead with that method, SPSS computes their significance adjusted for lack of HOV. Regards, Marta GG (*) as the saying goes: jumping from the frying pan into the fire |
Re: significant Anova, but no significance by multiple comparisons
|
|
In reply to this post by Tom
You have Ns of 108, 10, 27, and 11.
You hint that the variances might not be homogeneous. Most of the post-hoc procedures provide formulas that test with the pooled variance. Most of them also assume, and build into the test, the assumption that the group Ns are equal. You don't have the material for any good, general statement from a post-hoc test. I suggest - Order the other groups by their mean distances from group 1 (N=108) and show the CI for each. And the LSD p-level if you wish. If two of the small groups are extreme, you could add a note marking their two-group p-value. -- Rich Ulrich Date: Fri, 21 Sep 2012 13:18:28 +0000 From: [hidden email] Subject: significant Anova, but no significance by multiple comparisons To: [hidden email] Hello
Analysing with SPSS a dependent variable v1 by a factor f1 (4 groups) I receive a significant effect (F04.270; df=3; p=.006). In order to test which group is significantly different I perform a posthoc-test, because of the homogeneity of variance and not equivalent group sizes I’ve chosen Scheffé. But there are no pair of groups significantly different! How come? Should I chose another posthoc-test because of the really different group sizes (Group 1: 108; 2: 10; 3: 27; 4: 11). Or is there need for a completely other analysis?
Thanks for help. Tom |
Re: significant Anova, but no significance by multiple comparisons
|
How were your 4 levels of the factor
chosen/defined? Are the N's for levels proportionate to the pop
N's?
Are they pop N's? Are they achieved samples fro equal sized attempted samples? It was suggested that you look at more complex contrasts. Some may object to choosing contrasts after having gathered the data, but if your writeup clarifies that you chose the contrast post hoc it may not be so terrible. it will be less work than trying all possible subsets which would be more open to charges of data dredging. 1) look at the boxplots for the 4 groups. 2) try finding homogeneous subsets with a more relaxed p for scheffe. 3) decide which subsets you want to compare. Art Kendall Social Research ConsultantsOn 9/21/2012 12:48 PM, Rich Ulrich wrote:
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Art Kendall
Social Research Consultants |
| Free forum by Nabble | Edit this page |