Significan ANOVA, but non significant post-hoc
Locked
Significan ANOVA, but non significant post-hoc
 
|
Hello: I'm running an unbalanced ANOVA, (the variances across groups are homogeneos) and I have a significant F ( p = 0,013) because I'm working with a 5% signifcance level. When I ran the post-hoc test none of the comparations is significant. I'm not sure how to explain this, Kindly Andrés Mg. Andrés Burga León Coordinador de Análisis e Informática Unidad de Medición de la Calidad Educativa Ministerio de Educación del Perú Calle El Comercio s/n (espalda del Museo de la Nación) Lima 41 Perú Teléfono 615-5840 |
Re: Significan ANOVA, but non significant post-hoc
|
|
If you are running a post-hoc test that adjusts for the # of comparisons you are make (e.g., Bonferroni), and you have several groups, small sample sizes, or both - your post-hoc tests will be a more conservative test of differences between groups. If these are not sig. then I believe it suggests your overall 'significant' F is a type 1 error. |
Re: Significan ANOVA, but non significant post-hoc
|
|
In reply to this post by ANDRES ALBERTO BURGA LEON
Hi Andres,
A little bit more information would be helpful, what is the design of the ANOVA, is it between subjects (independent samples), within subjects (repeated measures) or mixed (both)? Are you referring to a main effect or an interaction? How many groups do you have for the factor(s)? Which post-hoc tests did you run? Assuming nothing has gone wrong then this result is not unusual. Please see this article for a perspective on it: ftp://ftp.spss.com/pub/spss/statistics/nichols/articles/testsnoa.txt You can also reference Cardinal & Aitken 2006 ANOVA for the Behavioural Sciences Researcher p91 (which is where I found the above link). To paraphrase, non-significant posthoc tests in the presence of a significant F-test are often interpreted as the effect of a more complex contrast (such as a linear trend). But really the problem results from a lack of statistical power to resolve the simple post-hoc comparisons. In my own experience I find this sort of inconsistency happens with smaller sample sizes or larger numbers of groups. all the best, Andrew ANDRES ALBERTO BURGA LEON wrote: > > Hello: > > I'm running an unbalanced ANOVA, (the variances across groups are > homogeneos) and I have a significant F ( p = 0,013) because I'm > working with a 5% signifcance level. When I ran the post-hoc test none > of the comparations is significant. > > I'm not sure how to explain this, > > Kindly > > Andrés > > Mg. Andrés Burga León > Coordinador de Análisis e Informática > Unidad de Medición de la Calidad Educativa > Ministerio de Educación del Perú > Calle El Comercio s/n (espalda del Museo de la Nación) > Lima 41 > Perú > Teléfono 615-5840 -- Andrew J. Lawrence Research Psychologist Centre for Clinical Neuroscience St George's University of London Cranmer Terrace London SW17 0RE [hidden email] tel: +44(0)20 8266 6468 fax: +44(0)20 8725 2950 ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Re: Significan ANOVA, but non significant post-hoc
|
|
In reply to this post by ANDRES ALBERTO BURGA LEON
Andres,
You did not specify what "post-hoc" test you ran... However, if it was a pair-wise test, it may well be that the contrast being detected by the anova is a more complex one, such as the difference between (mu1 + mu2)/2 and (m3 + mu4)/2... wbw __________________________________________________________________________ William B. Ware, Professor Educational Psychology, CB# 3500 Measurement, and Evaluation University of North Carolina PHONE (919)-962-7848 Chapel Hill, NC 27599-3500 FAX: (919)-962-1533 Office: 118 Peabody Hall EMAIL: [hidden email] Adjunct Professor School of Social Work Academy of Distinguished Teaching Scholars at UNC-Chapel Hill __________________________________________________________________________ On Mon, 23 Nov 2009, ANDRES ALBERTO BURGA LEON wrote: > > Hello: > > I'm running an unbalanced ANOVA, (the variances across groups are > homogeneos) and I have a significant F ( p = 0,013) because I'm working with > a 5% signifcance level. When I ran the post-hoc test none of the > comparations is significant. > > I'm not sure how to explain this, > > Kindly > > Andrés > > Mg. Andrés Burga León > Coordinador de Análisis e Informática > Unidad de Medición de la Calidad Educativa > Ministerio de Educación del Perú > Calle El Comercio s/n (espalda del Museo de la Nación) > Lima 41 > Perú > Teléfono 615-5840 > |
Re: Significan ANOVA, but non significant post-hoc
|
|
In reply to this post by ANDRES ALBERTO BURGA LEON
Andres ... The overall ANOVA F test identifies
significant differences between means. However, the F test will
identify significant differences for combinations of means as well as
differences between pairs of means. That is why an overall F test can be
significant, yet none of the pairwise differences are
significant.
Art
************************* Art
Burke Associate, Evaluation Program Education Northwest 101
SW Main St, Ste 500 Portland, OR 97204 Phone: 503.275.9592 Art.Burke[hidden email] We
have recently changed our name to "Education Northwest" from "Northwest Regional
Educational Laboratory." Please note the new e-mail and Web addresses in the
signature above. You may continue to find us on the Web at http://www.nwrel.org for
the immediate future as well. ************************ From: ANDRES ALBERTO BURGA LEON [mailto:[hidden email]] Sent: Monday, November 23, 2009 7:12 AM To: [hidden email] Subject: Significan ANOVA, but non significant post-hoc Hello: I'm running an unbalanced ANOVA, (the variances across groups are homogeneos) and I have a significant F ( p = 0,013) because I'm working with a 5% signifcance level. When I ran the post-hoc test none of the comparations is significant. I'm not sure how to explain this, Kindly Andrés Mg. Andrés Burga León Coordinador de Análisis e Informática Unidad de Medición de la Calidad Educativa Ministerio de Educación del Perú Calle El Comercio s/n (espalda del Museo de la Nación) Lima 41 Perú Teléfono 615-5840 |
Re: Significan ANOVA, but non significant post-hoc
|
|
In reply to this post by ANDRES ALBERTO BURGA LEON
ANDRES ALBERTO BURGA LEON wrote:
> I'm running an unbalanced ANOVA, (the variances across groups are > homogeneos) and I have a significant F ( p = 0,013) because I'm working > with a 5% signifcance level. When I ran the post-hoc test none of the > comparations is significant. > > I'm not sure how to explain this, In addition to the comments made so far, note * http://www.pmean.com/05/TukeyTest.html which has a link to a previous discussion of this issue in STAT-L. But the unbalanced nature of the data can create extra problems, beyond what I commented on in my webpage. Many post hoc tests were designed for balanced data. A common fix is to replace the individual sample sizes with the harmonic mean of the sample sizes, but this will produce, at best, an approximate solution. -- Steve Simon, Standard Disclaimer The Monthly Mean is celebrating its first anniversary. Find out more about the newsletter that dares to call itself "average" at www.pmean.com/news ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Re: Significan ANOVA, but non significant post-hoc
|
|
In reply to this post by William B. Ware
Thanks everybody for your comments.
I’m giving more information about the problem I’m trying to solve: I have a random sample of 677 mothers who answer open-ended question: “Do you think initial education is important? Why?” Those answers were subject to content analysis and we have 8 different groups, whose n are as follows:
I also have a student achievement measure. I want to investigate if there are significant differences on students mean achievement, considering their mothers conceptions about the importance of initial education. So I run an ANOVA and I check that the variances are homogeneous across groups. After that I use the Hochberg GT2 post hoc test. I’ve read that this is a recommended post-hoc test when the sample sizes are unequal. Andrés
|
| Free forum by Nabble | Edit this page |