Violation of ANOVA post-hoc test

[chengfoh]

Dear friends :

Hope someone has any idea about my problem.

From what i understand, if we have equal sample size and met the assumption of homogeniety of variance, HOV), then we proceed with Dunnett.

I am comparing measurements from treatment groups to a control group, so i perform ANOVA + post hoc dunnett test. However, the treatment/control groups have unequal sample size (n=3 or 4)


1) can i proceed with Dunnett in this situation (unequal sample size but having the homogeneity of variance, HOC assumption met)?

2) and what if I have unequal sample size and violated HOV? from searches there are recommendation on using Games-Howell, Dunnett C, T3, or Temphane T2, but none of seem seem to compare to a control group (like Dunnett test)

3) Addition: Also any advice if having equal sample size but violated HOV?

Really appreciate anyone could help on this. I/m struggling ...

Many thanks,
Cheng Foh

[David Marso]

With sample size of 3 and 4 it seems highly unlikely you have sufficient power to detect any difference.
With 2 groups (treatment/control)  Post hoc tests are irrelevant!
---

chengfoh wrote

Dear friends :

Hope someone has any idea about my problem.

From what i understand, if we have equal sample size and met the assumption of homogeniety of variance, HOV), then we proceed with Dunnett.

I am comparing measurements from treatment groups to a control group, so i perform ANOVA + post hoc dunnett test. However, the treatment/control groups have unequal sample size (n=3 or 4)


1) can i proceed with Dunnett in this situation (unequal sample size but having the homogeneity of variance, HOC assumption met)?

2) and what if I have unequal sample size and violated HOV? from searches there are recommendation on using Games-Howell, Dunnett C, T3, or Temphane T2, but none of seem seem to compare to a control group (like Dunnett test)

3) Addition: Also any advice if having equal sample size but violated HOV?

Really appreciate anyone could help on this. I/m struggling ...

Many thanks,
Cheng Foh

[chengfoh]

Dear Marso:

There are actually >1 treatment groups (Drug 1, drug 2, drug 3, drug 4, drug5) to be compared to the control and to see whether the treatment with the different drugs has significant effects to the mice as compared ot the untreated control.

With 3/4 mice do you think that is insufficient? form what i have pre-ran is that the Dunnett test able to detec some significant changes, but i am kinda concern with whether Dunnett test is appropriate for me in this case as it is unequal sample size

CF

[David Marso]

So you are torturing small furry creatures?
FOAD!!!!
--

chengfoh wrote

Dear Marso:

There are actually >1 treatment groups (Drug 1, drug 2, drug 3, drug 4, drug5) to be compared to the control and to see whether the treatment with the different drugs has significant effects to the mice as compared ot the untreated control.

With 3/4 mice do you think that is insufficient? form what i have pre-ran is that the Dunnett test able to detec some significant changes, but i am kinda concern with whether Dunnett test is appropriate for me in this case as it is unequal sample size

CF

[Swank, Paul R]

If you have unequal variances and unequal n's, you can do a t test for each pairwise compariosn you make, using the unequal variance result and Bonferroni to control the family-wise error rate. So if you have 4 reatments and one control, do 4 t tests using the eunequal variance result and test each at the .05/4=.0125 level. This is much less conservative than using Bonferroni with all pairwise comaprisons or .05/10 = .005.

Paul R. Swank, Ph.D., Professor
Health Promotions and Behavioral Sciences
School of Public Health
University of Texas Health Science Center Houston
________________________________________
From: SPSSX(r) Discussion [[hidden email]] On Behalf Of chengfoh [[hidden email]]
Sent: Tuesday, January 15, 2013 8:21 AM
To: [hidden email]
Subject: Violation of ANOVA post-hoc test

Dear friends :

Hope someone has any idea about my problem.

From what i understand, if we have equal sample size and met the assumption
of homogeniety of variance, HOV), then we proceed with Dunnett.

I am comparing measurements from treatment groups to a control group, so i
perform ANOVA + post hoc dunnett test. However, the treatment/control groups
have unequal sample size (n=3 or 4)


1) can i proceed with Dunnett in this situation (unequal sample size but
having the homogeneity of variance, HOC assumption met)?

2) and what if I have unequal sample size and violated HOV? from searches
there are recommendation on using Games-Howell, Dunnett C, T3, or Temphane
T2, but none of seem seem to compare to a control group (like Dunnett test)

3) Addition: Also any advice if having equal sample size but violated HOV?

Really appreciate anyone could help on this. I/m struggling ...

Many thanks,
Cheng Foh



--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Violation-of-ANOVA-post-hoc-test-tp5717388.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Ware, William B]

To this I would add that you might consider one of the sequentially-rejective modifications of the Bonferroni approach.  One is the Holm (1979) modification in which the four tests would be completed and resulting p-values from smallest to largest.  Then, the smallest p-value would be compared to an adjusted alpha of .05/4, or .0125.  If that is not "significant," stop!  However, if the first contrast is significant at .0125, then the second smallest p-value is compared to .05/3, or .0167.  If non-significant, stop.  However, if it is significant, test the third at .05/2, or .025.  The last test would be completed at the .05 level, if you get that far.  As you can see, the adjustment is accomplished by decreasing the denominator.

An alternative is the "false discovery rate" of Benjamini and Hochberg (1995).  It is accomplished by modifying the numerator.  The first test is completed at 1*.05/4.  The second at 2*.05/4.  etc.

Either is preferable to the "straight-up" Bonferroni correction, which is overly conservative.

wbw

William B. Ware, Ph.D.
McMichael Professor of Education 2011-2013
Educational Psychology, Measurement, and Evaluation
CB #3500 - 118 Peabody Hall
University of North Carolina at Chapel Hill
Chapel Hill, NC     27599-3500
Office: (919)-962-2511
Fax:    [phone]
Office:  118 Peabody Hall
EMAIL: [hidden email] <mailto:[hidden email]>
Adjunct Professor, School of Social Work
Academy of Distinguished Teaching Scholars at UNC-Chapel Hill



-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Swank, Paul R
Sent: Wednesday, January 16, 2013 4:18 PM
To: [hidden email]
Subject: Re: Violation of ANOVA post-hoc test

If you have unequal variances and unequal n's, you can do a t test for each pairwise compariosn you make, using the unequal variance result and Bonferroni to control the family-wise error rate. So if you have 4 reatments and one control, do 4 t tests using the eunequal variance result and test each at the .05/4=.0125 level. This is much less conservative than using Bonferroni with all pairwise comaprisons or .05/10 = .005.

Paul R. Swank, Ph.D., Professor
Health Promotions and Behavioral Sciences School of Public Health University of Texas Health Science Center Houston ________________________________________
From: SPSSX(r) Discussion [[hidden email]] On Behalf Of chengfoh [[hidden email]]
Sent: Tuesday, January 15, 2013 8:21 AM
To: [hidden email]
Subject: Violation of ANOVA post-hoc test

Dear friends :

Hope someone has any idea about my problem.

From what i understand, if we have equal sample size and met the assumption of homogeniety of variance, HOV), then we proceed with Dunnett.

I am comparing measurements from treatment groups to a control group, so i perform ANOVA + post hoc dunnett test. However, the treatment/control groups have unequal sample size (n=3 or 4)


1) can i proceed with Dunnett in this situation (unequal sample size but having the homogeneity of variance, HOC assumption met)?

2) and what if I have unequal sample size and violated HOV? from searches there are recommendation on using Games-Howell, Dunnett C, T3, or Temphane T2, but none of seem seem to compare to a control group (like Dunnett test)

3) Addition: Also any advice if having equal sample size but violated HOV?

Really appreciate anyone could help on this. I/m struggling ...

Many thanks,
Cheng Foh



--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Violation-of-ANOVA-post-hoc-test-tp5717388.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Swank, Paul R]

Howsever, they do not control for the family-wise error and so must be done in concert with a significant omnibus test.

Paul R. Swank, Ph.D., Professor
Health Promotions and Behavioral Sciences
School of Public Health
University of Texas Health Science Center Houston
________________________________________
From: Ware, William B [[hidden email]]
Sent: Wednesday, January 16, 2013 4:54 PM
To: Swank, Paul R; [hidden email]
Subject: RE: Violation of ANOVA post-hoc test

To this I would add that you might consider one of the sequentially-rejective modifications of the Bonferroni approach.  One is the Holm (1979) modification in which the four tests would be completed and resulting p-values from smallest to largest.  Then, the smallest p-value would be compared to an adjusted alpha of .05/4, or .0125.  If that is not "significant," stop!  However, if the first contrast is significant at .0125, then the second smallest p-value is compared to .05/3, or .0167.  If non-significant, stop.  However, if it is significant, test the third at .05/2, or .025.  The last test would be completed at the .05 level, if you get that far.  As you can see, the adjustment is accomplished by decreasing the denominator.

An alternative is the "false discovery rate" of Benjamini and Hochberg (1995).  It is accomplished by modifying the numerator.  The first test is completed at 1*.05/4.  The second at 2*.05/4.  etc.

Either is preferable to the "straight-up" Bonferroni correction, which is overly conservative.

wbw

William B. Ware, Ph.D.
McMichael Professor of Education 2011-2013
Educational Psychology, Measurement, and Evaluation
CB #3500 - 118 Peabody Hall
University of North Carolina at Chapel Hill
Chapel Hill, NC     27599-3500
Office: (919)-962-2511
Fax:    [phone]
Office:  118 Peabody Hall
EMAIL: [hidden email] <mailto:[hidden email]>
Adjunct Professor, School of Social Work
Academy of Distinguished Teaching Scholars at UNC-Chapel Hill



-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Swank, Paul R
Sent: Wednesday, January 16, 2013 4:18 PM
To: [hidden email]
Subject: Re: Violation of ANOVA post-hoc test

If you have unequal variances and unequal n's, you can do a t test for each pairwise compariosn you make, using the unequal variance result and Bonferroni to control the family-wise error rate. So if you have 4 reatments and one control, do 4 t tests using the eunequal variance result and test each at the .05/4=.0125 level. This is much less conservative than using Bonferroni with all pairwise comaprisons or .05/10 = .005.

Paul R. Swank, Ph.D., Professor
Health Promotions and Behavioral Sciences School of Public Health University of Texas Health Science Center Houston ________________________________________
From: SPSSX(r) Discussion [[hidden email]] On Behalf Of chengfoh [[hidden email]]
Sent: Tuesday, January 15, 2013 8:21 AM
To: [hidden email]
Subject: Violation of ANOVA post-hoc test

Dear friends :

Hope someone has any idea about my problem.

From what i understand, if we have equal sample size and met the assumption of homogeniety of variance, HOV), then we proceed with Dunnett.

I am comparing measurements from treatment groups to a control group, so i perform ANOVA + post hoc dunnett test. However, the treatment/control groups have unequal sample size (n=3 or 4)


1) can i proceed with Dunnett in this situation (unequal sample size but having the homogeneity of variance, HOC assumption met)?

2) and what if I have unequal sample size and violated HOV? from searches there are recommendation on using Games-Howell, Dunnett C, T3, or Temphane T2, but none of seem seem to compare to a control group (like Dunnett test)

3) Addition: Also any advice if having equal sample size but violated HOV?

Really appreciate anyone could help on this. I/m struggling ...

Many thanks,
Cheng Foh



--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Violation-of-ANOVA-post-hoc-test-tp5717388.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD