Re: Bonferroni correction and number of comparisons

Posted by Bruce Weaver on Jan 19, 2011; 11:04pm
URL: http://spssx-discussion.165.s1.nabble.com/Bonferroni-correction-and-number-of-comparisons-tp3348337p3348760.html

Hi Ryan.  Exporting to Excel and showing lots of decimals shows that you are correct.  I.e., SPSS appears to be computing the Bonferronii-corrected p-value as the uncorrected p * the number of contrasts, and if the result is greater than 1, setting it to 1.  I expect the details are given somewhere in the ONEWAY algorithms, if Jan wants to confirm this before proceeding.

R B wrote

Bruce,

See a couple of comments interspersed below.

On Wed, Jan 19, 2011 at 4:47 PM, Bruce Weaver <[e-mail]> wrote:
> Ryan, multiplying the observed (uncorrected) p-values by the number of
> contrasts can give you results that make no sense.  E.g., if the uncorrected
> p-value = .25 and the number of contrasts is 6, you'll get a corrected
> p-value = 1.5.  But p-values are conditional probabilities, and must fall
> within the range 0-1.

This is a valid point, but I've yet to encounter a situation in my
work where I would've made a different conclusion by correcting alpha
directly.

>
> Here's an example using oneway ANOVA.  Notice that the Bonferroni-corrected
> p-values (or Sig. values as SPSS labels them) are  not simply 3 x the
> uncorrected (LSD) p-values.

I disagree. In your example, Bonferroni corrected p values associated
with 1v2 and 1v3 are exactly the same uncorrected p-values times 3. Is
there something I'm missing here?

My guess is that SPSS automatically replaced the Bonferroni corrected
p-value associated with 2v3 with 1.0, since as you pointed out the
conditional probability cannot be above 1.0.

>
> MATRIX DATA VARIABLES=Group ROWTYPE_ Score /FACTORS=Group.
> BEGIN DATA
> 1 N 96
> 2 N 96
> 3 N 96
> 1 MEAN 22.98
> 2 MEAN 25.78
> 3 MEAN 26.56
> 1 STDDEV 8.79
> 2 STDDEV 9.08
> 3 STDDEV 8.50
> END DATA.
>
> ONEWAY Score BY group /
>  matrix = in(*) /
>  POSTHOC=LSD BONFERRONI ALPHA(0.05)
> .
>
> Jan, given that figuring out how to compute corrected p-values may be quite
> time-consuming, I would see if I could persuade the person who asked that
> the usual approach (i.e., comparing p to a corrected alpha level) is
> sufficient.
>
> HTH.
>
>
>
> R B wrote:
>>
>> You asked about  applying a Bonferroni correction in post hoc tests-->
>> "Is the Bonferroni adjustment for 6 comparisons and the corrected
>> p-value (.05/6=.008333)"
>>
>> Answer. No. You have presumably adjusted the alpha level, not the
>> p-values. Multiply the each p-value by the number of post hoc tests
>> performed (6 in the example you provided). Then compare the adjusted
>> p-values to the alpha level you set (e.g., .05).
>>
>> Ryan
>>
>> On Wed, Jan 19, 2011 at 1:56 PM, J McClure <[e-mail]> wrote:
>>> Hi,
>>> I am doing a post hoc analysis comparing column proportions and
>>> adjusting (Bonferroni) for multiple comparisons (CTables, test
>>> statistics option).
>>> I've been asked to provide the corrected p-value.
>>> I'd like to know if I am correct in reporting the corrected value as
>>> .008.
>>> The row variable has two levels (yes and no) and the column variable has
>>> 4 levels, so I am making 6 comparisons for 'no' and 6 for 'yes'.
>>> n*(n-1)/2
>>> The output shows the comparison for the 'no'  and for the yes level of
>>> the row variable. Is the Bonferroni adjustment for 6 comparisons and the
>>> corrected p-value (.05/6=.008333)
>>> Also, I have 19 row variables for which I am running column proportion
>>> comparisons. I think that the comparisons for each of the 19 variables
>>> are considered independent from each other so no further adjustment is
>>> made by SPSS.  Is this the way reviewers look at the question?
>>> Thanks for any help,
>>> Jan
>>>
>>> =====================
>>> To manage your subscription to SPSSX-L, send a message to
>>> [e-mail] (not to SPSSX-L), with no body text except the
>>> command. To leave the list, send the command
>>> SIGNOFF SPSSX-L
>>> For a list of commands to manage subscriptions, send the command
>>> INFO REFCARD
>>>
>>
>> =====================
>> To manage your subscription to SPSSX-L, send a message to
>> [e-mail] (not to SPSSX-L), with no body text except the
>> command. To leave the list, send the command
>> SIGNOFF SPSSX-L
>> For a list of commands to manage subscriptions, send the command
>> INFO REFCARD
>>
>>
>
>
> -----
> --
> Bruce Weaver
> [e-mail]
> http://sites.google.com/a/lakeheadu.ca/bweaver/
>
> "When all else fails, RTFM."
>
> NOTE: My Hotmail account is not monitored regularly.
> To send me an e-mail, please use the address shown above.
>
> --
> View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Bonferroni-correction-and-number-of-comparisons-tp3348337p3348646.html
> Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [e-mail] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>

=====================
To manage your subscription to SPSSX-L, send a message to
[e-mail] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD