Fw: Size or significance of correlations

[Mike]

 

----- Original Message -----

From: [hidden email]

To: [hidden email]

Cc: [hidden email]

Sent: Wednesday, May 12, 2010 8:43 AM

Subject: Re: Size or significance of correlations

First, compute the overall Type I error rate with the following formula:

 

overall alpha = ( 1 - (1-alpha percomparison)**K).

 

Where alpha percomparison is the Type I error rate, that is, the

Type I error rate you are using for each correlation, presumably

you claim that a correlation is statistically significant at the 0.05

level.

K is the number of correlations you're evaluating.

 

If you only had 3 correlations each using .05 alpha per comparison,

the above formula would look like this:

 

Overall alpha = (1 - (1 - .05)**3))

  = (1-(.95)**3)= (1 - .8574) = .1426

 

That is, you have about 14% chance of having commited a Type I

error after evaluating the three correlations.

 

Many people would consider an overall Type I error rate of 14%

as being too high (If you have more correlations, expect the

overall Type I error rate to skyrocket).  Some would suggest

fixing the Overall Type I error rate to 0.05 and then use an

adjusted per comparison alpha for each correlation.  The

 

Bonferroni correction allow you to set the Overall alpha = .05

and then use the following to calculate the per comparison alpha:

 

per comparison alpha = .05/K

 

Where K, as above, refer to the number of correlations you are

evaluating.  For K=3, per comparison alpha = .05/3 = .01667.

This means that correlations with a p-value less than .01667

are considered significant.  For a large number of correlations,

the per comparison alpha can be quite small.

 

There are more things you can do (e.g., adjust per comparison

alpha so that important correlation have a small value, etc.)

but it appears that you have limited knowledge about the nature

of the correlation and the phenomena they relate to.  That is,

If you had previous research and/or good theory about the nature

of the correlation, you would not be asking how to interpret your

obtained correlations because these would serve as your guide.

Without such guides, you have to purely mechanical guides

such as the Bonferroni correction.

 

In addition, regarding which correlations you should "trust",

I would recommend trusting only those correlations that are

shown through replications of the research they come from

to continue to be statistical significant.  Otherwise, mere size

of the correlation coefficient or its probability are at best only

tentative sources of information.  Even small values of a

correlation can be theoretically meaningful and non-significant

correlation may be non-significant because of a lack of statistical

power.

 

-Mike Palij

New York University

[hidden email]

 

 

----- Original Message -----

From: [hidden email]

To: [hidden email]

Sent: Wednesday, May 12, 2010 6:58 AM

Subject: Size or significance of correlations

Dear all,
I have acorrealted a neumber of varibales (Pearson) and have found correlations as low as .15 to be significant at 0.01 level (two-tailed). My sample size is 280.
I dont know how to interpret this.
the coefficinets are very small however statistically significant.
Which one should I trust, the size of the coefficients or the statistical significnace?
I will be thankful for comments.
Cheers
Humphrey