Q-Values and Homogeneity of effects sizes in Meta-analysis

[Laurie Petch]

I have calculated the Q-value of the effects sizes using a formula in Excel,
which I am 99% sure is correct. If this is right, I will export the data to
SPSS for further analysis. The degrees of freedom is 41 (for 42 studies).
The Q-value comes out as -82.47. I have checked a chi-squared distribution
table. Should I ignore the negative sign here, and treat it as if it were
82.47? If so, this would be greater than the p=0.05 critical value of 55.76
(for df 40 in chi-squared table), in which case I would accept the
hypothesis of homogeneity at alpha=0.05. The variance in the sample of
effects size _is_ greater than would be expected due to sampling error
alone. That is to say, it is probably due to the effects of the study
characteristics.

Am I on the right lines?

Thanks

Laurie

------
Laurie Petch
Chartered Educational Psychologist
(British Psychological Society)

Regina, Canada

[Marta García-Granero]

Hi Laurie

The Q-value CAN'T be negative. Check the formula, it must be wrong...
Perhaps you would like to try some meta-analytic code I wrote time
aga. It's at Ray's web page:

http://www.spsstools.net/SampleSyntax.htm#MetaAnalysis

I have validated them against published datasets using Egger,Smith &
Altman "Systematic Reviews in Health Care" (BMJ books).

Recently I modified the files and turnt them into a short collection
of MACROS (I'm in the process of publishing them, but I can send them
to you if you want).

LP> I have calculated the Q-value of the effects sizes using a formula in Excel,
LP> which I am 99% sure is correct. If this is right, I will export the data to
LP> SPSS for further analysis. The degrees of freedom is 41 (for 42 studies).
LP> The Q-value comes out as -82.47. I have checked a chi-squared distribution
LP> table. Should I ignore the negative sign here, and treat it as if it were
LP> 82.47? If so, this would be greater than the p=0.05 critical value of 55.76
LP> (for df 40 in chi-squared table), in which case I would accept the
LP> hypothesis of homogeneity at alpha=0.05. The variance in the sample of
LP> effects size _is_ greater than would be expected due to sampling error
LP> alone. That is to say, it is probably due to the effects of the study
LP> characteristics.

LP> Am I on the right lines?

--
Regards,
Dr. Marta García-Granero,PhD           mailto:[hidden email]
Statistician

---
"It is unwise to use a statistical procedure whose use one does
not understand. SPSS syntax guide cannot supply this knowledge, and it
is certainly no substitute for the basic understanding of statistics
and statistical thinking that is essential for the wise choice of
methods and the correct interpretation of their results".

(Adapted from WinPepi manual - I'm sure Joe Abrahmson will not mind)

[Laurie Petch]

Thanks for your reply Martha. It's good to find out that I'm going wrong,
but hard to see where. I have checked the formula using the worked examples
given in Lipsey & Wilson (2001). Practical meta-analysis. Thousand Oaks:
Sage and the formulae give the correct results based on their data.... I've
just tried a different way:

=AN2*((AL2-$AO$47)^2)

where AN2 is the weighted inverse varience weight for the effects size in
question, AL2 is the unbiassed effects size in question and $AO$47 is the
weighted mean effects size. I have then summed these figures to give a Q of
88.84. This is well above the p=0.05 critical value of 55.76, which is
given in my chi squared distribution table for df=40, the closest I have to
my df (41). Is this any better?

I'll try the code you wrote. If you could send the macros, that would be a
huge help.

kind regards,

Laurie

MG-G> The Q-value CAN'T be negative. Check the formula, it must be wrong...
>Perhaps you would like to try some meta-analytic code I wrote time
>ago. It's at Ray's web page:
>
>http://www.spsstools.net/SampleSyntax.htm#MetaAnalysis

----
Laurie Petch
Chartered Educational Psychologist

Regina, Canada

[Marta García-Granero]

Hi Laurie

LP> Thanks for your reply Martha. It's good to find out that I'm going wrong,
LP> but hard to see where. I have checked the formula using the worked examples
LP> given in Lipsey & Wilson (2001). Practical meta-analysis. Thousand Oaks:
LP> Sage and the formulae give the correct results based on their data.... I've
LP> just tried a different way:

LP> =AN2*((AL2-$AO$47)^2)

Yes, this is the formula.

LP> where AN2 is the weighted inverse varience weight for the effects size in
LP> question, AL2 is the unbiassed effects size in question and $AO$47 is the
LP> weighted mean effects size. I have then summed these figures to give a Q of
LP> 88.84. This is well above the p=0.05 critical value of 55.76, which is
LP> given in my chi squared distribution table for df=40, the closest I have to
LP> my df (41).

If you are using Excel, then you can get the p-value for your Q
statistic using one of the statistical functions imbedded in the
program: in English is CHIDIST, in French LOY.KHIDEUX (I've always
found surprising the fact that Excel functions have different names
according to the language...)

LP> I'll try the code you wrote. If you could send the macros, that
LP> would be a huge help.

Are you doing meta-analysis of countinuous or binary outcomes? (just
to know which one to send).


--
Regards,
Dr. Marta García-Granero,PhD           mailto:[hidden email]
Statistician

---
"It is unwise to use a statistical procedure whose use one does
not understand. SPSS syntax guide cannot supply this knowledge, and it
is certainly no substitute for the basic understanding of statistics
and statistical thinking that is essential for the wise choice of
methods and the correct interpretation of their results".

(Adapted from WinPepi manual - I'm sure Joe Abrahmson will not mind)