SPSS MACRO: Robust variance estimation in meta-regression with dependent effect size estimates

[David Marso]

Hello All,
This post builds from a recent thread:
"Multiple Meta-Regression SPSS Macro Issues" Initiated Oct 23, 2013
http://spssx-discussion.1045642.n5.nabble.com/Multiple-Meta-Regression-SPSS-Macro-Issues-tt5722677.html
In this thread a user was having difficulty running a rather dated SPSS macro called MetaReg.sps
programmed by David Wilson.  After a few iterations the user's issue was resolved.
------------------------------------------------------------------------------------------------
It is interesting how things sometimes come full circle.  
A couple of years ago I contracted to write an SPSS macro to perform meta-analysis which used the Wilson MetaReg as a baseline. This extends the work in several useful ways, the chief of which is to calculate robust standard errors for meta-analyses where the units of interest are not independent.  

I HOPE THE FOLLOWING FORMATS PROPERLY.

Since the postings of Oct 23, I have obtained permission from Emily Tanner-Smith (the person I contracted with) to make this macro publicly available.

Note that this macro is more or less a literal translation of code written in the R programming
language by Larry Hedges and described in

Robust variance estimation in meta-regression with dependent effect size estimates
Larry V. Hedges, Elizabeth Tipton and Matthew C. Johnson
Research Synthesis Methods 1: 39-65

Following is the abstract from that paper:

"Conventional meta-analytic techniques rely on the assumption that effect size estimates from different studies are independent and have sampling distributions with known conditional variances. The independence assumption is violated when studies produce several estimates based on the same individuals or there are clusters of studies that are not independent (such as those carried out by the same investigator or laboratory). This paper provides an estimator of the covariance matrix of meta-regression coefficients that are applicable when there are clusters of internally
correlated estimates. It makes no assumptions about the specific form of the sampling distributions of the effect sizes, nor does it require knowledge of the covariance structure of the dependent estimates. Moreover, this paper demonstrates that the meta-regression coefficients are consistent and asymptotically normally distributed and that the robust variance estimator is valid even when the covariates are random. The theory is asymptotic in the number of  studies, but simulations suggest that the theory may yield accurate results with as few as 20–40 studies. "
----
Copyright © 2010 John Wiley & Sons, Ltd."

Instructions for use are included in the header of the macro.

In addition, Emily provided me with a link to a paper she wrote with Elizabeth Tipton.
http://onlinelibrary.wiley.com/doi/10.1002/jrsm.1091/abstract 

You will need access to Wiley Online Library or Wiley InterScience to acquire the paper
(I believe most University libraries will have this.)
I cannot redistribute the paper due to copyright restrictions.

Tanner-Smith, E. E. and Tipton, E. (2013),
Robust variance estimation with dependent effect sizes:
practical considerations including a software tutorial in Stata and SPSS. Res. Synth. Method. doi: 10.1002/jrsm.1091

However here is the abstract from the paper.

"Methodologists have recently proposed robust variance estimation as one way to handle dependent effect sizes in meta-analysis.  Software macros for robust variance estimation in meta-analysis are currently available for Stata(StataCorp LP,CollegeStation,TX,USA) and SPSS (IBM,Armonk,NY,USA), yet there is little guidance for authors regarding the practical application and implementation of  those macros. This paper provides a brief tutorial on the implementation of the Stata and SPSS macros and discusses practical issues meta-analysts should consider when estimating meta-regression models with robust variance estimates.
Two example databases are used in the tutorial to illustrate the use of meta-analysis with robust variance estimates."

The instructions in the macro header should be sufficient to get up and running.
Any issues, feel free to post in this thread and I will attempt to assist to some degree.
I will ASSUME you HAVE read the macro header and followed the instructions.

Do not ask me questions about the specifics of the methodology
I am no expert, I merely did the programming.  
You should refer to the cited papers for the 411 on that.
Do not ask me to send you a copy of the papers because you can't access Wiley.
I have neither the time nor inclination to play librarian and I don't violate copyright for strangers.
---------------
Meanwhile, have fun with the Macro,
David
See Macro RobustMeta ATTACHED.
RobustMeta.sps

[Bruce Weaver]

Thanks David.  By the way, the Hedges et al. article is available free of charge here:

   http://psych.colorado.edu/~willcutt/pdfs/Hedges_2010.pdf

David Marso wrote

Hello All,
This post builds from a recent thread:
"Multiple Meta-Regression SPSS Macro Issues" Initiated Oct 23, 2013
http://spssx-discussion.1045642.n5.nabble.com/Multiple-Meta-Regression-SPSS-Macro-Issues-tt5722677.html
In this thread a user was having difficulty running a rather dated SPSS macro called MetaReg.sps
programmed by David Wilson.  After a few iterations the user's issue was resolved.
------------------------------------------------------------------------------------------------
It is interesting how things sometimes come full circle.  
A couple of years ago I contracted to write an SPSS macro to perform meta-analysis which used the Wilson MetaReg as a baseline. This extends the work in several useful ways, the chief of which is to calculate robust standard errors for meta-analyses where the units of interest are not independent.  

I HOPE THE FOLLOWING FORMATS PROPERLY.

Since the postings of Oct 23, I have obtained permission from Emily Tanner-Smith (the person I contracted with) to make this macro publicly available.

Note that this macro is more or less a literal translation of code written in the R programming
language by Larry Hedges and described in

Robust variance estimation in meta-regression with dependent effect size estimates
Larry V. Hedges, Elizabeth Tipton and Matthew C. Johnson
Research Synthesis Methods 1: 39-65

Following is the abstract from that paper:

"Conventional meta-analytic techniques rely on the assumption that effect size estimates from different studies are independent and have sampling distributions with known conditional variances. The independence assumption is violated when studies produce several estimates based on the same individuals or there are clusters of studies that are not independent (such as those carried out by the same investigator or laboratory). This paper provides an estimator of the covariance matrix of meta-regression coefficients that are applicable when there are clusters of internally
correlated estimates. It makes no assumptions about the specific form of the sampling distributions of the effect sizes, nor does it require knowledge of the covariance structure of the dependent estimates. Moreover, this paper demonstrates that the meta-regression coefficients are consistent and asymptotically normally distributed and that the robust variance estimator is valid even when the covariates are random. The theory is asymptotic in the number of  studies, but simulations suggest that the theory may yield accurate results with as few as 20–40 studies. "
----
Copyright © 2010 John Wiley & Sons, Ltd."

Instructions for use are included in the header of the macro.

In addition, Emily provided me with a link to a paper she wrote with Elizabeth Tipton.
http://onlinelibrary.wiley.com/doi/10.1002/jrsm.1091/abstract 

You will need access to Wiley Online Library or Wiley InterScience to acquire the paper
(I believe most University libraries will have this.)
I cannot redistribute the paper due to copyright restrictions.

Tanner-Smith, E. E. and Tipton, E. (2013),
Robust variance estimation with dependent effect sizes:
practical considerations including a software tutorial in Stata and SPSS. Res. Synth. Method. doi: 10.1002/jrsm.1091

However here is the abstract from the paper.

"Methodologists have recently proposed robust variance estimation as one way to handle dependent effect sizes in meta-analysis.  Software macros for robust variance estimation in meta-analysis are currently available for Stata(StataCorp LP,CollegeStation,TX,USA) and SPSS (IBM,Armonk,NY,USA), yet there is little guidance for authors regarding the practical application and implementation of  those macros. This paper provides a brief tutorial on the implementation of the Stata and SPSS macros and discusses practical issues meta-analysts should consider when estimating meta-regression models with robust variance estimates.
Two example databases are used in the tutorial to illustrate the use of meta-analysis with robust variance estimates."

The instructions in the macro header should be sufficient to get up and running.
Any issues, feel free to post in this thread and I will attempt to assist to some degree.
I will ASSUME you HAVE read the macro header and followed the instructions.

Do not ask me questions about the specifics of the methodology
I am no expert, I merely did the programming.  
You should refer to the cited papers for the 411 on that.
Do not ask me to send you a copy of the papers because you can't access Wiley.
I have neither the time nor inclination to play librarian and I don't violate copyright for strangers.
---------------
Meanwhile, have fun with the Macro,
David
See Macro RobustMeta ATTACHED.
RobustMeta.sps

[BillT]

Awesome, thanks for making this macro public  since multiple meta-regression is common in meta-analysis I'm sure people will find this helpful!

[LucyJ]

Dear David Marso,

Firstly, thank you so much for making available the macro to calculate the
random variance estimation (RVE) introduced in the paper.  This has been
especially useful for me as it is now possible I can include many more
different outcomes whilst accounting for the dependency.

I have used the SPSS macro and the output summarises the mean effect size,
along with the mean standard error.  The one thing I was wondering, is
whether there is a breakdown of this showing the adjusted effect sizes (as a
result of the RVE) and therefore the possibility of a forest plot?  I have a
seen a paper which uses the equivalent robumeta function in the R stats
package and they present a forest plot summarising the output values for all
individual outcomes (paper attached, plot on page 272).  I notice that
Hedges' used R and these macros are translated from R, so I just wondered if
the same output is a feature with the SPSS macro please?  

Any advice you have would be greatly appreciated!

Many thanks
Lucy.
<http://spssx-discussion.1045642.n5.nabble.com/file/t341851/RVE_forest_plot.png>








--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD