Effect size for Three-Level Hierarchical Linear Growth Model

[E. Bernardo]

Any thoughts about Power analysis for a  three-Level Hierarchical Linear Growth Model?.  I want to compute the effect size for the model given the sample size, statistical power, number of predictors at each level, and level of significance.  

Thank you.

[Todd Alan Zoblotsky (tzbltsky)]

Take a look at the Optimal Design software.  It is free, and is specifically designed for HLM analyses.

 

http://sitemaker.umich.edu/group-based/optimal_design_software

 

Todd Zoblotsky

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of E. Bernardo
Sent: Thursday, January 17, 2013 2:40 AM
To: [hidden email]
Subject: Effect size for Three-Level Hierarchical Linear Growth Model

 

Any thoughts about Power analysis for a  three-Level Hierarchical Linear Growth Model?.  I want to compute the effect size for the model given the sample size, statistical power, number of predictors at each level, and level of significance.  

 

Thank you.

[statisticsdoc]

Eins,

 

With regard to statistical power, are you looking at the effects of an intervention, or at naturally occurring associations between variables?  As noted by Todd, Optimal Design is a very valuable tool for power analysis, but like most HLM software it tends to focus on designs involving experimental assignment to treatment conditions.  

 

If you are interested in computing effect size per se in a randomized experiment, the following paper by Larry Hedges may be helpful:

Hedges, L. (2011). Effect Sizes in Three-Level Cluster-Randomized Experiments Journal of Educational and Behavioral Statistics June 2011 36: 346-380,

The Raudenbush and Bryk (2002) HLM text also has more general instructions for computing effect sizes at each level of a three level model.

 

Best,

 

Stephen Brand

 

www.StatisticsDoc.com

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of E. Bernardo
Sent: Thursday, January 17, 2013 3:40 AM
To: [hidden email]
Subject: Effect size for Three-Level Hierarchical Linear Growth Model

 

Any thoughts about Power analysis for a  three-Level Hierarchical Linear Growth Model?.  I want to compute the effect size for the model given the sample size, statistical power, number of predictors at each level, and level of significance.  

 

Thank you.

[Rich Ulrich]

I've done a lot of power analyses.  I've seen a number of really crappy
power analyses.  "Complicated"  is a common characteristic of many
of the crappy efforts. 

What do you propose to have as the units of your power analysis? 
  - "standard deviations of change"?  -  change in WHAT? 
 - various error terms will exist in a 3-level hierarchical model. 
These depend on not only the TOTAL sample size, but various ways
of apportioning the Ns ... which is also apt to be a matter that you can
manipulate, and see varying results.  (Does the suggested software
help with that?  - real question, not rhetorical or sarcastic.)

It is *often* possible to collapse a complicated design in order to get a
good approximation to the effects that concern you by looking at something
like a simple t-test, using whatever should be acceptable estimates for the
error.  Then you provide the estimates along with the observation that the
actual design should offer slightly more power than this. 

The power analysis is written (a) to help you figure out your sample size and
design, and (b) to help sell it to your granting agency.  For a complicated
design, the choices get more and more finicky, and (often) more and more
questionable.  And the result is usually going to be an approximation.

Keep it simple.  Keep it simpler than the actual design, when the design is
complicated.

--
Rich Ulrich

---

Date: Thu, 17 Jan 2013 16:40:16 +0800
From: [hidden email]
Subject: Effect size for Three-Level Hierarchical Linear Growth Model
To: [hidden email]

Any thoughts about Power analysis for a  three-Level Hierarchical Linear Growth Model?.  I want to compute the effect size for the model given the sample size, statistical power, number of predictors at each level, and level of significance.  

Thank you.

[Bruce Weaver]

Here's another way to approach it when you have a somewhat complicated model.

1. Simulate a population that has the effect size(s) you want power to detect.

2. Repeatedly sample from that population using a guess at the required sample size (this guess could come from the simplified approach Rich suggests), and run your model.  Power = the proportion of samples for which the null hypothesis is rejected.  (OMS is useful for writing the table containing the p-value to another dataset.)

HTH.

Rich Ulrich-2 wrote

I've done a lot of power analyses.  I've seen a number of really crappy
power analyses.  "Complicated"  is a common characteristic of many
of the crappy efforts.  

What do you propose to have as the units of your power analysis?  
  - "standard deviations of change"?  -  change in WHAT?  
 - various error terms will exist in a 3-level hierarchical model.  
These depend on not only the TOTAL sample size, but various ways
of apportioning the Ns ... which is also apt to be a matter that you can
manipulate, and see varying results.  (Does the suggested software
help with that?  - real question, not rhetorical or sarcastic.)

It is *often* possible to collapse a complicated design in order to get a
good approximation to the effects that concern you by looking at something
like a simple t-test, using whatever should be acceptable estimates for the
error.  Then you provide the estimates along with the observation that the
actual design should offer slightly more power than this.  

The power analysis is written (a) to help you figure out your sample size and
design, and (b) to help sell it to your granting agency.  For a complicated
design, the choices get more and more finicky, and (often) more and more
questionable.  And the result is usually going to be an approximation.

Keep it simple.  Keep it simpler than the actual design, when the design is
complicated.

--
Rich Ulrich

Date: Thu, 17 Jan 2013 16:40:16 +0800
From: [hidden email]
Subject: Effect size for Three-Level Hierarchical Linear Growth Model
To: [hidden email]

Any thoughts about Power analysis for a  three-Level Hierarchical Linear Growth Model?.  I want to compute the effect size for the model given the sample size, statistical power, number of predictors at each level, and level of significance.  
Thank you.