http://spssx-discussion.165.s1.nabble.com/Power-estimated-sample-size-and-conundrum-tp5726815p5726820.html
If a proposed table of outcomes gives you a test exactly at the 5% alpha cutoff,
then the Ns of that table give you a "power" of 50%. - This follows immediately
from an elementary logical consideration: If the proposed table represents
the underlying effect, then, by chance, half of future results will be better and
half will be worse. That is one of the first things to learn, to get oriented to
understanding power analysis.
If we want a study to have 80% power instead of 50% power, then we need a
notably larger N than the N that gives 50% power. For a 2x2 table, you have
50% power where the X^2 is about 4.0 (obviously approximately), and 80%
power at 8.0 (look up the exact figures). For chi-squared, where X^2= N*r^2,
that implies a doubling of the N in order to raise 50% power to 80% power,
since 8.0 is double 4.0. That is what you observed.
Also handy: the effect size and N that gives a p-value of 0.001 for a test with
1 d.f. is going to have about 80% power for a future test at 5%.
You did a power analysis. And then you ignored it, because you never learned
what power analysis is about. And then you got burned. Does that sum it up?
--
Rich Ulrich
Date: Thu, 24 Jul 2014 09:43:56 -0700
From:
[hidden email]Subject: Power, estimated sample size and conundrum!
To:
[hidden email]
Greetings all....I have a
question/situation that on the surface seems very transparent but I am having
difficulties navigating. So any feedback will be much appreciated.
For a study two years
ago I conducted a power analysis comparing two proportions (p1 = .085 vs. p2 =
.045). Whether I used G*Power, Stata, or Power and Precision they all gave me
sample size estimates ranging from 600 to 640 (contingent if continuity
correction was incorporated) per group for alpha = .05 and power of .80.
However, when I ran the
SPSS macro for z test of proportions for two groups comparing .045 vs. .085, I found that n = 300 per
group was sufficient to obtain significance: z = -1.99, p = .047. I also
confirmed this in Stata using the following syntax: prtesti 300 .045 300 .085.
Hence, can I assume that
the difference (i.e., n = 600 per group per power analysis vs. n = 300 per
group being sufficient to obtain significance) is a function of the desired
power insofar with the conventional power of .80 you are setting the bar high
enough so as to find the optimal nexus of Type I and Type II error?
The conundrum is such
that our study ended up with p1 = .027 vs. p2 =.047 and p = .194 with the recommended sample
size being n = 300 per group given the simulation I ran in SPSS and Stata. Note that post hoc power analysis indicates I
would have needed n = 1496 per group (alpha =.05, power = .80) to obtain significance
for delta of 2%, though when I run .027 vs. .047 in Stata and SPSS with n = 685
per group z = 1.96, p = .05.
Anyway, this has become
an interesting/challenging discussion with PI and reviewers alike. We went with the sample size (n = 300 per
group) since that was sufficient for obtaining significance, but they are indicting we
should have gone with the larger sample size based on the power estimate (i.e.,
n = 600 per group).
Has anyone encountered
such a dilemma and how did you deal with it?
Thank you….Dale
=====================
To manage your subscription to SPSSX-L, send a message to
(not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD