Power, estimated sample size and conundrum!

[Dale Glaser]

Greetings all....I have a question/situation that on the surface seems very transparent but I am having difficulties navigating.  So any feedback will be much appreciated.

For a study two years ago I conducted a power analysis comparing two proportions (p1 = .085 vs. p2 = .045).  Whether I used G*Power, Stata, or Power and Precision they all gave me sample size estimates ranging from 600 to 640 (contingent if continuity correction was incorporated) per group for alpha = .05 and power of .80.

However, when I ran the SPSS macro for z test of proportions for two groups comparing .045 vs. .085, I found that n = 300 per group was sufficient to obtain significance: z = -1.99, p = .047. I also confirmed this in Stata using the following syntax: prtesti 300 .045 300 .085.

Hence, can I assume that the difference (i.e., n = 600 per group per power analysis vs. n = 300 per group being sufficient to obtain significance) is a function of the desired power insofar with the conventional power of .80 you are setting the bar high enough so as to find the optimal nexus of Type I and Type II error?

The conundrum is such that our study ended up with p1 = .027  vs. p2 =.047 and p = .194 with the recommended sample size being n = 300 per group given the simulation I ran in SPSS and Stata.  Note that post hoc power analysis indicates I would have needed n = 1496 per group (alpha =.05, power = .80) to obtain significance for delta of 2%, though when I run .027 vs. .047 in Stata and SPSS with n = 685 per group z = 1.96, p = .05. 

 

Anyway, this has become an interesting/challenging discussion with PI and reviewers alike.  We went with the sample size (n = 300 per group) since that was sufficient for obtaining significance, but they are indicting we should have gone with the larger sample size based on the power estimate (i.e., n = 600 per group).

Has anyone encountered such a dilemma and how did you deal with it?

Thank you….Dale

 

 

 
 

 

 

Dale Glaser, Ph.D.
Principal--Glaser Consulting
Lecturer/Adjunct Faculty--SDSU/USD/Alliant
3115 4th Avenue
San Diego, CA 92103
phone: [phone]
fax: [phone]
email: [hidden email]
website: www.glaserconsult.com

---

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Rich Ulrich]

If a proposed table of outcomes gives you a test exactly at the 5% alpha cutoff,
then the Ns of that table give you a "power" of 50%.  - This follows immediately
from an elementary logical consideration:  If the proposed table represents
the underlying effect, then, by chance, half of future results will be better and
half will be worse.  That is one of the first things to learn, to get oriented to
understanding power analysis.

If we want a study to have 80% power instead of 50% power, then we need a
notably larger N than the N that gives 50% power.  For a 2x2 table, you have
50% power where the X^2 is about 4.0 (obviously approximately), and 80%
power at 8.0 (look up the exact figures).  For chi-squared, where X^2= N*r^2, 
that implies a doubling of the N in order to raise 50% power to 80% power,
since 8.0 is double 4.0.  That is what you observed.

Also handy:  the effect size and N that gives a p-value of 0.001 for a  test with
1 d.f.  is going to have about 80% power for a future test at 5%. 

You did a power analysis.  And then you ignored it, because you never learned
what power analysis is about.  And then you got burned.  Does that sum it up?

--
Rich Ulrich

---

Date: Thu, 24 Jul 2014 09:43:56 -0700
From: [hidden email]
Subject: Power, estimated sample size and conundrum!
To: [hidden email]

Greetings all....I have a question/situation that on the surface seems very transparent but I am having difficulties navigating.  So any feedback will be much appreciated.

For a study two years ago I conducted a power analysis comparing two proportions (p1 = .085 vs. p2 = .045).  Whether I used G*Power, Stata, or Power and Precision they all gave me sample size estimates ranging from 600 to 640 (contingent if continuity correction was incorporated) per group for alpha = .05 and power of .80.

However, when I ran the SPSS macro for z test of proportions for two groups comparing .045 vs. .085, I found that n = 300 per group was sufficient to obtain significance: z = -1.99, p = .047. I also confirmed this in Stata using the following syntax: prtesti 300 .045 300 .085.

Hence, can I assume that the difference (i.e., n = 600 per group per power analysis vs. n = 300 per group being sufficient to obtain significance) is a function of the desired power insofar with the conventional power of .80 you are setting the bar high enough so as to find the optimal nexus of Type I and Type II error?

The conundrum is such that our study ended up with p1 = .027  vs. p2 =.047 and p = .194 with the recommended sample size being n = 300 per group given the simulation I ran in SPSS and Stata.  Note that post hoc power analysis indicates I would have needed n = 1496 per group (alpha =.05, power = .80) to obtain significance for delta of 2%, though when I run .027 vs. .047 in Stata and SPSS with n = 685 per group z = 1.96, p = .05. 

 

Anyway, this has become an interesting/challenging discussion with PI and reviewers alike.  We went with the sample size (n = 300 per group) since that was sufficient for obtaining significance, but they are indicting we should have gone with the larger sample size based on the power estimate (i.e., n = 600 per group).

Has anyone encountered such a dilemma and how did you deal with it?

Thank you….Dale

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Bruce Weaver]

Speaking of Stata, here's what I get for your original question (i.e., sample size for proportions of .085 & .045):


. power twoproportions .085 .045

Performing iteration ...

Estimated sample sizes for a two-sample proportions test
Pearson's chi-squared test
Ho: p2 = p1  versus  Ha: p2 != p1

Study parameters:

        alpha =    0.0500
        power =    0.8000
        delta =   -0.0400  (difference)
           p1 =    0.0850
           p2 =    0.0450

Estimated sample sizes:

            N =      1192
  N per group =       596


I found this site helpful in working out how to do that:

  http://www.stata-press.com/manuals/power-sample-size-reference-manual/

HTH.

Dale Glaser wrote

Greetings all....I have a
question/situation that on the surface seems very transparent but I am having
difficulties navigating.  So any feedback will be much appreciated.


For a study two years
ago I conducted a power analysis comparing two proportions (p1 = .085 vs. p2 =
.045).  Whether I used G*Power, Stata, or Power and Precision they all gave me
sample size estimates ranging from 600 to 640 (contingent if continuity
correction was incorporated) per group for alpha = .05 and power of .80.



However, when I ran the
SPSS macro for z test of proportions for two groups comparing .045 vs. .085, I found that n = 300 per
group was sufficient to obtain significance: z = -1.99, p = .047. I also
confirmed this in Stata using the following syntax: prtesti 300 .045 300 .085.

Hence, can I assume that
the difference (i.e., n = 600 per group per power analysis vs. n = 300 per
group being sufficient to obtain significance) is a function of the desired
power insofar with the conventional power of .80 you are setting the bar high
enough so as to find the optimal nexus of Type I and Type II error?

The conundrum is such
that our study ended up with p1 = .027  vs. p2 =.047 and p = .194 with the recommended sample
size being n = 300 per group given the simulation I ran in SPSS and Stata.  Note that post hoc power analysis indicates I
would have needed n = 1496 per group (alpha =.05, power = .80) to obtain significance
for delta of 2%, though when I run .027 vs. .047 in Stata and SPSS with n = 685
per group z = 1.96, p = .05. 
 
Anyway, this has become
an interesting/challenging discussion with PI and reviewers alike.  We went with the sample size (n = 300 per
group) since that was sufficient for obtaining significance, but they are indicting we
should have gone with the larger sample size based on the power estimate (i.e.,
n = 600 per group).

Has anyone encountered
such a dilemma and how did you deal with it?

Thank you….Dale
 
 
 
 
 
 
Dale Glaser, Ph.D.
Principal--Glaser Consulting
Lecturer/Adjunct Faculty--SDSU/USD/Alliant
3115 4th Avenue
San Diego, CA 92103
phone: [phone]
fax: [phone]
email: [hidden email]
website: www.glaserconsult.com


________________________________

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Dale Glaser]

Thank you Bruce....yes, that is exactly what I got with Stata when I initially ran the power analysis  two years ago (i.e., n = 596 per group).  However, the discussion I've been having is when one runs a z-test for two proportions (whether using the SPSS macro or Stata option) n = 300 per group will suffice to obtain significance for 4.5% vs. 8.5% (z = 1.99, p = .047).

I do understand that with power, in part being based on the noncentral distribution, in conjunction with the rigors of the desired power (e.g., .8) may make sample size estimates larger than what one needs to obtain significance via simulation (as I did with the SPSS macro for z test of proportions). However, it seems that there is a nontrivial difference between n = 300 per group sufficing to have p < .05 (per the z test) as opposed to n = 596 per group as per the power analysis (with desired power of .8, two tailed test, and alpha = .05)

So though I understand the statistical difference between conducting power and running the actual z-test, I'm having difficulties reconciling the large sample size differences when discussing this with the PI (and making the ultimate recommendation).  Any feedback how you all broach the subject re: difference of sample size estimate via power analysis as opposed to simulation obtaining the actual test statistic (and p-value) would be much appreciated.

Thank you....Dale

 

Dale Glaser, Ph.D.
Principal--Glaser Consulting
Lecturer/Adjunct Faculty--SDSU/USD/Alliant
3115 4th Avenue
San Diego, CA 92103
phone: [phone]
fax: [phone]
email: [hidden email]
website: www.glaserconsult.com

---

From: Bruce Weaver <[hidden email]>
To: [hidden email]
Sent: Thursday, July 24, 2014 6:38 PM
Subject: Re: Power, estimated sample size and conundrum!

Speaking of Stata, here's what I get for your original question (i.e., sample
size for proportions of .085 & .045):


. power twoproportions .085 .045

Performing iteration ...

Estimated sample sizes for a two-sample proportions test
Pearson's chi-squared test
Ho: p2 = p1  versus  Ha: p2 != p1

Study parameters:

        alpha =    0.0500
        power =    0.8000
        delta =  -0.0400  (difference)
          p1 =    0.0850
          p2 =    0.0450

Estimated sample sizes:

            N =      1192
  N per group =      596


I found this site helpful in working out how to do that:

  http://www.stata-press.com/manuals/power-sample-size-reference-manual/

HTH.




Dale Glaser wrote

> Greetings all....I have a
> question/situation that on the surface seems very transparent but I am
> having
> difficulties navigating.  So any feedback will be much appreciated.
>
>
> For a study two years
> ago I conducted a power analysis comparing two proportions (p1 = .085 vs.
> p2 =
> .045).  Whether I used G*Power, Stata, or Power and Precision they all
> gave me
> sample size estimates ranging from 600 to 640 (contingent if continuity
> correction was incorporated) per group for alpha = .05 and power of .80.
>
>
>
> However, when I ran the
> SPSS macro for z test of proportions for two groups comparing .045 vs.
> .085, I found that n = 300 per
> group was sufficient to obtain significance: z = -1.99, p = .047. I also
> confirmed this in Stata using the following syntax: prtesti 300 .045 300
> .085.
>
> Hence, can I assume that
> the difference (i.e., n = 600 per group per power analysis vs. n = 300 per
> group being sufficient to obtain significance) is a function of the
> desired
> power insofar with the conventional power of .80 you are setting the bar
> high
> enough so as to find the optimal nexus of Type I and Type II error?
>
> The conundrum is such
> that our study ended up with p1 = .027  vs. p2 =.047 and p = .194 with the
> recommended sample
> size being n = 300 per group given the simulation I ran in SPSS and
> Stata.  Note that post hoc power analysis indicates I
> would have needed n = 1496 per group (alpha =.05, power = .80) to obtain
> significance
> for delta of 2%, though when I run .027 vs. .047 in Stata and SPSS with n
> = 685
> per group z = 1.96, p = .05. 
>  
> Anyway, this has become
> an interesting/challenging discussion with PI and reviewers alike.  We
> went with the sample size (n = 300 per
> group) since that was sufficient for obtaining significance, but they are
> indicting we
> should have gone with the larger sample size based on the power estimate
> (i.e.,
> n = 600 per group).
>
> Has anyone encountered
> such a dilemma and how did you deal with it?
>
> Thank you….Dale
>  
>  
>  
>  
>  
>  
> Dale Glaser, Ph.D.
> Principal--Glaser Consulting
> Lecturer/Adjunct Faculty--SDSU/USD/Alliant
> 3115 4th Avenue
> San Diego, CA 92103
> phone: [phone]
> fax: [phone]
> email:

> glaserconsult@

> website: www.glaserconsult.com

>
>
> ________________________________
>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> [hidden email]

>  (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Power-estimated-sample-size-and-conundrum-tp5726815p5726821.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Bruce Weaver]

Here's a simulation I cobbled together.  When I plug in 600 per group, I get power of around 80% for each of the methods CROSSTABS offers (as expected, given my sample size estimate obtained via Stata).  When I plug in 300 per group, I get power of around 50%.

HTH.


* Simulate power analysis for comparing two
* independent proportions when the true
* proportions are .085  and .045.
* Use a macros to set the N per group
* and the number of iterations.

* Sample sizes of 600 per group were computed via
* Stata-- power twoproportions .085 .045 .

******************************** .
DEFINE !NperGrp () 600 !ENDDEFINE.
DEFINE !Iterations () 1000 !ENDDEFINE.
******************************** .

NEW FILE.
DATASET CLOSE all.

SET RNG=MT MTINDEX=RANDOM.

INPUT PROGRAM.
LOOP Table = 1 to !Iterations.
LEAVE Table.
LOOP row = 1 to 2.
LEAVE row.
LOOP col = 1 to 2.
END CASE.
END LOOP.
END LOOP.
END LOOP.
END FILE.
END INPUT PROGRAM.
EXECUTE.

* Generate the data.

DO IF row EQ 1.
- IF col EQ 1 kount = RV.BINOM(!NperGrp, .085).
ELSE.
- IF col EQ 1 kount = RV.BINOM(!NperGrp, .045).
END IF.
IF MISSING(kount) kount = !NperGrp - LAG(kount).
EXECUTE.
FORMATS Table kount (F5.0) / row col (F1).

* OMS.
DATASET DECLARE  Xtab.
OMS
  /SELECT TABLES
  /IF COMMANDS=['Crosstabs'] SUBTYPES=['Crosstabulation']
  /DESTINATION FORMAT=SAV NUMBERED=TableNumber_
   OUTFILE='Xtab' VIEWER=NO.
* OMS.
DATASET DECLARE  ChiSq.
OMS
  /SELECT TABLES
  /IF COMMANDS=['Crosstabs'] SUBTYPES=['Chi Square Tests']
  /DESTINATION FORMAT=SAV NUMBERED=TableNumber_
   OUTFILE='ChiSq' VIEWER=NO.

WEIGHT by kount.
CROSSTABS row by col by Table / Stat = chisqr.
OMSEND.

DATASET ACTIVATE Chisq.
SELECT if VAR1 NE "Total".
EXECUTE.
COMPUTE Significant = Asymp.Sig.2sided LE .05.
IF MISSING(Significant) Significant = ExactSig.2sided LE .05.
FORMATS Significant (f1).

* Use MEANS procedure to show the proportion of
* tests that achieve statistical significance.
* Remember, the mean of a 1-0 variable is the
* proportion of cases equal to 1.

MEANS  Significant by Var2 /cells = mean count.

* The Means show the power.
* The Linear-by-linear result is equaivalent to the N-1 chi-square
* (see http://www.iancampbell.co.uk/twobytwo/twobytwo.htm).
* NOTE:  Ignore the Total row.

* When I set N per group = 600, I get power around .80.
* When I set N per group = 300, I get power around .50.

Dale Glaser wrote

Thank you Bruce....yes, that is exactly what I got with Stata when I initially ran the power analysis  two years ago (i.e., n = 596 per group).  However, the discussion I've been having is when one runs a z-test for two proportions (whether using the SPSS macro or Stata option) n = 300 per group will suffice to obtain significance for 4.5% vs. 8.5% (z = 1.99, p = .047).

I do understand that with power, in part being based on the noncentral distribution, in conjunction with the rigors of the desired power (e.g., .8) may make sample size estimates larger than what one needs to obtain significance via simulation (as I did with the SPSS macro for z test of proportions). However, it seems that there is a nontrivial difference between n = 300 per group sufficing to have p < .05 (per the z test) as opposed to n = 596 per group as per the power analysis (with desired power of .8, two tailed test, and alpha = .05)

So though I understand the statistical difference between conducting power and running the actual z-test, I'm having difficulties reconciling the large sample size differences when discussing this with the PI (and making the ultimate recommendation).  Any feedback how you all broach the subject re: difference of sample size estimate via power analysis as opposed to simulation obtaining the actual test statistic (and p-value) would be much appreciated.

Thank you....Dale

 
Dale Glaser, Ph.D.
Principal--Glaser Consulting
Lecturer/Adjunct Faculty--SDSU/USD/Alliant
3115 4th Avenue
San Diego, CA 92103
phone: [phone]
fax: [phone]
email: [hidden email]
website: www.glaserconsult.com


________________________________
 From: Bruce Weaver <[hidden email]>
To: [hidden email] 
Sent: Thursday, July 24, 2014 6:38 PM
Subject: Re: Power, estimated sample size and conundrum!
 

Speaking of Stata, here's what I get for your original question (i.e., sample
size for proportions of .085 & .045):


. power twoproportions .085 .045

Performing iteration ...

Estimated sample sizes for a two-sample proportions test
Pearson's chi-squared test
Ho: p2 = p1  versus  Ha: p2 != p1

Study parameters:

        alpha =    0.0500
        power =    0.8000
        delta =   -0.0400  (difference)
           p1 =    0.0850
           p2 =    0.0450

Estimated sample sizes:

            N =      1192
  N per group =       596


I found this site helpful in working out how to do that:

  http://www.stata-press.com/manuals/power-sample-size-reference-manual/

HTH.




Dale Glaser wrote

> Greetings all....I have a
> question/situation that on the surface seems very transparent but I am
> having
> difficulties navigating.  So any feedback will be much appreciated.
>
>
> For a study two years
> ago I conducted a power analysis comparing two proportions (p1 = .085 vs.
> p2 =
> .045).  Whether I used G*Power, Stata, or Power and Precision they all
> gave me
> sample size estimates ranging from 600 to 640 (contingent if continuity
> correction was incorporated) per group for alpha = .05 and power of .80.
>
>
>
> However, when I ran the
> SPSS macro for z test of proportions for two groups comparing .045 vs.
> .085, I found that n = 300 per
> group was sufficient to obtain significance: z = -1.99, p = .047. I also
> confirmed this in Stata using the following syntax: prtesti 300 .045 300
> .085.
>
> Hence, can I assume that
> the difference (i.e., n = 600 per group per power analysis vs. n = 300 per
> group being sufficient to obtain significance) is a function of the
> desired
> power insofar with the conventional power of .80 you are setting the bar
> high
> enough so as to find the optimal nexus of Type I and Type II error?
>
> The conundrum is such
> that our study ended up with p1 = .027  vs. p2 =.047 and p = .194 with the
> recommended sample
> size being n = 300 per group given the simulation I ran in SPSS and
> Stata.  Note that post hoc power analysis indicates I
> would have needed n = 1496 per group (alpha =.05, power = .80) to obtain
> significance
> for delta of 2%, though when I run .027 vs. .047 in Stata and SPSS with n
> = 685
> per group z = 1.96, p = .05. 
>  
> Anyway, this has become
> an interesting/challenging discussion with PI and reviewers alike.  We
> went with the sample size (n = 300 per
> group) since that was sufficient for obtaining significance, but they are
> indicting we
> should have gone with the larger sample size based on the power estimate
> (i.e.,
> n = 600 per group).
>
> Has anyone encountered
> such a dilemma and how did you deal with it?
>
> Thank you….Dale
>  
>  
>  
>  
>  
>  
> Dale Glaser, Ph.D.
> Principal--Glaser Consulting
> Lecturer/Adjunct Faculty--SDSU/USD/Alliant
> 3115 4th Avenue
> San Diego, CA 92103
> phone: [phone]
> fax: [phone]
> email:

> glaserconsult@

> website: www.glaserconsult.com
>
>
> ________________________________
>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Power-estimated-sample-size-and-conundrum-tp5726815p5726821.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[peter link]

Dale – the simple answer is look up the Power/Sample size formula for the difference of 2 proportions and the z-test for the difference of 2 proportions.  You will notice that the “z”-term in the Power/Sample size formula is of the form z(beta)+z(alpha/2), while in the z-test it is of the form z(alpha/2).

 

Hth.

 

pl

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Dale Glaser
Sent: Friday, July 25, 2014 9:08 AM
To: [hidden email]
Subject: Re: [SPSSX-L] Power, estimated sample size and conundrum!

 

Thank you Bruce....yes, that is exactly what I got with Stata when I initially ran the power analysis  two years ago (i.e., n = 596 per group).  However, the discussion I've been having is when one runs a z-test for two proportions (whether using the SPSS macro or Stata option) n = 300 per group will suffice to obtain significance for 4.5% vs. 8.5% (z = 1.99, p = .047).

 

I do understand that with power, in part being based on the noncentral distribution, in conjunction with the rigors of the desired power (e.g., .8) may make sample size estimates larger than what one needs to obtain significance via simulation (as I did with the SPSS macro for z test of proportions). However, it seems that there is a nontrivial difference between n = 300 per group sufficing to have p < .05 (per the z test) as opposed to n = 596 per group as per the power analysis (with desired power of .8, two tailed test, and alpha = .05)

 

So though I understand the statistical difference between conducting power and running the actual z-test, I'm having difficulties reconciling the large sample size differences when discussing this with the PI (and making the ultimate recommendation).  Any feedback how you all broach the subject re: difference of sample size estimate via power analysis as opposed to simulation obtaining the actual test statistic (and p-value) would be much appreciated.

 

Thank you....Dale

 

Dale Glaser, Ph.D.
Principal--Glaser Consulting
Lecturer/Adjunct Faculty--SDSU/USD/Alliant
3115 4th Avenue
San Diego, CA 92103
phone: [phone]
fax: [phone]
email: [hidden email]
website: www.glaserconsult.com

 

---

From: Bruce Weaver <[hidden email]>
To: [hidden email]
Sent: Thursday, July 24, 2014 6:38 PM
Subject: Re: Power, estimated sample size and conundrum!

Speaking of Stata, here's what I get for your original question (i.e., sample
size for proportions of .085 & .045):


. power twoproportions .085 .045

Performing iteration ...

Estimated sample sizes for a two-sample proportions test
Pearson's chi-squared test
Ho: p2 = p1  versus  Ha: p2 != p1

Study parameters:

        alpha =    0.0500
        power =    0.8000
        delta =  -0.0400  (difference)
          p1 =    0.0850
          p2 =    0.0450

Estimated sample sizes:

            N =      1192
  N per group =      596


I found this site helpful in working out how to do that:

  http://www.stata-press.com/manuals/power-sample-size-reference-manual/

HTH.




Dale Glaser wrote

> Greetings all....I have a
> question/situation that on the surface seems very transparent but I am
> having
> difficulties navigating.  So any feedback will be much appreciated.
>
>
> For a study two years
> ago I conducted a power analysis comparing two proportions (p1 = .085 vs.
> p2 =
> .045).  Whether I used G*Power, Stata, or Power and Precision they all
> gave me
> sample size estimates ranging from 600 to 640 (contingent if continuity
> correction was incorporated) per group for alpha = .05 and power of .80.
>
>
>
> However, when I ran the
> SPSS macro for z test of proportions for two groups comparing .045 vs.
> .085, I found that n = 300 per
> group was sufficient to obtain significance: z = -1.99, p = .047. I also
> confirmed this in Stata using the following syntax: prtesti 300 .045 300
> .085.
>
> Hence, can I assume that
> the difference (i.e., n = 600 per group per power analysis vs. n = 300 per
> group being sufficient to obtain significance) is a function of the
> desired
> power insofar with the conventional power of .80 you are setting the bar
> high
> enough so as to find the optimal nexus of Type I and Type II error?
>
> The conundrum is such
> that our study ended up with p1 = .027  vs. p2 =.047 and p = .194 with the
> recommended sample
> size being n = 300 per group given the simulation I ran in SPSS and
> Stata.  Note that post hoc power analysis indicates I
> would have needed n = 1496 per group (alpha =.05, power = .80) to obtain
> significance
> for delta of 2%, though when I run .027 vs. .047 in Stata and SPSS with n
> = 685
> per group z = 1.96, p = .05. 
>  
> Anyway, this has become
> an interesting/challenging discussion with PI and reviewers alike.  We
> went with the sample size (n = 300 per
> group) since that was sufficient for obtaining significance, but they are
> indicting we
> should have gone with the larger sample size based on the power estimate
> (i.e.,
> n = 600 per group).
>
> Has anyone encountered
> such a dilemma and how did you deal with it?
>
> Thank you….Dale
>  
>  
>  
>  
>  
>  
> Dale Glaser, Ph.D.
> Principal--Glaser Consulting
> Lecturer/Adjunct Faculty--SDSU/USD/Alliant
> 3115 4th Avenue
> San Diego, CA 92103
> phone: [phone]
> fax: [phone]
> email:

> glaserconsult@

> website: www.glaserconsult.com

>
>
> ________________________________
>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> [hidden email]

>  (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Power-estimated-sample-size-and-conundrum-tp5726815p5726821.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD