Re: Bootstrap sampling for evaluating hypothesis tests

Posted by Rich Ulrich on Mar 14, 2013; 8:22pm
URL: http://spssx-discussion.165.s1.nabble.com/Bootstrap-sampling-for-evaluating-hypothesis-tests-tp5718617p5718672.html

Gene,
I think you are right is saying that it is a Monte Carlo problem.

But she is assessing the robustness for small samples, not looking
for the power.  That is:  She needs to show that the test does not
reject too many times (or, less problematically, too few) for small
samples.  That is the test under the null.

Of course, if the test itself is a version of bootstrapping, then
there's still a question of bootstrapping.

--
Rich Ulrich

---

Date: Thu, 14 Mar 2013 15:46:05 -0400
From: [hidden email]
Subject: Re: Bootstrap sampling for evaluating hypothesis tests
To: [hidden email]

Matt and others are more knowledgeable about this question, I’d guess, but this seems to me to be a MonteCarlo problem rather than a bootstrap problem. It sounds like you might fix the numerator of effect size (I’m presuming that the ‘betterness’ of your statistic is related to the standard error calculation) and vary the sample size and the hypothesis is that the test statistic is better than the ‘standard’ statistic. So you need adequate number of replications at each sample size point to assess power.

 

Gene Maguin

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Margaret MacDougall
Sent: Thursday, March 14, 2013 3:18 PM
To: [hidden email]
Subject: Re: Bootstrap sampling for evaluating hypothesis tests

 

Dear Matt

Thanks for your message. My interest in bootsrapping stems from the fact that I have developed a hyothesis test which can be justified on theoretical grounds for sufficiently large sample size and I would like to see how well it behaves for a range of smaller sample sizes. I expect that this would entail generating multiple samples for which the null hypothesis is actually true and using these as a basis for assessing the probability of a Type I error.

I have not used SPSS previously for bootstrapping but I am aware that the standard package offers some level of functionality in this area, via the existing menu commands and that there is a bootstrapping add-on for SPSS. I am unclear how powerful the add-on is in erms of functionality or whether SPSS syntax could cover my needs.

Best wishes

Margaret

 ... [snip, previous]