Re: Bootstrap sampling for evaluating hypothesis tests
Posted by
Margaret MacDougall on
Mar 16, 2013; 8:52am
URL: http://spssx-discussion.165.s1.nabble.com/Bootstrap-sampling-for-evaluating-hypothesis-tests-tp5718617p5718745.html
Dear Ulrich
Thanks for chipping in. I understand that a Monte Carlo approach would be of interest in producing a confidence interval for the proportion of replications in which the test statistic is more extreme than that of the original data. However, in terms of my original query re use of bootstrapping, I have also discovered very recently that Stata covers my needs extrenely well., so much so that I think that this is going to be my choice of package for the tasks in hand.
Best wishes
Margaret
From: Rich Ulrich <[hidden email]>
To: [hidden email]
Sent: Thursday, 14 March 2013, 20:22
Subject: Re: Bootstrap sampling for evaluating hypothesis tests
Gene,
I think you are right is saying that it is a Monte Carlo problem.
But she is assessing the robustness for small samples, not looking
for the power. That is: She needs to show that the test does not
reject too many times (or, less problematically, too few) for small
samples. That is the test under the null.
Of course, if the test itself is a version of bootstrapping, then
there's still a question of bootstrapping.
--
Rich Ulrich
Date: Thu, 14 Mar 2013 15:46:05 -0400
From:
[hidden email]Subject: Re: Bootstrap sampling for evaluating hypothesis tests
To:
[hidden email]Matt and others are more knowledgeable about this question, I’d guess, but this seems to me to be a MonteCarlo problem rather than a bootstrap problem. It sounds like you might fix the numerator of effect size (I’m presuming that the ‘betterness’ of your statistic is related to the standard error calculation) and vary the sample size and the hypothesis is that the test statistic is better than the ‘standard’ statistic. So you need adequate number of replications at each sample size point to assess power.
Gene Maguin
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Margaret MacDougall
Sent: Thursday, March 14, 2013 3:18 PM
To: [hidden email]
Subject: Re: Bootstrap sampling for evaluating hypothesis tests
Thanks for your message. My interest in bootsrapping
stems from the fact that I have developed a hyothesis test which can be justified on theoretical grounds for sufficiently large sample size and I would like to see how well it behaves for a range of smaller sample sizes. I expect that this would entail generating multiple samples for which the null hypothesis is actually true and using these as a basis for assessing the probability of a Type I error.
I have not used SPSS previously for bootstrapping but I am aware that the standard package offers some level of functionality in this area, via the existing menu commands and that there is a bootstrapping add-on for SPSS. I am unclear how powerful the add-on is in erms of functionality or whether SPSS syntax could cover my needs.
Margaret
... [snip, previous]