bootstrap CI for dep t confusing
Locked
 
|
Hi All: I just ran a dependent t-test with bootstrapping. The non bootstrap result provides a t test value of 2.971 (10df) with a two tailed p value of .014. The bootstrap results from 1000 samples provides a two-tailed p value of .072.
So I understand that the bootstrapping might be in disagreement with the non bootstrap result. OK. However, when I look at the bootstrap 95% CI I find the interval is from 0.3636 to 1.9090. Give the bootstrap p value of non-significance I was expected the
95% CI to include 0. But it doesn’t. Any ideas why I am getting such a result. Thanks, martin Martin F. Sherman, Ph.D. Professor of Psychology Director of Masters Education: Thesis Track Loyola University Maryland 4501 North Charles Street 222 B Beatty Hall Baltimore, MD 21210 410 617-2417 |
|
|
Martin,
It's probably your small N. Since bootstrapping samples with replacement, it's very likely that you're getting multiple skewed samples, increasing the variance and therefore the CI. Brian ________________________________________ From: SPSSX(r) Discussion [[hidden email]] on behalf of Martin Sherman [[hidden email]] Sent: Monday, November 20, 2017 3:31 PM To: [hidden email] Subject: bootstrap CI for dep t confusing Hi All: I just ran a dependent t-test with bootstrapping. The non bootstrap result provides a t test value of 2.971 (10df) with a two tailed p value of .014. The bootstrap results from 1000 samples provides a two-tailed p value of .072. So I understand that the bootstrapping might be in disagreement with the non bootstrap result. OK. However, when I look at the bootstrap 95% CI I find the interval is from 0.3636 to 1.9090. Give the bootstrap p value of non-significance I was expected the 95% CI to include 0. But it doesn’t. Any ideas why I am getting such a result. Thanks, martin Martin F. Sherman, Ph.D. Professor of Psychology Director of Masters Education: Thesis Track Loyola University Maryland 4501 North Charles Street 222 B Beatty Hall Baltimore, MD 21210 410 617-2417 [hidden email] ===================== To manage your subscription to SPSSX-L, send a message to [hidden email]<mailto:[hidden email]> (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
|
I may be confused but the problem is not that the variance has been increased but that the CI is not wide enough to include 0.00. If the variances have been increased,then the standard error of the mean difference should have also been increased, right? The critical t-value for df=10 remains the same, so an increased standard error would produce a wider CI if the formula used is Mean difference +/- (crit t(10))*SE of Mean Difference Or am I missing something? -Mike Palij New York University On Mon, Nov 20, 2017 at 3:49 PM, Dates, Brian <[hidden email]> wrote: Martin, |
Re: bootstrap CI for dep t confusing
Administrator
|
In reply to this post by msherman
I don't think it is uncommon for bootstrapped CIs and p-values to "disagree".
E.g., https://stats.stackexchange.com/questions/302928/disagreement-between-p-values-and-confidence-intervals https://stats.stackexchange.com/questions/29546/p-value-vs-confidence-interval-obtained-in-bootstrapping HTH. msherman wrote > Hi All: I just ran a dependent t-test with bootstrapping. The non > bootstrap result provides a t test value of 2.971 (10df) with a two tailed > p value of .014. The bootstrap results from 1000 samples provides a > two-tailed p value of .072. So I understand that the bootstrapping might > be in disagreement with the non bootstrap result. OK. However, when I look > at the bootstrap 95% CI I find the interval is from 0.3636 to 1.9090. Give > the bootstrap p value of non-significance I was expected the 95% CI to > include 0. But it doesn't. Any ideas why I am getting such a result. > Thanks, martin > > > > > Martin F. Sherman, Ph.D. > Professor of Psychology > Director of Masters Education: Thesis Track > Loyola University Maryland > 4501 North Charles Street > 222 B Beatty Hall > Baltimore, MD 21210 > > 410 617-2417 > msherman@ > > > ===================== > To manage your subscription to SPSSX-L, send a message to > LISTSERV@.UGA > (not to SPSSX-L), with no body text except the > command. To leave the list, send the command > SIGNOFF SPSSX-L > For a list of commands to manage subscriptions, send the command > INFO REFCARD ----- -- Bruce Weaver [hidden email] http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." NOTE: My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. -- Sent from: http://spssx-discussion.1045642.n5.nabble.com/ ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
--
Bruce Weaver bweaver@lakeheadu.ca http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." PLEASE NOTE THE FOLLOWING: 1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. 2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/). |
|
|
Hmmm, after doing some review of literature, a couple of questions arise (1) The bootstrap CI is supposed to be provide a better estimate of theDiCiccio, T. J., & Efron, B. (1996). Bootstrap confidence intervals. Statistical science, 189-212. This article is available on the Project Euclid website: click on: It would probably help if the OP identified which bootstrap CI method was used. The methods are listed in the SPSS Algorithms manual. -Mike Palij New York University On Mon, Nov 20, 2017 at 9:38 PM, Bruce Weaver <[hidden email]> wrote: I don't think it is uncommon for bootstrapped CIs and p-values to "disagree". |
|
|
Based on the nice DiCiccio and Efron article, it seems that bootstrapping using method BCa, which is available but not the default in Statistics, would be superior for CI purposes. On Tue, Nov 21, 2017 at 4:31 AM, Michael Palij <[hidden email]> wrote:
|
|
|
I have a paper with a complete list of resampling methods, including explanations, with formulae of 7 types of bootstrapping and the approximate bootstrap method, which is really a Bayesian approach to an actual bootstrap. It was the basis for Chapter 16 in The SAGE Handbook of Quantitative Methods in Psychology edited by Millsap and Maydeau-Olivares. It's very readable and a good reference for any resampling method, permutation, jackknife, and bootstrap. If you'd like a copy please indicate that. I believe that I can upload a file to Nabble if requested.
Brian ________________________________________ From: SPSSX(r) Discussion [[hidden email]] on behalf of Jon Peck [[hidden email]] Sent: Tuesday, November 21, 2017 11:11 AM To: [hidden email] Subject: Re: bootstrap CI for dep t confusing Based on the nice DiCiccio and Efron article, it seems that bootstrapping using method BCa, which is available but not the default in Statistics, would be superior for CI purposes. On Tue, Nov 21, 2017 at 4:31 AM, Michael Palij <[hidden email]<mailto:[hidden email]>> wrote: Hmmm, after doing some review of literature, a couple of questions arise (1) The bootstrap CI is supposed to be provide a better estimate of the CI (if one believe such things) than the normal theory CI one usually calculates either with Z or t. Situations where it appears that bootstrap CI does worse or a situation as described by the OP, indicates some pathological condition. (2) One has to identify which bootstrap CI is being used because I reckon that there are probably more than a half dozen versions. The following article by DiCiccio and Efron provides more information on both of these points but there are recent articles that provide additional perspectives: DiCiccio, T. J., & Efron, B. (1996). Bootstrap confidence intervals. Statistical science, 189-212. This article is available on the Project Euclid website: click on: http://projecteuclid.org/download/pdf_1/euclid.ss/1032280214 It would probably help if the OP identified which bootstrap CI method was used. The methods are listed in the SPSS Algorithms manual. -Mike Palij New York University [hidden email]<mailto:[hidden email]> On Mon, Nov 20, 2017 at 9:38 PM, Bruce Weaver <[hidden email]<mailto:[hidden email]>> wrote: I don't think it is uncommon for bootstrapped CIs and p-values to "disagree". E.g., https://stats.stackexchange.com/questions/302928/disagreement-between-p-values-and-confidence-intervals https://stats.stackexchange.com/questions/29546/p-value-vs-confidence-interval-obtained-in-bootstrapping HTH. msherman wrote > Hi All: I just ran a dependent t-test with bootstrapping. The non > bootstrap result provides a t test value of 2.971 (10df) with a two tailed > p value of .014. The bootstrap results from 1000 samples provides a > two-tailed p value of .072. So I understand that the bootstrapping might > be in disagreement with the non bootstrap result. OK. However, when I look > at the bootstrap 95% CI I find the interval is from 0.3636 to 1.9090. Give > the bootstrap p value of non-significance I was expected the 95% CI to > include 0. But it doesn't. Any ideas why I am getting such a result. > Thanks, martin > > > > > Martin F. Sherman, Ph.D. > Professor of Psychology > Director of Masters Education: Thesis Track > Loyola University Maryland > 4501 North Charles Street<https://maps.google.com/?q=4501+North+Charles+Street&entry=gmail&source=g> > 222 B Beatty Hall > Baltimore, MD 21210 > > 410 617-2417<tel:(410)%20617-2417> > msherman@ > > > ===================== > To manage your subscription to SPSSX-L, send a message to > LISTSERV@.UGA > (not to SPSSX-L), with no body text except the > command. To leave the list, send the command > SIGNOFF SPSSX-L > For a list of commands to manage subscriptions, send the command > INFO REFCARD ----- -- Bruce Weaver [hidden email]<mailto:[hidden email]> http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." NOTE: My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. -- Sent from: http://spssx-discussion.1045642.n5.nabble.com/ ===================== To manage your subscription to SPSSX-L, send a message to [hidden email]<mailto:[hidden email]> (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email]<mailto:[hidden email]> (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD -- Jon K Peck [hidden email]<mailto:[hidden email]> ===================== To manage your subscription to SPSSX-L, send a message to [hidden email]<mailto:[hidden email]> (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
«
Return to SPSSX Discussion
|
1 view|%1 views
| Free forum by Nabble | Edit this page |