Dear List: I just used Hayes’ Process macro within spss 23.0 to test for an indirect mediational effect and have a result which is a little confusing. The print out for
the indirect effect shows that the 95% CI for the indirect coefficient is -0.00097 (lower Limit) and 0.25978 (Upper limit) with a Z value of 2.09 and a p value of .03641. Since the 95% CI includes zero the results are not statistically significant yet the
p value is statistically significant. Is this inconsistency due the p value representing only one tail whereas the 95% CI utilizes two tails (which would relate to .03641 X 2 = .07282 and thus would include zero contained within the 95% CI)? I looked at another
result where the Z value was 0.95 and it had a p value of .34338 which, to me, suggests that this value represents .17169 in each tail or a total p value of .34338. If this is correct then the p value for the Z of 2.09 actually is .018205 in one tail and .03641
in two tails (.018205 X 2 = .03641). If so then the 95% CI should not contain the value of zero. What am I missing?
Martin F. Sherman, Ph.D. Professor of Psychology Director of Master’s Education: Thesis Track Department of Psychology 222 B Beatty Hall 4501 North Charles Street Baltimore, MD 21210 410-617-2417 tel 410-617-5341 fax |
Administrator
|
Disclaimer: I have never used the PROCESS macro. However, I have heard people talking about it, and based on what I've heard, I wonder if the CIs are bootstrap CIs. If they are, and if the test is not obtained via bootstrapping, then one could easily get the kind of result you describe.
HTH.
--
Bruce Weaver bweaver@lakeheadu.ca http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." PLEASE NOTE THE FOLLOWING: 1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. 2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/). |
To confirm what Bruce wrote - The Indirect Effect is the multiplication of 2
point estimates, so the variance (and CI) are bootstrapped. peter -----Original Message----- From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Bruce Weaver Sent: Thursday, June 09, 2016 10:40 AM To: [hidden email] Subject: Re: [SPSSX-L] One tail vs two tail results using Hayes Process Mediational analysis Disclaimer: I have never used the PROCESS macro. However, I have heard people talking about it, and based on what I've heard, I wonder if the CIs are bootstrap CIs. If they are, and if the test is not obtained via bootstrapping, then one could easily get the kind of result you describe. HTH. msherman wrote > Dear List: I just used Hayes' Process macro within spss 23.0 to test > for an indirect mediational effect and have a result which is a little > confusing. The print out for the indirect effect shows that the 95% CI > for the indirect coefficient is -0.00097 (lower Limit) and 0.25978 > (Upper > limit) with a Z value of 2.09 and a p value of .03641. Since the 95% > CI includes zero the results are not statistically significant yet the > p value is statistically significant. Is this inconsistency due the p > value representing only one tail whereas the 95% CI utilizes two tails > (which would relate to .03641 X 2 = .07282 and thus would include zero > contained within the 95% CI)? I looked at another result where the Z > value was 0.95 and it had a p value of .34338 which, to me, suggests > that this value represents .17169 in each tail or a total p value of > .34338. If this is correct then the p value for the Z of 2.09 actually > is .018205 in one tail and .03641 in two tails (.018205 X 2 = > .03641). If so then the 95% CI should not contain the value of zero. What > > Martin F. Sherman, Ph.D. > Professor of Psychology > Director of Master's Education: Thesis Track > > Department of Psychology > 222 B Beatty Hall > 4501 North Charles Street > Baltimore, MD 21210 > msherman@ > <mailto: > msherman@ > > > 410-617-2417 tel > 410-617-5341 fax > > > ===================== > To manage your subscription to SPSSX-L, send a message to > LISTSERV@.UGA > (not to SPSSX-L), with no body text except the command. To leave the > list, send the command SIGNOFF SPSSX-L For a list of commands to > manage subscriptions, send the command INFO REFCARD ----- -- Bruce Weaver [hidden email] http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." NOTE: My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/One-tail-vs-two-tail-results-u sing-Hayes-Process-Mediational-analysis-tp5732365p5732368.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Ah, that makes sense. In fact I have one result where the test statistic is not significant but the 95% CI does not contain ZERO. The 95% CI uses the Boot SE. Thus, one should go by the boot strapped 95% CI for determination of a statistically significant result and not the test statistic which I believe is using a regular SE. Is this correct. martin
-----Original Message----- From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Peter Link Sent: Thursday, June 09, 2016 2:09 PM To: [hidden email] Subject: Re: One tail vs two tail results using Hayes Process Mediational analysis To confirm what Bruce wrote - The Indirect Effect is the multiplication of 2 point estimates, so the variance (and CI) are bootstrapped. peter -----Original Message----- From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Bruce Weaver Sent: Thursday, June 09, 2016 10:40 AM To: [hidden email] Subject: Re: [SPSSX-L] One tail vs two tail results using Hayes Process Mediational analysis Disclaimer: I have never used the PROCESS macro. However, I have heard people talking about it, and based on what I've heard, I wonder if the CIs are bootstrap CIs. If they are, and if the test is not obtained via bootstrapping, then one could easily get the kind of result you describe. HTH. msherman wrote > Dear List: I just used Hayes' Process macro within spss 23.0 to test > for an indirect mediational effect and have a result which is a little > confusing. The print out for the indirect effect shows that the 95% CI > for the indirect coefficient is -0.00097 (lower Limit) and 0.25978 > (Upper > limit) with a Z value of 2.09 and a p value of .03641. Since the 95% > CI includes zero the results are not statistically significant yet the > p value is statistically significant. Is this inconsistency due the p > value representing only one tail whereas the 95% CI utilizes two tails > (which would relate to .03641 X 2 = .07282 and thus would include zero > contained within the 95% CI)? I looked at another result where the Z > value was 0.95 and it had a p value of .34338 which, to me, suggests > that this value represents .17169 in each tail or a total p value of > .34338. If this is correct then the p value for the Z of 2.09 actually > is .018205 in one tail and .03641 in two tails (.018205 X 2 = > .03641). If so then the 95% CI should not contain the value of zero. > What > > Martin F. Sherman, Ph.D. > Professor of Psychology > Director of Master's Education: Thesis Track > > Department of Psychology > 222 B Beatty Hall > 4501 North Charles Street > Baltimore, MD 21210 > msherman@ > <mailto: > msherman@ > > > 410-617-2417 tel > 410-617-5341 fax > > > ===================== > To manage your subscription to SPSSX-L, send a message to > LISTSERV@.UGA > (not to SPSSX-L), with no body text except the command. To leave the > list, send the command SIGNOFF SPSSX-L For a list of commands to > manage subscriptions, send the command INFO REFCARD ----- -- Bruce Weaver [hidden email] http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." NOTE: My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/One-tail-vs-two-tail-results-u sing-Hayes-Process-Mediational-analysis-tp5732365p5732368.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Obtaining a one-tailed p-value is not so simple if the bootstrap CI is bias-corrected, many of which are. Ryan On Thu, Jun 9, 2016 at 2:44 PM, Martin Sherman <[hidden email]> wrote: Ah, that makes sense. In fact I have one result where the test statistic is not significant but the 95% CI does not contain ZERO. The 95% CI uses the Boot SE. Thus, one should go by the boot strapped 95% CI for determination of a statistically significant result and not the test statistic which I believe is using a regular SE. Is this correct. martin |
And the Hayes’ Process module gives you a choice between using the bias corrected CI or the “Percentile” bootstrap method. I believe Hayes recommends the bias-corrected
approach. From: SPSSX(r) Discussion [mailto:[hidden email]]
On Behalf Of Ryan Black Obtaining a one-tailed p-value is not so simple if the bootstrap CI is bias-corrected, many of which are.
Ryan On Thu, Jun 9, 2016 at 2:44 PM, Martin Sherman <[hidden email]> wrote:
===================== To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
|
If there is a Wald-Z test in the output - that would use the variance from the bootstrapped distribution of the indirect effect. But, the CI wouldn’t be constructed this way. That would use the percentile, bias-corrected, etc. bootstrapped CI. These would be 2 different methods for testing the estimate of the indirect effect. peter From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Martin Sherman And the Hayes’ Process module gives you a choice between using the bias corrected CI or the “Percentile” bootstrap method. I believe Hayes recommends the bias-corrected approach. From: SPSSX(r) Discussion [[hidden email]] On Behalf Of Ryan Black Obtaining a one-tailed p-value is not so simple if the bootstrap CI is bias-corrected, many of which are. Ryan On Thu, Jun 9, 2016 at 2:44 PM, Martin Sherman <[hidden email]> wrote:
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Peter: With a little additional research I determined that the Normal Test Theory results are similar to Normal Theory-based Sobel test which is predicated upon
the conditional indirect effects being a product of normally distributed regression coefficients-which according the Hayes is untenable in most cases and results in lost power. Hayes thus recommends the use of Bootstrap Confidence Intervals. The boot strap
CIs are based upon empirically generated representation of the sampling distribution of indirect effect (Hayes, 2013). martin From: Peter Link [mailto:[hidden email]]
Martin – My best guess is that is the Sobel Test. Generally, the product of the indirect effect is not normally distributed. I think it has been shown to have
worse coverage than bootstrapped CI. Often these are given just so you have a p-value. Maybe some journals still require p-values with all tests of parameter estimates. peter From: Martin Sherman [[hidden email]]
It is Normal Theory Test value reported as a Z value in the Process output. martin From: Peter Link [[hidden email]]
So this is a bit tricky. The Z-test gives a p-value and the CI doesn’t. I think if you had to choose 1, most prefer to present the CI in these situations. I see
CI presented a lot more than Z-tests. Also, note: If this is the old Sobel test, then definitely use the CI . peter From: Martin Sherman [[hidden email]]
Peter: so if one wanted to determine if there was evidence of an indirect effect one would use the 95% CI and not the statistical test? If this correct? thanks,
martin From: SPSSX(r) Discussion [[hidden email]]
On Behalf Of Peter Link If there is a Wald-Z test in the output - that would use the variance from the bootstrapped distribution of the indirect effect. But, the CI wouldn’t be constructed
this way. That would use the percentile, bias-corrected, etc. bootstrapped CI. These would be 2 different methods for testing the estimate of the indirect effect. peter From: SPSSX(r) Discussion [[hidden email]]
On Behalf Of Martin Sherman And the Hayes’ Process module gives you a choice between using the bias corrected CI or the “Percentile” bootstrap method. I believe Hayes recommends the bias-corrected
approach. From: SPSSX(r) Discussion [[hidden email]]
On Behalf Of Ryan Black Obtaining a one-tailed p-value is not so simple if the bootstrap CI is bias-corrected, many of which are.
Ryan On Thu, Jun 9, 2016 at 2:44 PM, Martin Sherman <[hidden email]> wrote:
===================== To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Free forum by Nabble | Edit this page |