One-Sample Test Options

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

One-Sample Test Options

Eric Graig-2
We have a survey designed to ascertain whether respondents feel worse off, about the same or better off following an intervention in thirty different areas of functioning (after the intervention, do you feel much worse off, about the same, or much better off with respect to a, b, c....). The survey utilized a nine point anchored scale that looked something like this:

Area A Much Worse Off  o o o o  About the Same o o o o Much Better Off
Area B Much Worse Off  o o o o  About the Same o o o o Much Better Off
Area C Much Worse Off  o o o o  About the Same o o o o Much Better Off

We are interested in whether people feel better off or not which appear as a response greater than 5, which is the midpoint of our scale. N was about 40 (it depends on the item, there is some missing data).

We let SPSS Non Parametric do out thinking for us and it returned most of the items as normally distributed (one sample Kolmogorov - Smirnov). We then ran:

NPTESTS
  /ONESAMPLE TEST (area of functioning) WILCOXON(TESTVALUE=5)
  /MISSING SCOPE=ANALYSIS USERMISSING=EXCLUDE
  /CRITERIA ALPHA=0.05 CILEVEL=95.

in order to test whether the sample tended to feel better than "About the Same" which, as above, is represented by a value of "5" in the data set.

I was very surprised to find that even with relatively small mean differences (they ranged from 0.67 to 1.85) all were statistically significant according the the test above.

Some Questions:

  1. Is the Wilcoxon the correct test to run? The only examples that I find of its use outside of this SPSS procedure is in a paired design.  Indeed, I haven't been able to find any references to a nonparametric analog to the one-sample t-test.
  2. Can the SPSS output for the Wilcoxon be used to calculate effect size?
  3. I ran the same test using a one sample t-test which also returned significance for all the areas of functioning and used that data to calculate effect size using R and also Cohen's D. All were close to or above 0.5. Can I legitimately use these measures of effect size despite the fact that some of my outcome measures are not normally distributed?

Thanks MUCH for any assistance. I'm in the unenviable position of knowing just enough statistics to be dangerous.

Eric

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: One-Sample Test Options

Maguin, Eugene

Maybe you did this and I didn’t understand your posting. Don’t you want to work with a dichotomous variable (same/worse versus better)? Null hypothesis would be 50/50.

 

Gene Maguin

 

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Eric
Sent: Monday, January 04, 2016 1:26 PM
To: [hidden email]
Subject: One-Sample Test Options

 

We have a survey designed to ascertain whether respondents feel worse off, about the same or better off following an intervention in thirty different areas of functioning (after the intervention, do you feel much worse off, about the same, or much better off with respect to a, b, c....). The survey utilized a nine point anchored scale that looked something like this:

Area A Much Worse Off  o o o o  About the Same o o o o Much Better Off
Area B Much Worse Off  o o o o  About the Same o o o o Much Better Off
Area C Much Worse Off  o o o o  About the Same o o o o Much Better Off

We are interested in whether people feel better off or not which appear as a response greater than 5, which is the midpoint of our scale. N was about 40 (it depends on the item, there is some missing data).

We let SPSS Non Parametric do out thinking for us and it returned most of the items as normally distributed (one sample Kolmogorov - Smirnov). We then ran:

NPTESTS
  /ONESAMPLE TEST (area of functioning) WILCOXON(TESTVALUE=5)
  /MISSING SCOPE=ANALYSIS USERMISSING=EXCLUDE
  /CRITERIA ALPHA=0.05 CILEVEL=95.

 

in order to test whether the sample tended to feel better than "About the Same" which, as above, is represented by a value of "5" in the data set.

I was very surprised to find that even with relatively small mean differences (they ranged from 0.67 to 1.85) all were statistically significant according the the test above.

Some Questions:

  1. Is the Wilcoxon the correct test to run? The only examples that I find of its use outside of this SPSS procedure is in a paired design.  Indeed, I haven't been able to find any references to a nonparametric analog to the one-sample t-test.
  2. Can the SPSS output for the Wilcoxon be used to calculate effect size?
  3. I ran the same test using a one sample t-test which also returned significance for all the areas of functioning and used that data to calculate effect size using R and also Cohen's D. All were close to or above 0.5. Can I legitimately use these measures of effect size despite the fact that some of my outcome measures are not normally distributed?

Thanks MUCH for any assistance. I'm in the unenviable position of knowing just enough statistics to be dangerous.

Eric

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: One-Sample Test Options

Mike
In reply to this post by Eric Graig-2

Hi,
 
Not really my area but your results actually sound consistent.  That
being said, you may want to do the following:
 
(1) Check out the following webpage on the IBM website:
It explains how to do the one-sample Wilcoxon test (but you already
know that) and that it is a special case of the Wilcoxon matched-pairs
test where the second value is the null hypothesized value (i.e., "5")
which you don't seem to appreciate.
 
(2) The IBM website above cites Section 5.1 in the following:
W. J. Conover's (1971) Practical Nonparametric Statistics (Wiley).
Conover's book is up to a 3rd edition, more info about that here:
You might want to check out either the earlier or the later edition (I'd
go for the later edition because new developments may have altered
some of the interpretation of the one-sample Wilcoxon text).  So,
I suggest that Ye Get Thee To A Library or a bookstore where you
read the book for free (I couldn't find an electronic copy online; you
might have better luck).
 
(3) A general Google search turns up about 3,800 hits for "one-sample
Wilcoxon test" and a search of scholar.google.com turns up about 483
hits; for the latter see:
So, I don't understand how/why you can't find references for the test.
If you have access to www.jstor.org, which has the Journal of the
American Statistical Association (JASA), the American Statistician,
and a shipload of other statistical-ish journal, you'll find article there
as well.  But you might want to start just by looking at the above
mentioned Conover book first.
 
(4) One point that you seem to miss is that you are focusing on the
difference between means (or, technically in the one sample Wilcoxon
test, medians) but forget that this difference has to be interpreted
relative to the amount of sampling error in your sample statistics.
Cohen's d (if you use sample means and standard deviations) is
the most representative difference to use because it reflects the
difference between means relative to a measure of their sampling
error.  So, don't just look at the simple difference between means.
 
I'm sure if I said anything crazy/absurd/wrong above, someone will
correct me.
 
-Mike Palij
New York University
 
 
----- Original Message -----
Sent: Monday, January 04, 2016 1:26 PM
Subject: One-Sample Test Options

We have a survey designed to ascertain whether respondents feel worse off, about the same or better off following an intervention in thirty different areas of functioning (after the intervention, do you feel much worse off, about the same, or much better off with respect to a, b, c....). The survey utilized a nine point anchored scale that looked something like this:

Area A Much Worse Off  o o o o  About the Same o o o o Much Better Off
Area B Much Worse Off  o o o o  About the Same o o o o Much Better Off
Area C Much Worse Off  o o o o  About the Same o o o o Much Better Off

We are interested in whether people feel better off or not which appear as a response greater than 5, which is the midpoint of our scale. N was about 40 (it depends on the item, there is some missing data).

We let SPSS Non Parametric do out thinking for us and it returned most of the items as normally distributed (one sample Kolmogorov - Smirnov). We then ran:

NPTESTS
  /ONESAMPLE TEST (area of functioning) WILCOXON(TESTVALUE=5)
  /MISSING SCOPE=ANALYSIS USERMISSING=EXCLUDE
  /CRITERIA ALPHA=0.05 CILEVEL=95.

in order to test whether the sample tended to feel better than "About the Same" which, as above, is represented by a value of "5" in the data set.

I was very surprised to find that even with relatively small mean differences (they ranged from 0.67 to 1.85) all were statistically significant according the the test above.

Some Questions:

  1. Is the Wilcoxon the correct test to run? The only examples that I find of its use outside of this SPSS procedure is in a paired design.  Indeed, I haven't been able to find any references to a nonparametric analog to the one-sample t-test.
  2. Can the SPSS output for the Wilcoxon be used to calculate effect size?
  3. I ran the same test using a one sample t-test which also returned significance for all the areas of functioning and used that data to calculate effect size using R and also Cohen's D. All were close to or above 0.5. Can I legitimately use these measures of effect size despite the fact that some of my outcome measures are not normally distributed?

Thanks MUCH for any assistance. I'm in the unenviable position of knowing just enough statistics to be dangerous.

Eric

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD