Bad advice on SPSS wiki

[Allan Reese (Cefas)]

Happened on the SPSS wiki (http://spss.wikia.com/wiki/SPSS_Wiki) which
says it's inactive.  I'm not inclined to attempt to revive it given this
weirdly misguided understanding of hypothesis testing.

 

"Highly significant results: p<0.001 is better than p<0.05 right? What
you're really saying is that the probability of thinking something is
happening when it is not (type I error) is less than 1 chance in 1000 as
opposed to 1 chance in 20. Still sound good? Here's the problem - most
sample sizes are far less than 1000, or even 100 - so how can you
logically be claiming the probability of a type I error is less than 1
chance in 100, or 1000, or whatever, when your sample was only (say) 52
participants?"

The wiki article, titled "Common misunderstandings" but I hope not,
continues

"There is also the problem that by artificially reducing the chance of a
type I error (i.e. using 0.01 or 0.001 cut-offs) you are likely to think
nothing is happening when it may well be (a type II error). At any rate,
significant doesn't mean big, it means consistent. The size of the
effect might be very small even when it is consistent. If you want to
talk about effect size, that's a different matter. One last downside -
using extreme significance cut-offs almost always reduces the
probability that you will correctly detect an effect (power) to
unacceptable levels. Take our advice, stick with reporting the p<0.05
criteria regardless of what SPSS spits out - unless you really know what
you're doing."

 

For another view, see Nature 455, 1023-1028 6/11/08 and response in
Nature 456, 315 20/11/08.

 

R Allan Reese
Senior statistician, Cefas
The Nothe, Weymouth DT4 8UB

Tel: [phone] -direct
Fax: [phone]

www.cefas.co.uk
<file:///C:\Documents%20and%20Settings\rar00\Application%20Data\Microsof
t\Signatures\www.cefas.co.uk>  

 



***********************************************************************************
This email and any attachments are intended for the named recipient only.  Its unauthorised use, distribution, disclosure, storage or copying is not permitted.  If you have received it in error, please destroy all copies and notify the sender.  In messages of a non-business nature, the views and opinions expressed are the author's own and do not necessarily reflect those of the organisation from which it is sent.  All emails may be subject to monitoring.
***********************************************************************************

====================To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD