Rounding algorithm
I conducted a 1-way ANOVA on a small data set and SPSS (v20 for Mac) gave me a SSgroups of 413.500*. However, when I calculate it by hand, and never round any steps using a calculator with 16-digits, I get a SSgroups of 413.495xxx (the x’s are additional digits). My assumption has always been that SPSS gives all responses to 3 decimal places. Is it common for it to round to the nearest tenth, dropping the rest? I know the difference between the two is quite small, but I have never come across this before and was wondering if anyone had an explanation.
*If I double-click on the summary table to look beyond the 3 decimal places displayed, the result is 413.50000).
--
Fredric E. Rose, Ph.D.
Associate Professor of Psychology
Palomar College
760-744-1150 x2344
[hidden email]
====================To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD