Rounding algorithm

Posted by Rose, Fred on Nov 07, 2012; 9:20pm
URL: http://spssx-discussion.165.s1.nabble.com/Rounding-algorithm-tp5716096.html

I conducted a 1-way ANOVA on a small data set and SPSS (v20 for Mac) gave me a SSgroups of 413.500*.  However, when I calculate it by hand, and never round any steps using a calculator with 16-digits, I get a SSgroups of 413.495xxx (the x’s are additional digits).  My assumption has always been that SPSS gives all responses to 3 decimal places.  Is it common for it to round to the nearest tenth, dropping the rest?  I know the difference between the two is quite small, but I have never come across this before and was wondering if anyone had an explanation.

*If I double-click on the summary table to look beyond the 3 decimal places displayed, the result is 413.50000).


--
Fredric E. Rose, Ph.D.
Associate Professor of Psychology
Palomar College
760-744-1150 x2344
frose@...
====================To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD