Exact test vs Monte Carlo

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Exact test vs Monte Carlo

Walker
Hello all,

We have just recently made the switch from SPSS 15 up to SPSS 19 (with
patches) and added the Exact Tests module.  While running a series crosstabs
with exact tests, I came across some confusing results (shown below - with a
sample crosstab and some background on the analysis).  I am getting large
discrepancies between 1) the asympt Pearson chi-sq and the exact 2-sided
Pearson chi sq  2) The Monte Carlo simulation of the Pearson 2-sided chi-sq
and the exact Pearson 2-sided chi-square and 3) when running exact tests
Fisher's exact test's p-value always equals 1 for all tables.  Can anyone
explain this?

Additionally, when would I use Fisher's exact over Pearson' exact?  I've
done some searching online and in the exact tests manual and was not able to
find anything that would say when to use one versus the other.

Thank you very much for any insight you are able to give.


Sample table:
            yr1     yr2    yr3
StrDis       1       2      2
Dis          4       0      1
Agr          17      16     5
StrAgr       4       4      3

Results:
               Pearson Chi-Sq       Fisher's Exact
Exact               .040                1.000
Monte Carlo         .364                 .288
Asymptotic          .353


Background:
I am analyzing teacher satisfaction with a classroom curriculum over the
course of 3 years.  They are not the same teachers across the 3 years.  The
questions are on a 4 level scale from Strongly Disagree to Strongly Agree.
I have many cells with very low counts and the distributions are unbalanced
by satisfaction rating (more 'agree' than 'disagree' )and also by year (more
responses in earlier years).

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Exact test vs Monte Carlo

Jamie
3) when running exact tests
Fisher's exact test's p-value always equals 1 for all tables.

I too kept getting Fisher exact significance = 1.000 on 2x2 contingency table.

I'm sure this is a rookie mistake...

I had a blank value in one variable so it was trying to calculate a 2x3 rather than 2x2.

Once I excluded the case with the missing data (Data > Select Cases > If condition is satisfied > Not($CASENUM=12) > Filter out unselected cases) everything started working as expected.

Not sure if this helps with your example but thought I'd share.

/Jamie