Fisher's Exact test with SPSS
Locked
 
|
Hello I am conscious that there is some debate about the use of Fisher’s Exact test for dimensions other than 2 x 2 (e.g. where the number of table rows is greater than 2). Nevertheless, I am also aware that SPSS generates p-values for such cases.
I would be grateful for advice on the use of these p-values and whether they are trustworthy. The context is teaching statistics to non-specialists who lack time to dabble in alternative tests unless strictly necessary. Many thanks ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dr Margaret MacDougall Medical Statistician and Researcher in Education Centre for Population Health Sciences College of Medicine and Veterinary Medicine Teviot Place University of Edinburgh Edinburgh EH8 9AG Email: [hidden email] Tel: +44(0)131 650 3211 Fax: +44(0)131 650 6909 Personal profile:
http://www.ed.ac.uk/profile/margaret-macdougall The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
|
P.S. In saying “alternative tests”, I meant tests other than Fisher’s Exact test when the 80% rule for expected counts is violated for use of the chi-square test of association. From: MACDOUGALL Margaret
Hello I am conscious that there is some debate about the use of Fisher’s Exact test for dimensions other than 2 x 2 (e.g. where the number of table rows is greater than 2). Nevertheless, I am also aware that SPSS generates p-values for such cases.
I would be grateful for advice on the use of these p-values and whether they are trustworthy. The context is teaching statistics to non-specialists who lack time to dabble in alternative tests unless strictly necessary. Many thanks ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dr Margaret MacDougall Medical Statistician and Researcher in Education Centre for Population Health Sciences College of Medicine and Veterinary Medicine Teviot Place University of Edinburgh Edinburgh EH8 9AG Email: [hidden email] Tel: +44(0)131 650 3211 Fax: +44(0)131 650 6909 Personal profile:
http://www.ed.ac.uk/profile/margaret-macdougall The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
|
In reply to this post by margmacd
Hello To address my own query and hopefully save others some time, I have come across a relevant discussion at
https://stats.stackexchange.com/questions/1805/fishers-exact-test-in-contingency-tables-larger-than-2x2 . Most of the answers provided here reflect advice I have relied on previously in assuming that it is okay to encourage use of Fisher’s Exact test in SPSS for dimensions other than 2 x 2 when the well-known rule for expected counts in not satisfied for use of the chi-square test of association. Comments are still welcome! Best wishes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dr Margaret MacDougall Medical Statistician and Researcher in Education Centre for Population Health Sciences College of Medicine and Veterinary Medicine Teviot Place University of Edinburgh Edinburgh EH8 9AG Email:
[hidden email] Tel: +44(0)131 650 3211 Fax: +44(0)131 650 6909 Personal profile:
http://www.ed.ac.uk/profile/margaret-macdougall From: MACDOUGALL Margaret
P.S. In saying “alternative tests”, I meant tests other than Fisher’s Exact test when the 80% rule for expected counts is violated for use of the chi-square test of association. From: MACDOUGALL Margaret
Hello I am conscious that there is some debate about the use of Fisher’s Exact test for dimensions other than 2 x 2 (e.g. where the number of table rows is greater than 2). Nevertheless, I am also aware that SPSS generates p-values for such cases.
I would be grateful for advice on the use of these p-values and whether they are trustworthy. The context is teaching statistics to non-specialists who lack time to dabble in alternative tests unless strictly necessary. Many thanks ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dr Margaret MacDougall Medical Statistician and Researcher in Education Centre for Population Health Sciences College of Medicine and Veterinary Medicine Teviot Place University of Edinburgh Edinburgh EH8 9AG Email: [hidden email] Tel: +44(0)131 650 3211 Fax: +44(0)131 650 6909 Personal profile:
http://www.ed.ac.uk/profile/margaret-macdougall The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Re: Fisher's Exact test with SPSS - possible answer
|
|
Margaret, I believe that you found a good answer for your question. The usual chi-square test is justified asymptotically, as sample size grows large. The rules of thumb that are commonly taught are not very good. With plentiful computer power and good algorithms available, you could teach or use the exact test form of tests of independence in the general rxc table. If the exact test requires too much time or memory, the Monte Carlo version will do fine. Tony Babinec From: SPSSX(r) Discussion <[hidden email]> On Behalf Of MACDOUGALL Margaret Hello To address my own query and hopefully save others some time, I have come across a relevant discussion at https://stats.stackexchange.com/questions/1805/fishers-exact-test-in-contingency-tables-larger-than-2x2 . Most of the answers provided here reflect advice I have relied on previously in assuming that it is okay to encourage use of Fisher’s Exact test in SPSS for dimensions other than 2 x 2 when the well-known rule for expected counts in not satisfied for use of the chi-square test of association. Comments are still welcome! Best wishes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dr Margaret MacDougall Medical Statistician and Researcher in Education Centre for Population Health Sciences College of Medicine and Veterinary Medicine Teviot Place University of Edinburgh Edinburgh EH8 9AG Email: [hidden email] Tel: +44(0)131 650 3211 Fax: +44(0)131 650 6909 Personal profile: http://www.ed.ac.uk/profile/margaret-macdougall |
|
|
Thanks, Tony. The nagging concern at the back of my mind is the quality of the algorithm that is used by SPSS for Fisher’s Exact test for dimensions other than 2 x 2. I expect that if I had time,
I could do lots of comparative simulations across packages but I don’t and it is unrealistic to expect my students to learn R on top of their other non-specialist learning given their time-scales and lack of prior learning. Best wishes From: SPSSX(r) Discussion <[hidden email]>
On Behalf Of Anthony Babinec Margaret, I believe that you found a good answer for your question. The usual chi-square test is justified asymptotically, as sample size grows large. The rules of thumb that are commonly taught are not very good. With plentiful computer power and good algorithms available, you could teach or use the exact test form of tests of independence in the general rxc table. If the exact test requires too much time or memory, the Monte Carlo version will do fine. Tony Babinec From: SPSSX(r) Discussion <[hidden email]>
On Behalf Of MACDOUGALL Margaret Hello To address my own query and hopefully save others some time, I have come across a relevant discussion at
https://stats.stackexchange.com/questions/1805/fishers-exact-test-in-contingency-tables-larger-than-2x2 . Most of the answers provided here reflect advice I have relied on previously in assuming that it is okay to encourage use of Fisher’s Exact test in SPSS for dimensions other than 2 x 2 when the well-known rule for expected counts in not satisfied for use of the chi-square test of association. Comments are still welcome! Best wishes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dr Margaret MacDougall Medical Statistician and Researcher in Education Centre for Population Health Sciences College of Medicine and Veterinary Medicine Teviot Place University of Edinburgh Edinburgh EH8 9AG Email: [hidden email] Tel: +44(0)131 650 3211 Fax: +44(0)131 650 6909 Personal profile:
http://www.ed.ac.uk/profile/margaret-macdougall ===================== To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Re: Fisher's Exact test with SPSS - possible answer
|
|
Margaret, SPSS Inc. worked with Cytel Software to deliver the Exact algorithms offered in IBM SPSS Statistics. Cytel’s first product was StatXact, based on work by Mehta, Patel, and their students and collaborators. The current version of StatXact is version 11. You can browse their website and look for fact sheets and other info at http://www.cytel.com/software/statxact I have not looked closely at the R implementation, but the description mentions a “network” algorithm and could be based on the original work referenced above. Tony Babinec From: MACDOUGALL Margaret <[hidden email]> Thanks, Tony. The nagging concern at the back of my mind is the quality of the algorithm that is used by SPSS for Fisher’s Exact test for dimensions other than 2 x 2. I expect that if I had time, I could do lots of comparative simulations across packages but I don’t and it is unrealistic to expect my students to learn R on top of their other non-specialist learning given their time-scales and lack of prior learning. Best wishes |
Re: Fisher's Exact test with SPSS - possible answer
|
|
In reply to this post by margmacd
Margaret, Please notice that the 2nd reply on that page tells why the Fisher's-type answer
for "exact" is not the only "exact" answer that is possible. - For tables bigger than 2x2, there are several ways of ordering what-is-extreme.
I always keep in mind that the Fisher's test AND the others are based on "fixed marginals", which is what occurs with pre-determined equal group sizes (say) and a median split on a criterion. Chi-squared with Yates's correction gives a very good estimator of that test, for 2x2. That result may be conservative when the assumption on the Margins is not met. IIRC, Bruce's site has references for another version of chi-squared that has tested better in simulations for the more general case.
Finally - and this nullifies a lot of questions - the "error" introduced by having tiny Ns in the contingency table is that a tiny denominator will yield an inflated contribution to the sum of
the terms by cell. So the test result is questionable when the chi-squared is large, but not
when it is small. Thus, a non-significant chi-squared result is robust.
-- Rich Ulrich From: SPSSX(r) Discussion <[hidden email]> on behalf of MACDOUGALL Margaret <[hidden email]>
Sent: Tuesday, May 1, 2018 7:20:20 AM To: [hidden email] Subject: Fisher's Exact test with SPSS - possible answer Hello
To address my own query and hopefully save others some time, I have come across a relevant discussion at https://stats.stackexchange.com/questions/1805/fishers-exact-test-in-contingency-tables-larger-than-2x2 . Most of the answers provided here reflect advice I have relied on previously in assuming that it is okay to encourage use of Fisher’s Exact test in SPSS for dimensions other than 2 x 2 when the well-known rule for expected counts in not satisfied for use of the chi-square test of association.
Comments are still welcome!
Best wishes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dr Margaret MacDougall Medical Statistician and Researcher in Education Centre for Population Health Sciences College of Medicine and Veterinary Medicine Teviot Place University of Edinburgh Edinburgh EH8 9AG
Email: [hidden email] Tel: +44(0)131 650 3211 Fax: +44(0)131 650 6909 Personal profile: http://www.ed.ac.uk/profile/margaret-macdougall
From: MACDOUGALL Margaret
P.S. In saying “alternative tests”, I meant tests other than Fisher’s Exact test when the 80% rule for expected counts is violated for use of the chi-square test of association.
From: MACDOUGALL Margaret
Hello
I am conscious that there is some debate about the use of Fisher’s Exact test for dimensions other than 2 x 2 (e.g. where the number of table rows is greater than 2). Nevertheless, I am also aware that SPSS generates p-values for such cases. I would be grateful for advice on the use of these p-values and whether they are trustworthy. The context is teaching statistics to non-specialists who lack time to dabble in alternative tests unless strictly necessary.
Many thanks
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dr Margaret MacDougall Medical Statistician and Researcher in Education Centre for Population Health Sciences College of Medicine and Veterinary Medicine Teviot Place University of Edinburgh Edinburgh EH8 9AG
Email: [hidden email] Tel: +44(0)131 650 3211 Fax: +44(0)131 650 6909 Personal profile: http://www.ed.ac.uk/profile/margaret-macdougall
|
|
|
In reply to this post by margmacd
The Exact Tests both Monte Carlo and exact calculations code beyond 2x2 comes from well known experts in this area. On Tue, May 1, 2018 at 2:04 PM, MACDOUGALL Margaret <[hidden email]> wrote:
|
|
|
In reply to this post by Anthony Babinec
Thanks, Tony. I wasn’t aware of any such connection. On searching the StatXact website for documentation for Fisher’s Exact test, I am pointed to the networked algorithm at
https://www.jstor.org/stable/2288652?seq=1#page_scan_tab_contents, which allows extension beyond the 2 x 2 case. This suggests that the above algorithm is the one used by StatXact
and, it seems from your reply, also by SPSS. Best wishes From: SPSSX(r) Discussion <[hidden email]>
On Behalf Of Anthony Babinec Margaret, SPSS Inc. worked with Cytel Software to deliver the Exact algorithms offered in IBM SPSS Statistics. Cytel’s first product was StatXact, based on work by Mehta, Patel, and their students and collaborators. The current version of StatXact is version 11. You can browse their website and look for fact sheets and other info at http://www.cytel.com/software/statxact I have not looked closely at the R implementation, but the description mentions a “network” algorithm and could be based on the original work referenced above. Tony Babinec From: MACDOUGALL Margaret <[hidden email]>
Thanks, Tony. The nagging concern at the back of my mind is the quality of the algorithm that is used by SPSS for Fisher’s Exact test for dimensions other than 2 x 2. I expect that if I had time, I could do lots of comparative simulations
across packages but I don’t and it is unrealistic to expect my students to learn R on top of their other non-specialist learning given their time-scales and lack of prior learning. Best wishes ===================== To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Re: Fisher's Exact test with SPSS - possible answer
Administrator
|
In reply to this post by Rich Ulrich
The alternative test for 2x2 tables that Rich is referring to below is the
N-1 Chi-square. You can read about it here: http://www.iancampbell.co.uk/twobytwo/twobytwo.htm Ian Campbell's simulations show that for designs where the marginal totals are not fixed, the N-1 Chi-square outperforms the Fisher-Irwin test (aka., Fisher's exact test) provided all expected frequencies are 1 or more. (When marginal totals are fixed, Campbell recommends using the Fisher-Irwin test.) For 2x2 tables, the N-1 Chi-square is equivalent to the Linear-by-linear Chi-square that CROSSTABS reports. I am not aware of any variation on the N-1 Chi-square for tables larger than 2x2. However, I do remember that Ray Koopman once speculated (in an e-mail conversation with me) that multiplying Pearson's Chi-square by (N-1)/N might provide a better test statistic even for larger tables. I don't know if anyone has ever investigated that possibility. HTH. p.s. - Here's some info for anyone who is interested in why Campbell calls FET the Fisher-Irwin test. https://en.wikipedia.org/wiki/Joseph_Oscar_Irwin J. O. Irwin (1935) Tests of Significance for Differences between Percentages Based on Small Numbers, Metron, Vol. 12, pp. 83–94. Rich Ulrich wrote > Margaret, > > Please notice that the 2nd reply on that page tells why the Fisher's-type > answer > > for "exact" is not the only "exact" answer that is possible. - For tables > bigger than 2x2, > > there are several ways of ordering what-is-extreme. > > > I always keep in mind that the Fisher's test AND the others are based on > "fixed marginals", > > which is what occurs with pre-determined equal group sizes (say) and a > median split on > > a criterion. Chi-squared with Yates's correction gives a very good > estimator of that test, for 2x2. > > That result may be conservative when the assumption on the Margins is not > met. IIRC, Bruce's > > site has references for another version of chi-squared that has tested > better in simulations > > for the more general case. > > > Finally - and this nullifies a lot of questions - the "error" introduced > by having tiny Ns in the > > contingency table is that a tiny denominator will yield an inflated > contribution to the sum of > > the terms by cell. So the test result is questionable when the chi-squared > is large, but not > > when it is small. Thus, a non-significant chi-squared result is robust. > > > -- > > Rich Ulrich > > ________________________________ > From: SPSSX(r) Discussion < > SPSSX-L@.UGA > > on behalf of MACDOUGALL Margaret < > Margaret.MacDougall@.ac > > > Sent: Tuesday, May 1, 2018 7:20:20 AM > To: > SPSSX-L@.UGA > Subject: Fisher's Exact test with SPSS - possible answer > > > Hello > > > > To address my own query and hopefully save others some time, I have come > across a relevant discussion at > https://stats.stackexchange.com/questions/1805/fishers-exact-test-in-contingency-tables-larger-than-2x2 > . > > Most of the answers provided here reflect advice I have relied on > previously in assuming that it is okay to encourage use of Fisher’s Exact > test in SPSS > > for dimensions other than 2 x 2 when the well-known rule for expected > counts in not satisfied for use of the chi-square test of association. > > > > Comments are still welcome! > > > > Best wishes > Margaret > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Dr Margaret MacDougall > > Medical Statistician and Researcher in Education > (Senior Lecturer) > > Centre for Population Health Sciences > > College of Medicine and Veterinary Medicine > > Teviot Place > > University of Edinburgh > > Edinburgh EH8 9AG > > > > Email: > Margaret.MacDougall@.ac > <mailto: > Margaret.MacDougall@.ac > > > > Tel: +44(0)131 650 3211 > > Fax: +44(0)131 650 6909 > > Personal profile: http://www.ed.ac.uk/profile/margaret-macdougall > > > > From: MACDOUGALL Margaret > Sent: 01 May 2018 10:35 > To: ' > SPSSX-L@.UGA > ' < > SPSSX-L@.UGA > > > Subject: RE: Fisher's Exact test with SPSS > > > > > > P.S. In saying “alternative tests”, I meant tests other than Fisher’s > Exact test when the 80% rule for expected counts is violated for use of > the chi-square test of association. > > > > From: MACDOUGALL Margaret > Sent: 01 May 2018 10:29 > To: ' > SPSSX-L@.UGA > ' < > SPSSX-L@.UGA > <mailto: > SPSSX-L@.UGA > >> > Subject: Fisher's Exact test with SPSS > > > > Hello > > > > I am conscious that there is some debate about the use of Fisher’s Exact > test for dimensions other than 2 x 2 (e.g. where the number of table rows > is greater than 2). Nevertheless, I am also aware that SPSS generates > p-values for such cases. I would be grateful for advice on the use of > these p-values and whether they are trustworthy. The context is teaching > statistics to non-specialists who lack time to dabble in alternative tests > unless strictly necessary. > > > > Many thanks > Best wishes > Margaret > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Dr Margaret MacDougall > > Medical Statistician and Researcher in Education > (Senior Lecturer) > > Centre for Population Health Sciences > > College of Medicine and Veterinary Medicine > > Teviot Place > > University of Edinburgh > > Edinburgh EH8 9AG > > > > Email: > Margaret.MacDougall@.ac > <mailto: > Margaret.MacDougall@.ac > > > > Tel: +44(0)131 650 3211 > > Fax: +44(0)131 650 6909 > > Personal profile: http://www.ed.ac.uk/profile/margaret-macdougall > > > > ===================== To manage your subscription to SPSSX-L, send a > message to > LISTSERV@.UGA > <mailto: > LISTSERV@.UGA > > (not to SPSSX-L), with no body text except the command. To leave the > list, send the command SIGNOFF SPSSX-L For a list of commands to manage > subscriptions, send the command INFO REFCARD > > ===================== > To manage your subscription to SPSSX-L, send a message to > LISTSERV@.UGA > (not to SPSSX-L), with no body text except the > command. To leave the list, send the command > SIGNOFF SPSSX-L > For a list of commands to manage subscriptions, send the command > INFO REFCARD ----- -- Bruce Weaver [hidden email] http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." NOTE: My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. -- Sent from: http://spssx-discussion.1045642.n5.nabble.com/ ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
--
Bruce Weaver bweaver@lakeheadu.ca http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." PLEASE NOTE THE FOLLOWING: 1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. 2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/). |
| Free forum by Nabble | Edit this page |