Historical query regarding SPSS functionality
Locked
 
|
Season’s greetings! I have a historical question! Is anyone able to advise me on when functionality to run the following tests first became available in SPSS:
and
Thanks in advance ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ https://www.ed.ac.uk/profile/margaret-macdougall |
|
|
I think that Jon Peck will have to join this thread in order to correct errors or clarify points in my presentation. That being said, consider the following: (1) If I am not mistaken, prior to the 1990s, SPSS was not able to perform "exact tests" that are represented by the McNemar-Bowker test and the Fisher-Freeman-Halton. This was a general limitation of the major statistical software packages (e.g., SAS) because of hardware and programming limitations. (2) The statisticians Cyrus Mehta and Nitin Patel had worked on computer programming of exact tests and developed the software StatXact (misc statistical tests) and LogXact (exact logistic regression). Their algorithms were able to overcome the limitations of previous implementations and was used in the development of commercial statistical software. Version 1 of the PC version of StatXact was released around 1990 and here are a couple reviews of the early software: Mehta, C. (1991). StatXact: A Statistical Package for Exact Nonparametric Inference. The American Statistician, 45(1), 74-75. doi:10.2307/2685246 Sprent, P., & Mehta, C. (1990). Journal of the Royal Statistical Society. Series C (Applied Statistics), 39(3), 391-395. doi:10.2307/2347397 StatXact was developed on mainframe and similar computers but StatXact was developed specifically for MS-DOS systems. Subsequent versions continued to be available across different computer platforms. (3) What is significant about Mehta and Patel's work is that it was easier for the major statistical packages to either provide the capability to call StatXact (if it were on the computer system) or the algorithms could be licensed into the software. I believe that SAS was the first system to incorporate M&P's exact test procedure. By 1995 SPSS provide an exact tests module and incorporated some of the tests into existing procedures (e.g., crosstabs; not sure which version of SPSS this was). Hilbe (2004) wrote a review of SPSS ver 12 in which he focuses on the logistic regression procedure (apparently the LogXact software was not used by SPSS) and the Exact tests module -- he covers some of the history between SPSS and StatXact which might be useful. See: Hilbe, J. (2004). A Review of SPSS 12.01, Part 2. The American Statistician, 58(2), 168-171. Retrieved December 31, 2020, from http://www.jstor.org/stable/27643530
(4) Jon Peck might know if StatXact code was used in SPSS in version before version 12 and may be able to give a better/more accurate/exact? date (I bet it is between 1993 to 1995). I am curious though as to why you are interested in this question. I would also suggest searching Jstor (which has ASA journal and other relevant statistical journals) for reviews of software and related issues. -Mike Palij New York University On Thu, Dec 31, 2020 at 1:26 PM MACDOUGALL Margaret <[hidden email]> wrote:
|
|
|
It is true that the Fisher-Freeman-Halton and McNeman-Bowker tests depend on the StatXact code via the METHOD subcommand of CROSSTABS and require the Exact Tests option. The Command Syntax Reference manual started listing new and updated syntax as of SPSS version 13. This subcommand is not mentioned as a change, so it would have been added in version 12 or earlier, but I don't have an exact date. I no longer have access to the SPSS code base, which would have an exact record of the changes. My recollection is that integrating the StatXact code, which was fortunately not my task, was a nightmare due to the way that code was written. On Thu, Dec 31, 2020 at 1:49 PM Michael Palij <[hidden email]> wrote:
|
|
|
Hilbe (2004, see full reference below) goes into a fair amount of detail about SPSS ver 12.01 and goes over StatXact's contribution. I have the feeling that it is unlikely that StatXact code was included in earlier versions. On a related note, I dug up my copy of the SPSS Exact Tests ver 21 manual which has the curious copyright dates of 1989 and 2012 -- 1989 is before the release of StatXact version 1 (both copyrights of IBM). I have no idea why 1989 is used especially since the reference list has no publication for 1989 (does it refer to an unpublished document?). Anyway, in the front matter of the manual is the following Acknowledgment: Acknowledgments Exact Tests is the result of a collaboration between Cytel Software Corporation and SPSS Inc. The exact algorithms were developed by Cytel. Integrating the exact engines into the user interface and documenting the statistical methods in a comprehensive user manual were tasks shared by both organizations. We would like to thank our fellow developers, Yogesh Gajjar, Hemant Govil, Pralay Senchaudhuri, and Shailesh Vasundhara of Cytel. We owe a special debt to Professor Marvin Zelen for creating an exciting intellectual environment in the Department of Biostatistics at Harvard. He encouraged us to work on a number of challenging research problems in computational statistics, and this research has culminated in the development of Exact Tests. Cyrus R. Mehta and Nitin R. Patel Cytel Software Corporation and Harvard School of Public Health Cambridge, Massachusetts **************************************** Final point: one of the references is for a 1986 paper that describes a Fortran program titled FEXACT for the Fisher test of contingency tables. This leads me to think that most of StatXact was also probably written in Fortran. But by the 1990s I would assume that SPSS was no longer using Fortran code -- wasn't there a conversion to some version of C by that time (I think SAS had overhauled its code around this time), If this is true, then I can imagine converting the StatXact Fortran into SPSS C code would have been a difficult task. Wasn't this the problem with BMDP? Just curious. Take care and have a Better New Year, -Mike Palij New York University On Thu, Dec 31, 2020 at 4:25 PM Jon Peck <[hidden email]> wrote:
|
Re: Historical query regarding SPSS functionality
|
It would be interesting to know list members' experiences wrt how often and
under what conditions exact tests led to different substantive statements. ----- Art Kendall Social Research Consultants -- Sent from: http://spssx-discussion.1045642.n5.nabble.com/ ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Art Kendall
Social Research Consultants |
|
|
In reply to this post by Mike
Mr Pack Rat! The copyright dates are an artifact of the IBM acquisition of SPSS. IBM dictated that everything be marked with a 1989 copyright notice - I have no idea why, but there was a lot of SPSS code and other materials that didn't have a copyright date in it at the time, so that may have been an approximation. IBM required a copyright notice in every single thing that SPSS owned. The Cytel code was written in C, but it used a number of not so good practices that made it hard to port or adapt. SPSS was using both Fortran and C code at the time - maybe C++ by then, but had stopped doing new development in Fortran (it used a Ratprep preprocessor for it). There is still plenty of good Fortran code in the product, but Fortran wasn't the problem for the StatXact code. On Thu, Dec 31, 2020 at 2:55 PM Michael Palij <[hidden email]> wrote:
|
|
|
In reply to this post by Art Kendall
On Thu, Dec 31, 2020 at 5:14 PM Art Kendall <[hidden email]> wrote: It would be interesting to know list members' experiences wrt how often and I haven't had the need to use exact tests in recent years (decades?) but in Chapter 2 of the SPSS Exact Tests ver 21 manual (p31) the authors identify the conditions under which the "asymptotic test/prob" (i.e., usual test statistic) can be different from the "Exact test/prob". I copy the relevant text below: When to Use Asymptotic P Values Although the exact p value can be shown to converge mathematically to the corresponding asymptotic p value as the sample size becomes infinitely large, this property is not of much practical value in guaranteeing the accuracy of the asymptotic p value for any specific data set. There are many different data configurations where the asymptotic methods perform poorly. These include small data sets, data sets containing ties, large but unbalanced data sets, and sparse data sets. A numerical example follows for each of these situations. ********************************************************* There are probably other conditions where the obtained/asymptotic test/probability will differ from the exact test/prob but I think the major distinction might be between experimental designs and observational studies, with the latter providing more opportunities for differences between asymptotic vs exact value. With good experimental design one should be able to avoid small datasets, unbalanced groupings, sparse data set, etc. But this may also depend upon whether the independent variables are actually under control of the research, the populations that are sampled have adequate numbers of available participants, and other oddities that can affect data collection. -Mike Palij New York University ----- |
Re: Historical query regarding SPSS functionality
|
|
There are circumstances where results CAN be different.
From a theoretical and policy perspective, I am a strong believer in applying the SO WHAT test. I was interested in hearing about experiences when results WERE or WERE NOT different in a meaningful way in comparing conventional test results and exact test results. I.e., for many purposes P LT .055, .050. and .045 are substantively equivalent. Similarly, on this list (and in other contexts) people have often worried whether intervals between values are exactly equal. (To me the question is whether or not the intervals are severely discrepant.) What I have suggested is using CATREG to actually test whether there was a substantive difference in the results under varied level of measurement assumptions. *To date* I have not heard of a situation when reporting the conventional interval level assumption results made a substantive difference. To prevent problems with reviewers, it helps to add a note that the level of measurement assumption was looked at. ----- Art Kendall Social Research Consultants -- Sent from: http://spssx-discussion.1045642.n5.nabble.com/ ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Art Kendall
Social Research Consultants |
|
|
I have used CATREG on occasion to model Likert scales. The transformation plots it can produce can be helpful in evaluating the measurement level. If the transformation plot is pretty linear, it's a good sign that ordinary regression and the like will not be misleading. On Fri, Jan 1, 2021 at 7:53 AM Art Kendall <[hidden email]> wrote: There are circumstances where results CAN be different. |
|
|
In reply to this post by Art Kendall
Art,
It is pretty common to get higher power for exact tests. So if you have smaller sample sizes (and assuming the test itself makes sense, many say Fisher's only is valid if the marginals are fixed), they can make quite a bit of sense. Here are a few examples I have written about: - Quantifying racial bias is peremptory challenges is an example for Fisher's (https://andrewpwheeler.com/2014/09/25/quantifying-racial-bias-in-peremptory-challenges/) - Benford's test with small samples (https://andrewpwheeler.com/2017/03/31/using-the-exact-reference-distribution-for-small-sample-benford-tests/). So this isn't for bivariate tables, but a single set of categories, e.g. A = 25%, B = 50%, C = 25%, etc. and see if they deviate from those null proportions. Andy ----- Andy W [hidden email] http://andrewpwheeler.wordpress.com/ -- Sent from: http://spssx-discussion.1045642.n5.nabble.com/ ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Re: Historical query regarding SPSS functionality
|
|
Thanks for the feedback.
I really like the careful reasoning in the examples. I have rarely dealt with samples this small except as exercises or in discovery sampling. Usually when I have seen numbers of cases smaller than a few hundred it was in an actual experiment. In quasi-experimental situations, small numbers of cases were N=n or close to N=n. In my later years when the exact tests were available in SPSS, I ran tests both conventionally and exactly but rarely had fewer than 200 or so cases. I was aware there could be differences in results, but did not happen on them. I never systematically looked at differences in conclusions. ----- Art Kendall Social Research Consultants -- Sent from: http://spssx-discussion.1045642.n5.nabble.com/ ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Art Kendall
Social Research Consultants |
Re: Historical query regarding SPSS functionality
Administrator
|
In reply to this post by Art Kendall
A related issue concerns exact vs approximate confidence intervals for
binomial proportions. As Agresti & Coull (1998) argued, approximate can be better than exact by some criteria. Agresti, A., & Coull, B. A. (1998). Approximate is better than “exact” for interval estimation of binomial proportions. The American Statistician, 52(2), 119-126. Art Kendall wrote > It would be interesting to know list members' experiences wrt how often > and > under what conditions exact tests led to different substantive statements. > > > > ----- > Art Kendall > Social Research Consultants > -- > Sent from: http://spssx-discussion.1045642.n5.nabble.com/ > > ===================== > To manage your subscription to SPSSX-L, send a message to > LISTSERV@.UGA > (not to SPSSX-L), with no body text except the > command. To leave the list, send the command > SIGNOFF SPSSX-L > For a list of commands to manage subscriptions, send the command > INFO REFCARD ----- -- Bruce Weaver [hidden email] http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." NOTE: My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. -- Sent from: http://spssx-discussion.1045642.n5.nabble.com/ ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
--
Bruce Weaver bweaver@lakeheadu.ca http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." PLEASE NOTE THE FOLLOWING: 1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. 2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/). |
«
Return to SPSSX Discussion
|
1 view|%1 views
| Free forum by Nabble | Edit this page |