Comparing two chi-squares on same sample
Locked
Comparing two chi-squares on same sample
 
|
Dear Listers: Here's what seems to be a fairly straightforward question I can't seem to find a clear answer to. I have a number of dichotomous variables obtained from the same sample (N only about 60). This comes from a survey in which respondents could check or not check items in a list of resources they used. Thus, some used resource A only, others used A, B, D, etc. The question is whether those who used A were more likely to also use B than C. Then, do those who used C also tend to use D more than E? So I can do lots of chi-squares A x B, A x C, etc., and get different values of chi-square for each. Then the question is whether some chi-square values are significantly larger than others (with adjustments for the fact that I'm doing dozens of comparisons). This is analogous to comparing different correlations to each other, which I do all the time (and I have an Excel spreadsheet to do this automatically, formulas from Glass & Hopkins, 1984; if anyone wants a copy, please contact me off-list). But I can't dig up a single straightforward explanation of how to do this by Googling. A couple of years ago there was an exchange of posts on a similar question (see 2/3/2011, from R B, "difference between chi squares"), but that seemed to relate to comparing chi-squares between different samples. So: (1) Does anyone know of a relatively simple way to do the comparisons? (2) If I convert chi-squares to phi's, can I treat them like r's in my Glass & Hopkins formulas? (3) I can just run the data to obtain a correlation matrix based on dichotomous values of 0 or 1 for each variable, which I assume should approximate phis. Can I then do the Glass & Hopkins comparisons on those r's? Thanks to all! Allan Research Consulting [hidden email] Business & Cell (any time): 215-820-8100 NEW Address: 587 Shotgun Spring Rd, New Market, VA 22844 Visit my Web site at www.dissertationconsulting.net ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
|
Hi,
Let me review some points:
(1) All of the variables A, B, C, ..., etc are dichotomies, right?
Since you mention phi-coefficients, this would make sense but
correct me if I am wrong (if one or more is not a dichotomy,
then you'd use Cramer's V, right?).
(2) I assume that you're relying upon pages 310-311 in Glass &
Hopkins 1984 for testing the difference between two related
correlation coefficients (section 15.8)? If so, note the last
paragraph (on p311) which states that var1 and var2 have to
be "fixed" while var3 has to have a normal distribution with
common variance across var1 and var2. So, if you have
a chi-square test for AxB and AxC, you could convert these
into phi-AxB and phi-AxC but it should be clear that you
violate the assumption for formula 15.7. I think that it is
telling that Glass & Hopkins 1984 (or 1996) does not provide
a test for differences between phi coefficients. There may
be such a test in the literature but you would have to find it.
(3) The test between related correlations described in (2)
assume that the two correlations have a variable in common,
that is, phi-AxB and phi-AxC. It is not clear how applicable
it is if there are three different variables, i.e., phi-AxB and
phi-CxD.
(4) It would be useful to keep the limitations of the phi correlation
in mind. In Glass & Hopkins 1984, see pages
97-100. G&H
note that phi cannot attain +1 or -1 unless the marginal distributions
for the two variables are the same, with the upper limit depending
on the magnitude of the difference between the proportions (assuming
dichotomies). Unless you have the same marginal distributions
for
all of the variables you're testing, the size of the phi correlations
will
indicate (a) the effect size of the relationship AND (b) the
difference
between the marginal distributions.
It is possible that there is a simple solution to your problem but
I have a feeling that you'll have to take a look at statistics
literature.
If you treat this as a meta-analysis, I'd suggest using a ratio of
the
obtained phi divided by maximum phi if you have nonconstant
marginal distributions and then summarize these effect sizes.
-Mike Palij
New York University
----- Original Message -----
|
Re: Comparing two chi-squares on same sample
Administrator
|
A quick Google search turned up this article, which might help to point the OP in the right direction:
http://link.springer.com/article/10.1007%2Fs00362-007-0105-0#page-1 HTH.
--
Bruce Weaver bweaver@lakeheadu.ca http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." PLEASE NOTE THE FOLLOWING: 1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. 2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/). |
Re: Comparing two chi-squares on same sample
|
|
In reply to this post by Mike
[I am responding to Mike's post because I do not see Alan's original]
A phi coefficient is exactly a Pearson r. Here are lines that are adapted from what I had in the stats-FAQ that I maintained until about 10 years ago. Questions and answers were taken from discussions in the various stats groups on the Internet. - If you don't have any "fixed" dichotomy, the better test is one that uses Fisher's z. === compare correlated correlations? Here is a recent reference about comparing 'correlated' correlations. ( Is r(1,2) greater than r(1,3), taking into account r(2,3) ). It comes down solidly AGAINST Hotelling's so-called 'exact' solution which treats r as r and does not convert it to Fisher's z. Converting r to z as the first step seems to be the RIGHT first step, according to them, and to most people devising tests. Even if it is not perfect, it is better than NOT doing the conversion. Meng, Xiao-Li, Rosenthal, Robert; and Rubin, Donald B. (1992). "Comparing correlated correlation coefficients." _Psychological Bulletin_ , vol 111, 172-175. Hotelling's solution is included in Ferguson's textbook. Comment: Hotelling's is an exact test of a particular hypothesis, one that tests positive correlations against a *residual* of error variation in the criterion. The articles I have read have not made clear that one test is proper when the other is not. === Communication, 2002, from Paul von Hippel " ... if you read the appendix and related articles you realize that they're confining themselves to the case where the regressors are random variables. If the regressors are fixed, as in an experimental design, then Hotelling's test is appropriate. Hotelling (1940) was quite explicit about this, so what Meng, Rosenthal, & Rubin are really criticizing is the mistaken practice of using Hotelling's test with random regressors. "Williams (1959) adapted Hotelling's test to the case of random regressors. In simulation studies Williams' test has held up quite well against the alternatives described by Meng, Rosenthal, & Rubin. This is all in the papers cited in MR&R's bibliography." === end of communication. -- Rich Ulrich ----- Original Message -----
|
Re: Comparing two chi-squares on same sample
|
|
Seems to me that this is clearly a "Check All That Apply" Multiple Response question(s).
If so then you need to ask yourself what is "n" ? Is it the total number of respondents or total number of responses (checks) -- they are different.
Number of responses can differ for respondents ... Responses for a single respondent are not independent. You can of course convert the problem to a Multiple Dichotomy ( vector of 0/1 per person)
and then proceed to any number of analyses. You can generate a correlation matrix as Pearson R and Phi are the same thing in this context. You can fiddle around with partial correlations. There is also a world of Correspondence Analysis tools you might consider.
However, I have never been able to find any simple tools/models for statistical testing of MR questions. With your problem in mind, I googled around for a while and came up with at least one good
reference addressing this problem. Christopher R. Bilder, Thomas M. Loughin Estimation and Testing for Association with Multiple-Response Categorical Variables from Complex Surveys Department of Statistics, University of Nebraska-Lincoln, [hidden email], http://www.chrisbilder.com
Department of Statistics and Actuarial Science, Simon Fraser University Surrey, Surrey, BC, V3T0A3, [hidden email] KEY WORDS: Correlated binary data; Loglinear model; NHANES; Pearson statistic; Pick any/c; Rao-Scott adjustments
Amaya Zárraga and Beatriz Goitisolo Correspondence Analysis of Surveys with Multiple Response Questions S. Ingrassia et al. (eds.), New Perspectives in Statistical Modeling and Data Analysis,
Studies in Classification, Data Analysis, and Knowledge Organization, DOI 10.1007/978-3-642-11363-5_57, c Springer-Verlag Berlin Heidelberg 2011 Amaya Zárraga and Beatriz Goitisolo Correspondence Analysis of Surveys with Conditioned and Multiple Response Questions ... Mark Miller On Tue, Nov 12, 2013 at 2:31 PM, Rich Ulrich <[hidden email]> wrote:
|
|
|
In reply to this post by Allan Lundy, PhD
Allan, A random effects logistic regression model could be employed to perform contrasts such as: pr(A=1,B=1,C=0,D=0) versus pr(A=1,B=0,C=1,D=0) Ryan
|
«
Return to SPSSX Discussion
|
1 view|%1 views
| Free forum by Nabble | Edit this page |