> --- snip ---
> but I think Simpson's Paradox presents the fallacy most directly (see:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__en.wikipedia.org_wiki_Ecological-5Ffallacy-23Simpson-27s-5Fparadox&d=DwICAg&c=slrrB7dE8n7gBJbeO0g-IQ&r=A8kXUln5f-BYIUaapBvbXA&m=300e3cAU3FH4Wnoe4MY_n1Jmt3K-xsHo9cXm3y8sse0&s=sDi6eBkVmVauFo92kBooAiYs9NvPwMPBB0WifOkOGaY&e= ).
Hmm. You're going to have to explain this one to me. Simpson's Paradox is
often illustrated with examples where there appears to be no association
between X and Y, but when one "controls" for Z, the X-Y association becomes
apparent. As this article suggests, it is an example of suppression, or
negative confounding, as epidemiologists might call it:
https://urldefense.proofpoint.com/v2/url?u=https-3A__link.springer.com_article_10.1186_s12982-2D019-2D0087-2D0&d=DwICAg&c=slrrB7dE8n7gBJbeO0g-IQ&r=A8kXUln5f-BYIUaapBvbXA&m=300e3cAU3FH4Wnoe4MY_n1Jmt3K-xsHo9cXm3y8sse0&s=4zYrM0WEJGa9AyckGqWEStFDWkuDwpHt5FoH70LHtvQ&e=
See the example in Table 1.
A few points:
(1) I think that the case you are referring to, i.e., no association between X and Y
when Z is controlled for, is a special case of Simpson's paradox, that is,
sometimes suppression may give rise to the Simpson's paradox but
Simpson's paradox can still occur without suppression. More on this
point shortly.
(2) Please see the following article:
Kievit, R., Frankenhuis W., Waldorp L., & Borsboom, D. (2013). Simpson's paradox in
psychological science: a practical guide.Frontiers in Psychology, 4, 513.
The article can be accessed at:
The abstract to the article follows:
The direction of an association at the population-level may be reversed within the subgroups
comprising that population --- a striking observation called Simpson's paradox. When facing this
pattern, psychologists often view it as anomalous. Here, we argue that Simpson's paradox is
more common than conventionally thought, and typically results in incorrect interpretations --
potentially with harmful consequences. We support this claim by reviewing results from cognitive
neuroscience, behavior genetics, clinical psychology, personality psychology, educational psychology,
intelligence research, and simulation studies. We show that Simpson's paradox is most likely to
occur when inferences are drawn across different levels of explanation (e.g., from populations
to subgroups, or subgroups to individuals). We propose a set of statistical markers indicative
of the paradox, and offer psychometric solutions for dealing with the paradox when encountered --
including a toolbox in R for detecting Simpson's paradox. We show that explicit modeling of situations
in which the paradox might occur not only prevents incorrect interpretations of data, but also
results in a deeper understanding of what data tell us about the world.
NOTE: emphasis of the last sentence is added. Modeling the data pattern is important because
of the next point.
(3) On page 6 of the PDF for the article (scroll down on the webpage) the following quote
appears:
A Survival Guide to Simpson's Paradox
We have shown that SP may occur in a wide variety of research designs, methods, and questions.
As such, it would be useful to develop means to “control” or minimize the risk of SP occurring, much
like we wish to control instances of other statistical problems. Pearl (1999, 2000) has shown that
(unfortunately) there is no single mathematical property that all instances of SP have in common, and
therefore, there will not be a single, correct rule for analyzing data so as to prevent cases of SP.
Based on graphical models, Pearl (2000) shows that conditioning on subgroups may sometimes be
appropriate, but may sometimes increase spurious dependencies (see also Spellman et al., 2001).
It appears that some cases are observationally equivalent, and only when it can be assumed that the
cause of interest does not influence another variable associated with the effect, a test exists to determine
whether SP can arise (see Pearl, 2000, chapter 6 for details).
Note #1: Emphasis of the sentence containing Judah Pearl's statement that there is no single math property
that underlie all instances of Simpson's Paradox. This implies that some cases of SP may be due
to suppression but other mechanisms are probably operating to produce the pattern, hence the need
for something like the author's R toolkit to investigate an instance of SP in detail.
Note #2: I think that this article is helpful in thinking about Simpson's Paradox even though most of
the examples are from psychology because it shows how it can appear in a wide variety of situations
(sometimes unnoticed) as well as the difference between SP based on different groups of subjects
and SP based on repeated measurements of individuals in different groups.
Perhaps what you're suggesting is that to get the correct estimate of the
X-Y association, one must compute estimates within each stratum of the
confounder, and then a pooled estimate of those within-stratum estimates
(rather than pooling the data across strata)? I don't see that as being the
same thing as computing the association between aggregate measures of X and
Y, though.
--- snip the rest ---
No, I was trying to suggest that Simpson's paradox may reflect the operation of
different mechanisms which is one reason why I pointed out that multilevel analysis
is one strategy that some researchers are using to understand SP.
-Mike Pallij
New York University