Posted by
Rudobeck, Emil (LLU) on
Oct 13, 2016; 10:56pm
URL: http://spssx-discussion.165.s1.nabble.com/Undefined-Mauchly-s-Test-tp5733212p5733253.html
Mike, thank you for your explanations and references. Even without a widely used procedure, I think the methods you mentioned are rather thorough for finding out if sphericity is violated and I can use them whenever Mauchly’s test is unavailable (or in some cases to complement or substitute for Mauchly’s test).
I am familiar with the matrices in your link since I use linear mixed models (LMM). As long as we’re on the topic, I wanted to clarify something: I had read a while ago that repeated measures ANOVA assumes the CS structure and it’s one of its weaknesses as compared to LMM, which has flexible covariance structures. With a better understanding of sphericity, I’m curious as to how ANOVA could use CS where in fact sphericity (a special case of CS) is all that’s required to meet the conditions of the test. Maybe I have misunderstood something.
My main question was addressing specifically Mauchly’s test. Design is a separate question, we can certainly look at that. Before proceeding, I should say that I am in fact using LMM to analyze most repeated measures since it has many advantages. However, building models is rather time consuming and sometimes I still do resort to ANOVA for quick calculations. Another quick note: although the measurements I’ve been doing have been employed in neuroscience for decades, the vast majority of statistical analyses have been anything but rigorous (and many are usually considered wrong, such as running individual t-tests for specific repeated measures without adjusting alpha). Bad stats in papers is a well-known issue and unfortunately neuroscience is prone to incorrect stats much more than the social or geological sciences. Quite often there aren’t enough (or any) details to trace back the statistics.
Bare bones of the design: 30 animals are divided into 2 treatment doses and one control, for a total of 3 groups (sometimes more). Each animal’s hippocampus is sectioned into thin slices. Recordings are collected from 1-2 slices per animal. Essentially, each slice is electrically stimulated and the baseline response is recorded every 50 s, for about 15 min (18 repetitions). Then a strong train of pulses is applied, after which the recordings (by now potentiated due to the train) are resumed for another 60 minutes or longer (72+ repetitions). This is known as long-term potentiation and is thought to be the process that helps us learn new information. The final result looks like an exponential curve, as can be seen here in Fig.A:
http://www.pnas.org/content/109/43/17651/F1.large.jpg. The responses are normalized to the pre-train input and only the post-train curves are compared to each other. The repeated measures are a time-varying covariate, so I have been using LMM with polynomial regression to analyze the data, which I think is perfect for it. However, if there are other suggestions, I’d be curious to hear them. By the way, I have also tried SEM, but SEM is really sensitive to sample size. I cannot analyze this data with SEM unless I drop points or average them, which introduces its own statistical issues. I prefer to use the entire data since interactions can be important. The hypothesis is that the later phase of the curves will be decreased compared to the control group. Sure, I could just compare the later phases to each other, where the trends are purely linear, but having done the experiments, it would make no sense at all to ignore any possible differences during the early phase. Hence my notion that the entire duration is important – the data is too precious to waste.
While the biological mechanisms are different for the early vs late response, no strict cutoff has been established. I could choose an approximate cutoff and divide the curve into 2 or 3 pieces. I think this would require spline analysis, which SPSS can’t do easily. Furthermore, alpha would need to be further adjusted for each additional piece that’s created and I think this “punishment” could be rather severe. That’s why my solution remains LMM, despite that it's a pain in the ass to go through all the models.
ER
________________________________________
From: SPSSX(r) Discussion [
[hidden email]] on behalf of Mike Palij [
[hidden email]]
Sent: Saturday, October 08, 2016 1:13 PM
To:
[hidden email]
Subject: Re: Undefined Mauchly's Test
On Saturday, October 08, 2016 12:53 PM, Rich Ulrich writes:
> Yes, my question was about design. If you don't have a design,
>you don't have anything to talk about.
If I can re-state what Rich is saying: "Design drives Analysis".
Designs are set up so that certain variables/factors are allowed
to express an effect on an outcome/dependent variable, both
alone or in combination with other factors.
>If your method is "well established" and in use for decades,
>you should hardly have any question for us.
It might clarify things if the OP provided a reference to a published
article(s) that show the analysis/analyses he is trying to duplicate.
>If you can't follow their method, you must be doing something
>different. They never reported Mauchly's test in a situation where
>it is impossible to compute (I hope).
After doing a search of the literature, let me try to restate the OP
original question/situation. Let N equal the sample size and P equal
the number of repeated measures. Mauchly's test and other likelihood
tests are undefined when N < P. Are there tests for sphericity when
N < P? Muni S, Srivastava has done most of the work in this area
in the past few decades and one relevant source for the OP is:
Srivastava, M. S. (2006). Some tests criteria for the covariance
matrix with fewer observations than the dimension. Acta Comment.
Univ. Tartu. Math, 10, 77-93.
A copy can be obtained at:
http://www.utstat.utoronto.ca/~srivasta/covariance1.pdfA scholar.google.com search of Srivastava and sphericity test
will provide a shipload of references by and on Srivastava's work
in this area. The next question is whether Srivastava's test is
implemented in any of the standard statistical packages or does
one have row one's own version. I found one paper dealing with
this situation but it uses SAS IML for a macro called LINMOD
for conducting testing. It is:
Chi, Y.-Y., Gribbin, M., Lamers, Y., Gregory, J. F., & Muller, K. E.
(2012). Global hypothesis testing for high-dimensional repeated
measures outcomes. Statistics in Medicine, 31(8), 724-742.
http://doi.org/10.1002/sim.4435Available at pubmed at:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3396026/LINMOD and other software goodies is available at:
http://samplesizeshop.org/software-downloads/other/>Mauchly's is a warning that you should be careful about the tests.
>Well, with 72 periods of measure in an hour, you /should/ be careful
>about the tests, period.
The references above appear to deal with situations where P is
large (e.g., DNA microarrays) and the traditional methods fail.
The OP should be familiar with at least some of this literature.
Also, I guess that LINMOD might be translatable into SPSS
matrix language which the OP might consider doing (if doing
the analysis in SAS is not an option). Or he can pay Dave Marso
to do it. ;-)
>But the tests that you should be concerned about are tests of
>hypothesis. I preferred using the old BDMP program (2V? 4V?)
If you're talking about orthogonal polynomial analysis, it is 2V
(I still have the manuals; one specifies "Orthogonal." in the
/Design paragraph, along with "Point(j)= .." if the spacing is
not constant).
>because it gave linear trends (say) with the test performed
>against the actual variation measured for the trend. (I don't
>know if SPSS can do that testing, but it is not automatic.)
When done in GLM, a repeated measures ANOVA will automatically
generate the orthogonal polynomial or one can specify the degree
of the polynomial (one probably doesn't want the output for 71
polynomials). This is one of the annoying features of GLM because
it produces this output even when the within-subject factor is
a unordered category.
>Anyway, if you can't state the hypothesis, you can't get started.
Or, you can use that statistics to serve as an "automatic inference
engine" as Gerd Gigerenzer calls it when one engages in
"mindless statistics".
>If you expect an early response followed by a later decline
>(which could be reasonable for a brain-response to stimulus),
>the simplest execution might be to break the 72 periods into
>two or more sets: early response, middle, later. That is
>especially true if the early response is very strong: Look for
>the early linear trend, then see if it continues or if the means
>regress back to the start.
Or ask for linear, quadratic and cubic polynomials (maybe up
to quintic) as well as looking at the profiles.
NOTE: In support of Rich's point for using polynomials,
Tabachnick & Fidell (6th Ed) make the same point, in fact,
calling it the "best" solution (see page 332).
-Mike Palij
New York University
[hidden email]
>Rich Ulrich
----------- Original Message ---------
On Friday, October 7, 2016 1:27 PM, Emil Rudobeck wrote:
>When it comes to reporting the findings, despite the
>shortcomings, Mauchly's test is widely used and understood.
>I haven't come across running t-tests on variances. Where
>can I read more about that approach?
I think that Rich meant paired t-tests between means at
two time points. In another post I identify a t-test for testing
whether two related variances are equal.
>The measurement is time - brain responses are sampled
>from each subject for a period of time (e.g., 72 samples
>during an hour) after a "learning" stimulus is applied.
>So change over time is expected biologically. This is
>essentially a nonlinear growth curve and I know that there
>are more advanced approaches (LMM, SEM, etc) which
>I use as well, but my concern here is with sphericity in GLM.
>Transformation is not going to address the issue, nor will
>larger N be feasible. It is possible to perhaps average
>adjacent time points, but this would introduce its own problems.
>This is a mixed design since subjects are grouped into
>different treatments and it's the differences in treatment
>that's important.
See the references I provide above.
>Your question is more about design than stats. Certainly,
>if you have any suggestions, I would be interested.
It is typical to describe the design in terms of whether one
has within-subject factors, between-subjects factors, or
both and how many levels there are. Given what you say
above, one would assume you have a 2 x 72 mixed design
with one between-subjects factor with 2 levels and one
within-subject factor with 72 levels. But I think that your
design might be a little more complicated.
>The current method is well established and has been used
>for decades. Whether the individual repeated measures
>are different or not does not matter too much in this case.
>It's more important whether the curves themselves between
>treatment groups are different (the between factor). Using
>repeated measures overcomes the issue of correlations,
>since there is no other way around it.
I'm not sure I understand the last sentence but I'd just point
out that you can graph the profile (i.e., repeated measures)
for each group and see if they are parallel or have different
curves -- the latter would be indicated by a significant
group by level of polynomial effect in the polynomial results.
-MP
From: Rich Ulrich [
[hidden email]]
Sent: Thursday, October 06, 2016 8:06 PM
To:
[hidden email]; Rudobeck, Emil (LLU)
Subject: Re: Undefined Mauchly's Test
For small and moderate samples, a non-significant Mauchly's test does
not mean much at all. That is
why many people will recommend, wisely, that followup test be performed
as paired t-tests instead of
using some pooled variance term.
What are you measuring? Is it a good measure, with good scaling
expected and no outliers observed?
I don't like analyses where those corrections are made, unless I have a
decent understanding of why
they are required, such as, the presence of excess zeroes.
Would some transformation be thought of, by anyone? Analyzing with
unnecessarily-unequal variances
is a way to get into unneeded trouble. If the "levels" represent time,
it might be appropriate and proper
to test a much more powerful hypothesis that makes use of contrasts
(linear for growth, etc.) in order to
overcome the inevitable decline in correlations across time.
You say: more levels than subjects -- Is this because you have very
small N or because you have moderate N
but also have too many levels to test a sensible hypothesis across them
all?
State your hypotheses. What tests them? A single-d.f. test is what
gives best power, whenever one of those
can be used. I favor constructing contrasts -- sometimes in the form of
separate variables -- over tests that
include multiple d.f. and multiple hypotheses, all at once. And I
would rather remove the causes of
heterogeneity (variances or correlations) beforehand, than have to hope
that I have suitably corrected for it.
--
Rich Ulrich
From: SPSSX(r) Discussion <
[hidden email]> on behalf of
Rudobeck, Emil (LLU) <
[hidden email]>
Sent: Thursday, October 6, 2016 2:20 PM
To:
[hidden email]
Subject: Undefined Mauchly's Test
Given the paucity of information online, I was wondering if anyone knows
the procedural approach to the evaluation of sphericity when Mauchly's
test is undefined, which is the case when the number of repeated levels
is larger than the number of subjects (insufficient df). I am not sure
if sphericity can still be assumed based on the reported values of
epsilon larger than 0.75, whether based on Greenhouse-Geisser or
Huynh-Feldt. In one particular dataset, epsilon is less than 0.1.
Presumably it can be assumed that sphericity is violated when epsilon is
that low.
I am aware of using mixed models to overcome the assumptions of
sphericity. My concern is with GLM in this case.
WARNING: Please be vigilant when opening emails that appear to be the
least bit out of the ordinary, e.g. someone you usually don't hear from,
or attachments you usually don't receive or didn't expect, requests to
click links or log into systems, etc. If you receive suspicious emails,
please do not open attachments or links and immediately forward the
suspicious email to
[hidden email] and then delete the suspicious
email.
CONFIDENTIALITY NOTICE: This e-mail communication and any attachments
may contain confidential and privileged information for the use of the
designated recipients named above. If you are not the intended
recipient, you are hereby notified that you have received this
communication in error and that any review, disclosure, dissemination,
distribution or copying of it or its contents is prohibited. If you have
received this communication in error, please notify me immediately by
replying to this message and destroy all copies of this communication
and any attachments. Thank you.
===================== To manage your subscription to SPSSX-L, send a
message to
[hidden email] (not to SPSSX-L), with no body text
except the command. To leave the list, send the command SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command INFO
REFCARD
===================== To manage your subscription to SPSSX-L, send a
message to
[hidden email] (not to SPSSX-L), with no body text
except the command. To leave the list, send the command SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command INFO
REFCARD
=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD