http://spssx-discussion.165.s1.nabble.com/Undefined-Mauchly-s-Test-tp5733212p5733267.html
(Outlook has started presenting this posts in a new way, without ">" indentation. I'm trying to find what works for Replies.)
I have labeled the paragraphs below from A through E, and here are comments by paragraph.
For A. "Linear" is easy to understand. The problem with quadratic, cubic, quartic, etc., is that you seldom have just one.
So you have to look at whole plot. But I'll move on from that. I thought that you could "roll your own" models with tests
in SPSS non-linear ML regression, but that wants to start with an obvious model. Which you lack.
For B(mine) and C. Yes, you want to avoid statistical acrobatics. If your study is totally exploratory, you should not be
worried about experiment-wise alpha level - you are generating hypotheses, not testing them. If you know of
similar studies or pilot data, you have expectations, some great and some small. SOMETHING justified spending
the money to collect the data. In the biggest studies I worked on, in psychiatric research of schizophrenic outpatients,
the single hypothesis that justified the study was something like, "Are the rates of relapse (re-hospitalization) different?"
If "Yes", then the 20 or more rating scales provided supporting evidence of why. If "No", then the rating scales (hopefully)
would supply clues as to why. In either case, I would proceed with a hierarchy of testing -- Test a composite score: If it
is "significant" at 5%, then its sub-scores are legitimate to test separately at 5%, more or less, to describe why.
Scales that were included for exploratory purposes were explicitly recognized as such, even if they turned out to support
what was otherwise showing up in the hierarchy of tests.
In my own experience, the largest effects were almost always found where the PIs expected to find large effects, using
the best scales, where effects had been seen before. - The journal illustration that you cite shows curves that are
/fantastically/ well-separated, contrary to your description. After 5 or 10 minutes, the two groups are 3 or 4 s.e.'s apart,
minute by minute, with Ns of 6+7 and 6+9. In both figures (like in your figure A), one group is asymptotic near the 100% baseline
for Pre.
For D. Yes, fit each animal; except that it is merely a one-way ANOVA (t-test) if you do one variable at a time and
Bonferroni-correct for having tested two variables, slope and mean. Generating the contrast for each animal gets
you beyond all that concern with sphericity, etc. And it is clear from the pictures that the different slopes (if different)
are not blamed on simple "regression to the mean" ... which is something to consider, whenever initial means differ.
[If I recall correctly, the BMDP2V program I mentioned before had the excellent default of computing its between-S contrasts
based on errors for slopes as actually computed within-S, in place of using the conventional decomposition of SS that is
affected by sphericity.]
How you test the early, non-linear part of the curve depends on what you know about it and what you can figure out
to say about it. And that depends, probably, on what you know or suspect about the actual biology or chemistry or
physics that is taking place. My uneducated suggestion, from the pictures, would be to try an exponential decline
of the excess over "zero" where the zero is modeled as the lowest value (say) of the latter part of the fitted line.
If that is possible, on the basis of single animals.
For E. This is "Experimental Design", and it may go beyond "experimental design". I never took a course in that,
and I don't know how much they say about "replication studies". There is always a little controversy or discussion
of what comprises "separate and distinct hypotheses". When do you respect experiment-wise error, family-wise
error, or single test error of 5%? Or 1%. Or whatever. When I say that the question may go beyond design, what
I am thinking of is that your own area might have settled on standards for what to control. However, you still
must have (I think) the power to say that THIS is what I think is important... and not THAT... The latter part of
the line (say) is Main hypothesis; the early part is Exploratory.
How many hypotheses are you trying to control for? How new are they? How much power do you have to spare?
If a study has a bunch of hypotheses - 5? 10? - of equal merit and expectation-to-be-confirmed, are they
separate and distinct hypotheses which merit a 5% test, each? Really? And not exploratory?
If the pictures tell the story, your /main/ hypothesis of difference should be the latter minutes.
If, for other reasons, the first 5 minutes tell the important story, then... What story is that?
It might have seemed inconvenient to some people, but I thought it was fine that the protocol for our grant
applications wanted us to state our hypotheses before the study started. In one case, we wrote into a grant
that we intended to test one particular interaction with a 10% one-tailed test: because it was very relevant to
/extending/ the narrative that we expected, but the statistical power would be too low to draw conclusions
from the conventional, 2-tailed, 5% test. And a few years later, we got the editor and reviewers to accept the
report of the test. It was not cherry-picking, since it was the single such test that we had laid out in advance.
--
Rich Ulrich
I have found that cubic/quartic polynomials, along with the occasional transformation, provide a good fit with LMM - based on both visual examinations and curve fitting tests in SigmaPlot.
In some cases, non-linear mixed models would probably fit better, but SPSS wouldn't help here.
B.
"The question of adjusting alpha only arises if you are assuming that all the tests are equally important, and have no hierarchy. It does appear, if those error bars are meaningful, that there is a very clear difference in the latter portion of the curves."
C.
Need some clarification of the above. I always assume if you're publishing a result, then it's important. Without it, this could leave the door open for all kinds of statistical acrobatics. It seems you're also advocating analyzing the later portion since the
difference is there. However, here again alpha of 0.05 would be violated if one
looks at the graph and analyses the part with the greatest difference. Paramount to visual statistics vs true
a priori selection. The curves don't always look so nicely separated in either case:
http://anesthesiology.pubs.asahq.org/data/Journals/JASA/931052/17FF5.png. That's also true for some of my datasets.
D.
Are you suggesting fitting a line for each individual animal and then running two-way ANOVA comparing the slopes and means between treatments groups? No intercept? And how would the early, non-linear part of the curves be compared?
E.
I would be rather curious about references that would allow me to skip adjustments of alpha. I have talked to several statisticians and when they had suggested breaking the graph into several parts, I specifically asked about apha and was told that an adjustment
would need to be made. That's why some sort of a reference would be pretty helpful here. Maybe others can chime in.
From: Rich Ulrich [[hidden email]]
Sent: Thursday, October 13, 2016 11:03 PM
To: [hidden email]; Rudobeck, Emil (LLU)
Subject: Re: Undefined Mauchly's Test
Right, it /looks like/ the first 10 or 12 minutes are different from the later minutes. That rather undermines the hope of fitting
a good single, 1-parameter curve to the whole.
Design?
The question of adjusting alpha only arises if you are assuming that all the tests are equally important, and have no hierarchy.
It does appear, if those error bars are meaningful, that there is a very clear difference in the latter portion of the curves.
If that is a "primary and most important effect", it seems worth reporting based on its on difference in the linear trend lines,
both mean and slope. Whether the early (and different) part of the curve also differs would obviously be of interest, too, and
I would feel comfortable in no-correction, no "punishment" at all.
--
Rich Ulrich
"While the biological mechanisms are different for the early vs late response, no strict cutoff has been established. I could choose an approximate cutoff and divide the curve into 2 or 3 pieces. I think this would require spline analysis, which SPSS can’t
do easily. Furthermore, alpha would need to be further adjusted for each additional piece that’s created and I think this “punishment” could be rather severe."
=====================
To manage your subscription to SPSSX-L, send a message to
(not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD