Undefined Mauchly's Test
Locked
12
12
Re: Undefined Mauchly's Test
 
|
Thanks for the detailed explanations, Mike! I think there is little reason to stick to ANOVA and have to deal with all these issues with sphericity given that mixed models overcome that requirement, along with many other advantages. The last few weeks I was busy trying to learn the basics of SAS and I must say that the implementation of proc mixed is far superior to SPSS mixed. I had a model that took over 24 hours to run in SPSS (2700+ observations, AD1), but it took just over 3 hours in SAS. I think the issue of slow mixed model calculations in SPSS was raised before: CPU usage stays rather low and it's mostly RAM that's heavily used. But I digress.
________________________________________ From: SPSSX(r) Discussion [[hidden email]] on behalf of Mike Palij [[hidden email]] Sent: Friday, October 14, 2016 1:07 PM To: [hidden email] Subject: Re: Undefined Mauchly's Test On Friday, October 14, 2016 2:50 PM, Art Kendall wrote: > >A S.W.A.G. Uh, okay. > perhaps the reason that the CS was assumed was that made > hand calculation easier. > > We did not always have calculators that did square roots, let alone > the > software that we have in this century. This would make sense if one had to compute that variance-covariance matrix in order to do a repeated measures ANOVA but, as someone who has done a repeated measures ANOVA by hand calculator, one does not need it. I have handouts with the definitional and computational formulas for one-way repeated measures ANOVA that I still use though today I don't have students do the hand computations (they used to but some can so long as well as make errors) instead I show how one can do it via Excel (with the Data Analysis toolpak: 2-way ANOVA without replication; one could go through the process of calculating the sum squares from the raw data but again this is time consuming).and then what SPSS GLM provides in addition to the simple output of Excel. I think that it was easier to assume compound symmetry because it is a simple extension of the homogeneity of variance assumption -- it is the assumption of homogeneity of variance plus homogeneity of correlation. After making these assumptions, wave your hands, and, say "Presto! Here are the Rep Meas ANOVA results!" NOTE: GLM does not even allow one to print out the variance-covariance matrix if one wanted to examine it. One has to obtain it with another procedure like correlation or reliability or one of the regression procedures and so on. MANOVA allows one to print the covariance matrix and other useful statistics. -Mike Palij New York University [hidden email] ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ________________________________ WARNING: Please be vigilant when opening emails that appear to be the least bit out of the ordinary, e.g. someone you usually don’t hear from, or attachments you usually don’t receive or didn’t expect, requests to click links or log into systems, etc. If you receive suspicious emails, please do not open attachments or links and immediately forward the suspicious email to [hidden email] and then delete the suspicious email. CONFIDENTIALITY NOTICE: This e-mail communication and any attachments may contain confidential and privileged information for the use of the designated recipients named above. If you are not the intended recipient, you are hereby notified that you have received this communication in error and that any review, disclosure, dissemination, distribution or copying of it or its contents is prohibited. If you have received this communication in error, please notify me immediately by replying to this message and destroy all copies of this communication and any attachments. Thank you. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Re: Undefined Mauchly's Test
|
|
In reply to this post by Rich Ulrich
I see what you're saying about alpha, but I don't think the approach you had in your large studies would apply here. With patient data, you can get pretty large N and achieve good
power. In bench research like ours, N is extremely limited and I have to be careful about the statistical model. So if I were to run 20 tests and do a Bonferroni correction (which is overly conservative), I'd rarely uncover any findings. On the other hand,
if I don't adjust alpha, p would not be 0.05 anymore, but more like 0.64. The study isn't fully exploratory either. Furthermore, I am not quite sure about the value of purely exploratory research. What can one conclude from such a study when alpha is so large?
Good papers address what was different and why. In my field, if the paper simply claims that something was changed, it wouldn't be important enough to warrant a publication.
I am not entirely clear why the slopes and means should be extracted from each animal and run as an ANOVA or a t-test when mixed models already do that. Based on what you said, I think the most powerful approach (in both senses of the word) would be simply to run mixed models on the linear part of the curve, which I have done in some instances. But for the LTP data, the earlier part of the curve cannot be ignored since it has a distinct biological mechanism from the later part of the curve. Keep in mind that I have talked to statisticians about this in person. Just a few months ago I had a lengthy discussion about this with three statisticians from NASA and they approved my approach, though they did try to make things even harder by suggesting a cubic spline or a non-linear mixed model as an even better approach, both of which can't be done easily in SPSS. That might be possible if/when I transition to SAS in the future, but right now, a more pressing issue is calculating the effect size for mixed models. I might resurrect the thread I had created in that regard. From: Rich Ulrich [[hidden email]]
Sent: Saturday, October 15, 2016 11:36 PM To: [hidden email]; Rudobeck, Emil (LLU) Subject: Re: Undefined Mauchly's Test (Outlook has started presenting this posts in a new way, without ">" indentation. I'm trying to find what works for Replies.) I have labeled the paragraphs below from A through E, and here are comments by paragraph.
For A. "Linear" is easy to understand. The problem with quadratic, cubic, quartic, etc., is that you seldom have just one.
So you have to look at whole plot. But I'll move on from that. I thought that you could "roll your own" models with tests in SPSS non-linear ML regression, but that wants to start with an obvious model. Which you lack.
For B(mine) and C. Yes, you want to avoid statistical acrobatics. If your study is totally exploratory, you should not be
worried about experiment-wise alpha level - you are generating hypotheses, not testing them. If you know of
similar studies or pilot data, you have expectations, some great and some small. SOMETHING justified spending
the money to collect the data. In the biggest studies I worked on, in psychiatric research of schizophrenic outpatients, the single hypothesis that justified the study was something like, "Are the rates of relapse (re-hospitalization) different?"
If "Yes", then the 20 or more rating scales provided supporting evidence of why. If "No", then the rating scales (hopefully) would supply clues as to why. In either case, I would proceed with a hierarchy of testing -- Test a composite score: If it is "significant" at 5%, then its sub-scores are legitimate to test separately at 5%, more or less, to describe why. what was otherwise showing up in the hierarchy of tests. In my own experience, the largest effects were almost always found where the PIs expected to find large effects, using the best scales, where effects had been seen before. - The journal illustration that you cite shows curves that are /fantastically/ well-separated, contrary to your description. After 5 or 10 minutes, the two groups are 3 or 4 s.e.'s apart, minute by minute, with Ns of 6+7 and 6+9. In both figures (like in your figure A), one group is asymptotic near the 100% baseline for Pre. For D. Yes, fit each animal; except that it is merely a one-way ANOVA (t-test) if you do one variable at a time and
Bonferroni-correct for having tested two variables, slope and mean. Generating the contrast for each animal gets you beyond all that concern with sphericity, etc. And it is clear from the pictures that the different slopes (if different) are not blamed on simple "regression to the mean" ... which is something to consider, whenever initial means differ.
[If I recall correctly, the BMDP2V program I mentioned before had the excellent default of computing its between-S contrasts
based on errors for slopes as actually computed within-S, in place of using the conventional decomposition of SS that is affected by sphericity.]
How you test the early, non-linear part of the curve depends on what you know about it and what you can figure out
to say about it. And that depends, probably, on what you know or suspect about the actual biology or chemistry or
physics that is taking place. My uneducated suggestion, from the pictures, would be to try an exponential decline of the excess over "zero" where the zero is modeled as the lowest value (say) of the latter part of the fitted line.
If that is possible, on the basis of single animals. For E. This is "Experimental Design", and it may go beyond "experimental design". I never took a course in that, and I don't know how much they say about "replication studies". There is always a little controversy or discussion of what comprises "separate and distinct hypotheses". When do you respect experiment-wise error, family-wise error, or single test error of 5%? Or 1%. Or whatever. When I say that the question may go beyond design, what I am thinking of is that your own area might have settled on standards for what to control. However, you still must have (I think) the power to say that THIS is what I think is important... and not THAT... The latter part of the line (say) is Main hypothesis; the early part is Exploratory. How many hypotheses are you trying to control for? How new are they? How much power do you have to spare? If a study has a bunch of hypotheses - 5? 10? - of equal merit and expectation-to-be-confirmed, are they separate and distinct hypotheses which merit a 5% test, each? Really? And not exploratory? If the pictures tell the story, your /main/ hypothesis of difference should be the latter minutes. If, for other reasons, the first 5 minutes tell the important story, then... What story is that? It might have seemed inconvenient to some people, but I thought it was fine that the protocol for our grant applications wanted us to state our hypotheses before the study started. In one case, we wrote into a grant that we intended to test one particular interaction with a 10% one-tailed test: because it was very relevant to /extending/ the narrative that we expected, but the statistical power would be too low to draw conclusions from the conventional, 2-tailed, 5% test. And a few years later, we got the editor and reviewers to accept the report of the test. It was not cherry-picking, since it was the single such test that we had laid out in advance. -- Rich Ulrich From: SPSSX(r) Discussion <[hidden email]> on behalf of Rudobeck, Emil (LLU) <[hidden email]>
Sent: Friday, October 14, 2016 11:28 AM To: [hidden email] Subject: Re: Undefined Mauchly's Test A.
I have found that cubic/quartic polynomials, along with the occasional transformation, provide a good fit with LMM - based on both visual examinations and curve fitting tests in SigmaPlot.
In some cases, non-linear mixed models would probably fit better, but SPSS wouldn't help here.
B. "The question of adjusting alpha only arises if you are assuming that all the tests are equally important, and have no hierarchy. It does appear, if those error bars are meaningful, that there is a very clear difference in the latter portion of the curves." C. Need some clarification of the above. I always assume if you're publishing a result, then it's important. Without it, this could leave the door open for all kinds of statistical acrobatics. It seems you're also advocating analyzing the later portion since the difference is there. However, here again alpha of 0.05 would be violated if one looks at the graph and analyses the part with the greatest difference. Paramount to visual statistics vs true a priori selection. The curves don't always look so nicely separated in either case: http://anesthesiology.pubs.asahq.org/data/Journals/JASA/931052/17FF5.png. That's also true for some of my datasets. D. Are you suggesting fitting a line for each individual animal and then running two-way ANOVA comparing the slopes and means between treatments groups? No intercept? And how would the early, non-linear part of the curves be compared? E. I would be rather curious about references that would allow me to skip adjustments of alpha. I have talked to several statisticians and when they had suggested breaking the graph into several parts, I specifically asked about apha and was told that an adjustment would need to be made. That's why some sort of a reference would be pretty helpful here. Maybe others can chime in. From: Rich Ulrich [[hidden email]]
Sent: Thursday, October 13, 2016 11:03 PM To: [hidden email]; Rudobeck, Emil (LLU) Subject: Re: Undefined Mauchly's Test
Right, it /looks like/ the first 10 or 12 minutes are different from the later minutes. That rather undermines the hope of fitting
a good single, 1-parameter curve to the whole.
Design? The question of adjusting alpha only arises if you are assuming that all the tests are equally important, and have no hierarchy.
It does appear, if those error bars are meaningful, that there is a very clear difference in the latter portion of the curves.
If that is a "primary and most important effect", it seems worth reporting based on its on difference in the linear trend lines,
both mean and slope. Whether the early (and different) part of the curve also differs would obviously be of interest, too, and I would feel comfortable in no-correction, no "punishment" at all.
-- Rich Ulrich
WARNING: Please be vigilant when opening emails that appear to be the least bit out of the ordinary, e.g. someone you usually don’t hear from, or attachments you usually don’t receive or didn’t expect, requests to click links or log into systems, etc. If you receive suspicious emails, please do not open attachments or links and immediately forward the suspicious email to [hidden email] and then delete the suspicious email.
CONFIDENTIALITY NOTICE: This e-mail communication and any attachments may contain confidential and privileged information for the use of the designated recipients named above. If you are not the intended recipient, you are hereby notified that you have received this communication in error and that any review, disclosure, dissemination, distribution or copying of it or its contents is prohibited. If you have received this communication in error, please notify me immediately by replying to this message and destroy all copies of this communication and any attachments. Thank you.
=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
|
|
|
On the spline issue, the STATS SPLINES extension command will generate spline variables of various types that you could use in the process. It requires the (free) R Essentials. The extension can be installed from the Utilities menu (V22-23) or the Extensions menu (V24). On Mon, Nov 21, 2016 at 12:34 PM, Rudobeck, Emil (LLU) <[hidden email]> wrote:
|
|
|
In reply to this post by Rudobeck, Emil (LLU)
No, you do not see what I'm saying about alpha. What I said had /everything/ to do with carefully figuring out your hypotheses. Re-read it. It had /everything/ to do with having an N that is only large enough for decent power when there are very few "most important" hypotheses, like one or two or maybe three. That means that your numerous other hypotheses /must/ be regarded as in a hierarchy under them in some sense; or else be presented as exploratory; or else you present garbage. Separating the expected effects into contrasts was offered for two reasons: Your initial problem with Mauchley's; and for clarity in specifying hypotheses. Computing the contrasts on each animal gives you a proper error term, which cannot be assured (I think) for the Mixed model linear effect. Your suggestions from the NASA statisticians suggests to me that these guys were working far out of their comfort zone, regardless of how good they may be in their special areas. -- Rich Ulrich |
Re: Undefined Mauchly's Test
|
|
I wouldn't be so harsh on the NASA statisticians. Two other statisticians had told me pretty much the same thing years ago (+piecewise regression), one had a PhD with his thesis
in mixed models. It's based on those recommendations that I learned mixed models in the first place. Could they all be wrong? I guess anything has some small probability. I don't know your qualifications, but consider that having an in-person conversation
with Q&A and looking at the data allows one to have a better idea about the data structure than a few short paragraphs of explanation in a forum.
As for LMM, I am not worried that these models somehow fail to incorporate the proper error terms for subjects. In fact, modeling individual slopes and intercepts is one of the strengths of LMM, in contrast to marginal models like GEE. Usage of mixed models for growth curve data has been extensively documented, along with SEMs, if the focus is on latent variables. From: Rich Ulrich [[hidden email]]
Sent: Monday, November 21, 2016 10:11 PM To: [hidden email]; Rudobeck, Emil (LLU) Subject: Re: Undefined Mauchly's Test
No, you do not see what I'm saying about alpha. What I said had /everything/ to do with carefully figuring out your hypotheses. Re-read it. It had /everything/ to do with having an N that is only large enough for decent power when there are very few "most important" hypotheses, like one or two or maybe three. That means that your numerous other hypotheses /must/ be regarded as in a hierarchy under them in some sense; or else be presented as exploratory; or else you present garbage. Separating the expected effects into contrasts was offered for two reasons: Your initial problem with Mauchley's; and for clarity in specifying hypotheses. Computing the contrasts on each animal gives you a proper error term, which cannot be assured (I think) for the Mixed model linear effect. Your suggestions from the NASA statisticians suggests to me that these guys were working far out of their comfort zone, regardless of how good they may be in their special areas. -- Rich Ulrich WARNING: Please be vigilant when opening emails that appear to be the least bit out of the ordinary, e.g. someone you usually don’t hear from, or attachments you usually don’t receive or didn’t expect, requests to click links or log into systems, etc. If you receive suspicious emails, please do not open attachments or links and immediately forward the suspicious email to [hidden email] and then delete the suspicious email.
CONFIDENTIALITY NOTICE: This e-mail communication and any attachments may contain confidential and privileged information for the use of the designated recipients named above. If you are not the intended recipient, you are hereby notified that you have received this communication in error and that any review, disclosure, dissemination, distribution or copying of it or its contents is prohibited. If you have received this communication in error, please notify me immediately by replying to this message and destroy all copies of this communication and any attachments. Thank you.
=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
|
Administrator
|
I agree that linear mixed models (aka., multilevel models) provide a very nice and flexible way to handle repeated measures. The first couple pages in Chapter 5 of the book by Heck, Thomas & Tabata (2014, 2nd Ed) have some nice commentary on this. You might be able to see some of it via Google Books if you do a Google search on this exact string:
"Repeated measures data with within-subjects and between-subjects factors" Cheers, Bruce
--
Bruce Weaver bweaver@lakeheadu.ca http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." PLEASE NOTE THE FOLLOWING: 1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. 2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/). |
Re: Undefined Mauchly's Test
|
|
Heck's book has been a very useful reference for mixed models for me. And the author himself is rather helpful when questions arise. Another book that's pretty helpful is by West:
West BT, Welch KB, Galecki AT (2014) Linear Mixed Models: A Practical Guide Using Statistical Software, Second Edition: Taylor & Francis. ________________________________________ From: SPSSX(r) Discussion [[hidden email]] on behalf of Bruce Weaver [[hidden email]] Sent: Thursday, November 24, 2016 2:42 PM To: [hidden email] Subject: Re: Undefined Mauchly's Test I agree that linear mixed models (aka., multilevel models) provide a very nice and flexible way to handle repeated measures. The first couple pages in Chapter 5 of the book by Heck, Thomas & Tabata (2014, 2nd Ed) have some nice commentary on this. You might be able to see some of it via Google Books if you do a Google search on this exact string: "Repeated measures data with within-subjects and between-subjects factors" Cheers, Bruce Rudobeck, Emil (LLU) wrote > I wouldn't be so harsh on the NASA statisticians. Two other statisticians > had told me pretty much the same thing years ago (+piecewise regression), > one had a PhD with his thesis in mixed models. It's based on those > recommendations that I learned mixed models in the first place. Could they > all be wrong? I guess anything has some small probability. I don't know > your qualifications, but consider that having an in-person conversation > with Q&A and looking at the data allows one to have a better idea about > the data structure than a few short paragraphs of explanation in a forum. > > As for LMM, I am not worried that these models somehow fail to incorporate > the proper error terms for subjects. In fact, modeling individual slopes > and intercepts is one of the strengths of LMM, in contrast to marginal > models like GEE. Usage of mixed models for growth curve data has been > extensively documented, along with SEMs, if the focus is on latent > variables. ----- -- Bruce Weaver [hidden email] http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." NOTE: My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Undefined-Mauchly-s-Test-tp5733212p5733513.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ________________________________ WARNING: Please be vigilant when opening emails that appear to be the least bit out of the ordinary, e.g. someone you usually don’t hear from, or attachments you usually don’t receive or didn’t expect, requests to click links or log into systems, etc. If you receive suspicious emails, please do not open attachments or links and immediately forward the suspicious email to [hidden email] and then delete the suspicious email. CONFIDENTIALITY NOTICE: This e-mail communication and any attachments may contain confidential and privileged information for the use of the designated recipients named above. If you are not the intended recipient, you are hereby notified that you have received this communication in error and that any review, disclosure, dissemination, distribution or copying of it or its contents is prohibited. If you have received this communication in error, please notify me immediately by replying to this message and destroy all copies of this communication and any attachments. Thank you. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
«
Return to SPSSX Discussion
|
1 view|%1 views
| Free forum by Nabble | Edit this page |