2way mixed ANOVA significant interaction, but main effects not significant. What does this mean
Locked
2way mixed ANOVA significant interaction, but main effects not significant. What does this mean
 
|
Hello,
I performed a 2way mixed ANOVA with "group" (intervention and control) as the between subjects independent variable and "time" (pre-test and post-test) as the within the subjects IV. In the results I got a significant interaction between the two IVs, but main effects for either of them are not significant. how can I interpret this? Thank you. Nitya |
Re: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean
|
|
The problem that I have with that too-common test is that the
=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
test-results do not tell you what you want to know. Even if there is a big effect, you end up with all three tests as "significant" and you still have to look at the means to find out what is going on. What seems to be the natural test is to use the Pre as the covariate, while testing the difference in Post. That does have an assumption that the two groups do /not/ differ at Pre. Under randomization, that will usually be the case. If groups differ at Pre, then you do have to admit that the conditions for ideal testing do not exist. What you can fairly conclude will depend, then, on looking at the means. Similarly, if you there are differences in variance, you might need to make explanations based on knowledge of what you are measuring and what the nature of change was expected to be. (For instance, "bad scaling" could be a problem that makes variances nearly constant, either for both groups at Pre, or for the Treated group at Post.) -- Rich Ulrich > Date: Mon, 25 Apr 2016 22:39:39 -0700 > From: [hidden email] > Subject: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean > To: [hidden email] > > Hello, > > I performed a 2way mixed ANOVA with "group" (intervention and control) as > the between subjects independent variable and "time" (pre-test and > post-test) as the within the subjects IV. In the results I got a significant > interaction between the two IVs, but main effects for either of them are not > significant. how can I interpret this? > > Thank you. > > Nitya > |
|
|
In reply to this post by greykittyblues
It means you observed a sig diff on mean change scores between groups.
Homework assignment? Ryan Sent from my iPhone > On Apr 26, 2016, at 1:39 AM, greykittyblues <[hidden email]> wrote: > > Hello, > > I performed a 2way mixed ANOVA with "group" (intervention and control) as > the between subjects independent variable and "time" (pre-test and > post-test) as the within the subjects IV. In the results I got a significant > interaction between the two IVs, but main effects for either of them are not > significant. how can I interpret this? > > Thank you. > > Nitya > > > > -- > View this message in context: http://spssx-discussion.1045642.n5.nabble.com/2way-mixed-ANOVA-significant-interaction-but-main-effects-not-significant-What-does-this-mean-tp5732027.html > Sent from the SPSSX Discussion mailing list archive at Nabble.com. > > ===================== > To manage your subscription to SPSSX-L, send a message to > [hidden email] (not to SPSSX-L), with no body text except the > command. To leave the list, send the command > SIGNOFF SPSSX-L > For a list of commands to manage subscriptions, send the command > INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Re: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean
|
|
Thank you so much Ryan! Yes, its an assignment :) |
Re: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean
|
|
In reply to this post by Rich Ulrich
Thank you Rich.
|
Re: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean
|
|
In reply to this post by Rich Ulrich
Hi Rich,
That's an interesting approach to repeated measures. I wasn't aware that it's possible to treat a non-independent variable as a covariate. Could you (or anyone else who knows about this) provide some references for further reading about all the assumptions of this approach? Also wondering about the possibilities of extending such an approach to cases with more than two RMs. Thanks, Emil From: SPSSX(r) Discussion [[hidden email]] on behalf of Rich Ulrich [[hidden email]]
Sent: Monday, April 25, 2016 11:10 PM To: [hidden email] Subject: Re: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean The problem that I have with that too-common test is that the
===================== To manage your subscription to SPSSX-L, send a message to
LISTSERV@... (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
test-results do not tell you what you want to know. Even if there is a big effect, you end up with all three tests as "significant" and you still have to look at the means to find out what is going on. What seems to be the natural test is to use the Pre as the covariate, while testing the difference in Post. That does have an assumption that the two groups do /not/ differ at Pre. Under randomization, that will usually be the case. If groups differ at Pre, then you do have to admit that the conditions for ideal testing do not exist. What you can fairly conclude will depend, then, on looking at the means. Similarly, if you there are differences in variance, you might need to make explanations based on knowledge of what you are measuring and what the nature of change was expected to be. (For instance, "bad scaling" could be a problem that makes variances nearly constant, either for both groups at Pre, or for the Treated group at Post.) -- Rich Ulrich > Date: Mon, 25 Apr 2016 22:39:39 -0700
> From: [hidden email] > Subject: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean > To: [hidden email] > > Hello, > > I performed a 2way mixed ANOVA with "group" (intervention and control) as > the between subjects independent variable and "time" (pre-test and > post-test) as the within the subjects IV. In the results I got a significant > interaction between the two IVs, but main effects for either of them are not > significant. how can I interpret this? > > Thank you. > > Nitya > WARNING: Please be vigilant when opening emails that appear to be the least bit out of the ordinary, e.g. someone you usually don’t hear from, or attachments you usually don’t receive or didn’t expect, requests to click links or log into systems, etc. If you receive suspicious emails, please do not open attachments or links and immediately forward the suspicious email to [hidden email] and then delete the suspicious email.
CONFIDENTIALITY NOTICE: This e-mail communication and any attachments may contain confidential and privileged information for the use of the designated recipients named above. If you are not the intended recipient, you are hereby notified that you have received this communication in error and that any review, disclosure, dissemination, distribution or copying of it or its contents is prohibited. If you have received this communication in error, please notify me immediately by replying to this message and destroy all copies of this communication and any attachments. Thank you.
=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
|
Re: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean
Administrator
|
In reply to this post by greykittyblues
DATA LIST LIST / intType A B MeanY.
BEGIN DATA 1 1 1 -10 1 1 2 10 1 2 1 10 1 2 2 -10 2 1 1 10 2 1 2 -10 2 2 1 0 2 2 2 0 END DATA. FORMATS ALL (F2.0). * Chart Builder. GGRAPH /GRAPHDATASET NAME="graphdataset" VARIABLES=A MEAN(MeanY)[name="MEAN_MeanY"] B intType MISSING=LISTWISE REPORTMISSING=NO /GRAPHSPEC SOURCE=INLINE. BEGIN GPL SOURCE: s=userSource(id("graphdataset")) DATA: A=col(source(s), name("A"), unit.category()) DATA: MEAN_MeanY=col(source(s), name("MEAN_MeanY")) DATA: B=col(source(s), name("B"), unit.category()) DATA: intType=col(source(s), name("intType"), unit.category()) GUIDE: axis(dim(1), label("A")) GUIDE: axis(dim(2), label("Mean MeanY")) GUIDE: axis(dim(3), label("intType"), opposite()) GUIDE: legend(aesthetic(aesthetic.color.interior), label("B")) SCALE: linear(dim(2), include(0)) ELEMENT: line(position(A*MEAN_MeanY*intType), color.interior(B), missing.wings()) END GPL.
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me. --- "Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis." Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?" |
Re: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean
Administrator
|
In reply to this post by Rudobeck, Emil (LLU)
Emil, here is one short article on the use of ANCOVA for pre-post RCTs:
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1121605/ If you have access to "Biostatistics: The Bare Essentials", by Norman & Streiner (multiple editions), you could also look at their chapter on "Measuring Change". HTH.
--
Bruce Weaver bweaver@lakeheadu.ca http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." PLEASE NOTE THE FOLLOWING: 1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. 2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/). |
Re: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean
|
|
In reply to this post by Rudobeck, Emil (LLU)
There is a ton of literature on the subject of analyzing change. Unfortunately,
=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
much of it is not very well informed. On a quick Google, I found that the first answer at the URL below (the Reply referencing Senn) gives a pretty good overview, plus references. (The Reply-er endorses using ANCOVA for controlled studies.) http://stats.stackexchange.com/questions/3466/best-practice-when-analysing-pre-post-treatment-control-designs I will add: On assumptions: It is nice to have (1) equal variances everywhere, (2) equal means at Pre, and (3) a shared regression line. Given those, it is hard to fault the ANCOVA. In my experience, "unequal variances" are sometimes fixed by suitable transformation of the criterion. But the failure of the assumptions is why (at least, in my off-hand thoughts here) you might want to analyze the Outcome while ignoring Pre, or analyze the simple change score. Unequal-at-Pre raises serious logical conundrums, at times, and regression-not-to-the-shared-mean on top of unequal-at-Pre puts you into statistical complication, and controversy. The latter was the case of analyzing long-term outcome for Headstart vs. other students -- where Expected outcome with no intervention, according to other experience, would be that the lower-achieving target cases should fall further and further behind. -- Rich Ulrich Date: Wed, 27 Apr 2016 15:19:47 +0000 From: [hidden email] Subject: Re: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean To: [hidden email] Hi Rich, That's an interesting approach to repeated measures. I wasn't aware that it's possible to treat a non-independent variable as a covariate. Could you (or anyone else who knows about this) provide some references for further reading about all the assumptions of this approach? Also wondering about the possibilities of extending such an approach to cases with more than two RMs. Thanks, Emil From: SPSSX(r) Discussion [[hidden email]] on behalf of Rich Ulrich [[hidden email]]
Sent: Monday, April 25, 2016 11:10 PM To: [hidden email] Subject: Re: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean The problem that I have with that too-common test is that the test-results do not tell you what you want to know. Even if there is a big effect, you end up with all three tests as "significant" and you still have to look at the means to find out what is going on. What seems to be the natural test is to use the Pre as the covariate, while testing the difference in Post. That does have an assumption that the two groups do /not/ differ at Pre. Under randomization, that will usually be the case. If groups differ at Pre, then you do have to admit that the conditions for ideal testing do not exist. What you can fairly conclude will depend, then, on looking at the means. Similarly, if you there are differences in variance, you might need to make explanations based on knowledge of what you are measuring and what the nature of change was expected to be. (For instance, "bad scaling" could be a problem that makes variances nearly constant, either for both groups at Pre, or for the Treated group at Post.) -- Rich Ulrich > Date: Mon, 25 Apr 2016 22:39:39 -0700 > From: [hidden email] > Subject: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean > To: [hidden email] > > Hello, > > I performed a 2way mixed ANOVA with "group" (intervention and control) as > the between subjects independent variable and "time" (pre-test and > post-test) as the within the subjects IV. In the results I got a significant > interaction between the two IVs, but main effects for either of them are not > significant. how can I interpret this? > > Thank you. > > Nitya > |
Re: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean
|
Administrator
|
Another way to handle heterogeneity of variance is to estimate the model via MIXED, allowing for heterogeneous group variances. See below.
* ANCOVA with DV = post, covariate = pre estimated in the usual way. UNIANOVA post BY grp WITH pre /PRINT = parameter /EMMEANS = table(grp) /DESIGN = pre grp . * Same model via MIXED. MIXED post BY grp WITH pre /FIXED=pre grp | SSTYPE(3) /PRINT=SOLUTION TESTCOV /EMMEANS = tables(grp) . * Now include /REPEATED sub-command to * allow for heterogenous group variances. MIXED post BY grp WITH pre /FIXED=pre grp | SSTYPE(3) /PRINT=SOLUTION TESTCOV /REPEATED=grp | SUBJECT(ID) COVTYPE(DIAG) /EMMEANS = tables(grp) . HTH.
--
Bruce Weaver bweaver@lakeheadu.ca http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." PLEASE NOTE THE FOLLOWING: 1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. 2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/). |
Re: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean
|
|
Clearly, it does not address the problem of unequal values for Pre, between groups.
=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
That raises a problem of logical conclusions, even if the MIXED test works okay as a test. Clearly, it will be inferior testing compared to testing a simple, natural transformation of the measures, when that is what the data cry out for. - Those are the two big considerations that jump to my mind. It would be nice to know what situations have this as the ideal approach; and to know what other assumptions there are for this approach, and how serious it is to violate them. -- Rich Ulrich > Date: Wed, 27 Apr 2016 10:32:40 -0700 > From: [hidden email] > Subject: Re: 2way mixed ANOVA significant interaction, but main effects not significant. What does this mean > To: [hidden email] > > Another way to handle heterogeneity of variance is to estimate the model via > MIXED, allowing for heterogeneous group variances. See below. > > * ANCOVA with DV = post, covariate = pre estimated in the usual way. > > UNIANOVA post BY grp WITH pre > /PRINT = parameter > /EMMEANS = table(grp) > /DESIGN = pre grp > . > > * Same model via MIXED. > > MIXED post BY grp WITH pre > /FIXED=pre grp | SSTYPE(3) > /PRINT=SOLUTION TESTCOV > /EMMEANS = tables(grp) > . > > * Now include /REPEATED sub-command to > * allow for heterogenous group variances. > > MIXED post BY grp WITH pre > /FIXED=pre grp | SSTYPE(3) > /PRINT=SOLUTION TESTCOV > /REPEATED=grp | SUBJECT(ID) COVTYPE(DIAG) > /EMMEANS = tables(grp) > . > > HTH. > > > Rich Ulrich wrote > > There is a ton of literature on the subject of analyzing change. > > Unfortunately, > > much of it is not very well informed. On a quick Google, I found that the > > first > > answer at the URL below (the Reply referencing Senn) gives a pretty good > > overview, > > plus references. (The Reply-er endorses using ANCOVA for controlled > > studies.) > > > > http://stats.stackexchange.com/questions/3466/best-practice-when-analysing-pre-post-treatment-control-designs > > > > I will add: > > On assumptions: It is nice to have (1) equal variances everywhere, (2) > > equal means at Pre, and > > (3) a shared regression line. Given those, it is hard to fault the > > ANCOVA. In my experience, > > "unequal variances" are sometimes fixed by suitable transformation of the > > criterion. But the > > failure of the assumptions is why (at least, in my off-hand thoughts here) > > you might want to > > analyze the Outcome while ignoring Pre, or analyze the simple change > > score. Unequal-at-Pre > > raises serious logical conundrums, at times, and > > regression-not-to-the-shared-mean on top of > > unequal-at-Pre puts you into statistical complication, and controversy. > > The latter was the case > > of analyzing long-term outcome for Headstart vs. other students -- where > > Expected outcome with > > no intervention, according to other experience, would be that the > > lower-achieving target cases > > should fall further and further behind. > > > > -- > > Rich Ulrich > > > > --- snip --- > > > > > > ----- > -- > Bruce Weaver > [hidden email] > http://sites.google.com/a/lakeheadu.ca/bweaver/ > > "When all else fails, RTFM." > > NOTE: My Hotmail account is not monitored regularly. > To send me an e-mail, please use the address shown above. > > -- > View this message in context: http://spssx-discussion.1045642.n5.nabble.com/2way-mixed-ANOVA-significant-interaction-but-main-effects-not-significant-What-does-this-mean-tp5732027p5732051.html > Sent from the SPSSX Discussion mailing list archive at Nabble.com. > > ===================== > To manage your subscription to SPSSX-L, send a message to > [hidden email] (not to SPSSX-L), with no body text except the > command. To leave the list, send the command > SIGNOFF SPSSX-L > For a list of commands to manage subscriptions, send the command > INFO REFCARD |
|
|
The classic Behrens-Fisher problem was to perform a t-test where the residual variances across groups varied. SPSS provides a test of the null of equal means between independent groups by employing the Satterthwaite correction to the degrees of freedom. The model proposed by Bruce essentially applies that correction in the presence of a covariate. The model presented by Bruce does not loosen any of the assumptions of a covariate. Within the MIXED procedure. one could certainly evaluate the tenability of the assumptions of the covariate, while allowing for heterogeneous residual variances. For example, one could employ a MIXED model which incorporates the interaction between group and covariate and perform an LRT to see if the fit improves significantly, akin to Bartlett's test. Ryan On Thu, Apr 28, 2016 at 4:51 PM, Rich Ulrich <[hidden email]> wrote:
|
|
|
**Correction to last message. Disregard last post. The classic Behrens-Fisher problem was to perform a t-test where the residual variances across groups varied. SPSS provides a test of the null of equal means between independent groups by employing the Satterthwaite correction to the degrees of freedom. The model proposed by Bruce essentially applies that correction in the presence of a covariate. One could perform an LRT, analogous to Bartlett's test on the equality of residual variances, to see if permitting heterogeneous residual variances is warranted. The model presented by Bruce does not loosen any of the assumptions of a covariate. Within the MIXED procedure. one could certainly evaluate the tenability of the assumptions of the covariate, while allowing for heterogeneous residual variances. For example, one could employ a MIXED model which incorporates the interaction between group and covariate. Ryan On Fri, Apr 29, 2016 at 7:46 AM, Ryan Black <[hidden email]> wrote:
|
«
Return to SPSSX Discussion
|
1 view|%1 views
| Free forum by Nabble | Edit this page |