Options for Dealing with Missing Data in Mixed ANOVA
Locked
 
|
Hi,
I am looking to assess the effect of an intervention (vs control) on a number of outcomes at 4-time points. I have one between-groups IV (group) and one within-groups IV (time) and many continuous outcomes (DV) and am therefore looking to do a Mixed ANOVA. I know a linear mixed effects model is better at dealing with missing data, but I think this is beyond my ability at this point in time. I have also read a lot that suggests carrying forward the last observation is never a good idea and can introduce more bias. I am going to check to make sure the missing data points are not correlated to demographic variables as well. Is carrying forward the last observation (LOCF) for missing time points suitable in mixed ANOVA or is there another way I could be dealing with this - other than perhaps running the mixed ANOVA on both the original dataset and the LOCF dataset? Thank you for your help Brooklyn |
Re: Options for Dealing with Missing Data in Mixed ANOVA
|
|
Dear Brooklyn,
The best way to deal with missing data is Multiple imputation (in SPSS: Analyze, Multiple Imputation, Impute Missing Data Values). The procedure creates several plausible complete versions of the incomplete dataset. Each of these complete versions are
then analyzed with the statistical analysis of interest (a Mixed ANOVA in your case), yielding several different outcomes of your analysis. In the final step these analyses are pooled into one overall analysis. If you analyze a multiply imputed dataset in SPSS,
SPSS automatically recognizes this as a multiple imputation dataset and pools the results of the analyses for you, that is, for some statistical procedures. For ANOVA though, SPSS does no pooling, unfortunately. However, the following paper deals with this
problem:
Van Ginkel, J.R. & Kroonenberg, P.M (2014). Analysis of variance of multiply imputed data. Multivariate Behavioral Research, 49, 78–91.
The paper comes with an SPSS macro, called MI-MUL2.sps, which can be downloaded from:
If you need any help with this, please feel free to contact me.
Best regards,
Joost van Ginkel
-----Original Message-----
From: SPSSX(r) Discussion [[hidden email]] On Behalf Of Brooklyn Sent: Tuesday, February 21, 2017 4:53 AM To: [hidden email] Subject: Options for Dealing with Missing Data in Mixed ANOVA Hi,
I am looking to assess the effect of an intervention (vs control) on a number of outcomes at 4-time points. I have one between-groups IV (group) and one within-groups IV (time) and many continuous outcomes (DV) and am therefore looking to do a Mixed ANOVA.
I know a linear mixed effects model is better at dealing with missing data, but I think this is beyond my ability at this point in time. I have also read a lot that suggests carrying forward the last observation is never a good idea and can introduce
more bias. I am going to check to make sure the missing data points are not correlated to demographic variables as well.
Is carrying forward the last observation (LOCF) for missing time points suitable in mixed ANOVA or is there another way I could be dealing with this
- other than perhaps running the mixed ANOVA on both the original dataset and the LOCF dataset?
Thank you for your help
Brooklyn
--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Options-for-Dealing-with-Missing-Data-in-Mixed-ANOVA-tp5733847.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.
=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of
commands to manage subscriptions, send the command INFO REFCARD
|
Re: Options for Dealing with Missing Data in Mixed ANOVA
|
|
In reply to this post by Ecushla
What you /properly/ do about missing data has to take into account several factors, with
the easiest being, "Why is it missing?" and "How much are we talking about?" (compared to how much total data). In a design with four time points, you /might/ be talking -- partly or entirely -- about people who dropped out ... which could be "at random" or due to "treatment
failure." Also, "Do Intervention and Control show the same amount and have the same reasons?"
- A small amount of "missing", distributed equally between Intervention and Control, seemingly "missing at random", is the easy case to cope with, since the choice of solution will not make much difference.
- Carrying forward the last observation is only reasonable when that score /does/ represent what you would expect their status to be at that time. And when there is not very much of it. Hope this helps. -- Rich Ulrich From: SPSSX(r) Discussion <[hidden email]> on behalf of Brooklyn <[hidden email]>
Sent: Monday, February 20, 2017 10:52:38 PM To: [hidden email] Subject: Options for Dealing with Missing Data in Mixed ANOVA Hi,
=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
I am looking to assess the effect of an intervention (vs control) on a number of outcomes at 4-time points. I have one between-groups IV (group) and one within-groups IV (time) and many continuous outcomes (DV) and am therefore looking to do a Mixed ANOVA. I know a linear mixed effects model is better at dealing with missing data, but I think this is beyond my ability at this point in time. I have also read a lot that suggests carrying forward the last observation is never a good idea and can introduce more bias. I am going to check to make sure the missing data points are not correlated to demographic variables as well. Is carrying forward the last observation (LOCF) for missing time points suitable in mixed ANOVA or is there another way I could be dealing with this - other than perhaps running the mixed ANOVA on both the original dataset and the LOCF dataset? Thank you for your help Brooklyn -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Options-for-Dealing-with-Missing-Data-in-Mixed-ANOVA-tp5733847.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Re: Options for Dealing with Missing Data in Mixed ANOVA
Administrator
|
In reply to this post by Ecushla
You say that using the MIXED procedure is beyond your ability at this time. I think you are giving up far too easily on that approach. It sounds like your analysis is structurally identical to one of the examples on this UCLA page:
http://www.ats.ucla.edu/stat/spss/seminars/Repeated_Measures/ Scroll down to "Exercise example, model 2 using MIXED Command". In that example, Exercise Type is a between-Ss factor and Time a repeated measures factor. So you have a good model of what your MIXED syntax should look like. Re LOCF, you might find David Streiner's short note on it useful. http://ebmh.bmj.com/content/11/1/3.2 HTH.
--
Bruce Weaver bweaver@lakeheadu.ca http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." PLEASE NOTE THE FOLLOWING: 1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. 2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/). |
|
|
In reply to this post by Rich Ulrich
Thanks Rich, I am looking into the reasons for the missing data at the moment. Most is loss of patient contact, and I am unable to get a reason, however there are a couple of cases where the reason for dropout is that they were not happy with the intervention. I think you have confirmed that carrying forward the last observation is not the best approach. Kind regards Ecushla |
|
|
In reply to this post by Bruce Weaver
Thanks Bruce, The Uni statistician strongly advised against me trying to learn the linear mixed effects model without attending a proper course or having a solid regression knowledge. It sounds like it is the best statistical approach though. I might look into some online videos etc and see if I can work it out! Thanks for the heads up that it might be doable! I thought the mixed effects model handles the missing data by using all available data (i.e. not dropping cases) and therefore I would not need to 'add/substitute/impute missing data'. However I was told " While a linear mixed effects model handles missing data better than other statistical methods, it cannot be used as a substitute for imputation. With longitudinal data, you would first carry the last observation forward and then reshape the data (into long format) and run your mixed models" which seems contradictory. Am I missing a basic point? |
Re: Options for Dealing with Missing Data in Mixed ANOVA
|
Administrator
|
MIXED requires a long file structure (i.e., one row per repeated measure, so multiple rows per ID), and does indeed use all available data for each subject. If I remember correctly, it assumes that the missing data are at worst "missing at random" (MAR).
In my view, LOCF is a very poor way to deal with missing data--see the Streiner article I posted earlier. If you are going to use some kind of imputation, use multiple imputation instead. HTH.
--
Bruce Weaver bweaver@lakeheadu.ca http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." PLEASE NOTE THE FOLLOWING: 1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. 2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/). |
|
|
Yes, I have come to the conclusion NOT to do LOCF. So am I correct in my understanding that I could 1) do mixed ANOVA with some sort of multiple imputation (to input a value for the missing data, as per J.R. Van Ginkels post and publication) OR 2) run the linear mixed effects model on my raw data (I do not need to do any imputation to account for the missing data, the mixed effects model deals with this). Both are suitable options? On Wed, Feb 22, 2017 at 1:09 PM, Bruce Weaver [via SPSSX Discussion] <[hidden email]> wrote: MIXED requires a long file structure (i.e., one row per repeated measure, so multiple rows per ID), and does indeed use all available data for each subject. If I remember correctly, it assumes that the missing data are at worst "missing at random" (MAR). |
Re: Options for Dealing with Missing Data in Mixed ANOVA
|
|
In reply to this post by Ecushla
one-size-fits-all." I don't know enough about your data to know that it is not best for your data. By the way, as to another question: Your respondent does not seem to be distinguishing clearly between "mixed model" and "SPSS MIXED" procedure. Fixed effects. Random effects. Some of each gives you Mixed model. An analysis that ignores Missing is an "unbalanced blocks" design -- to give one technical name for it. I have had data where there was varying lengths of followup where I found it instructive to look separately at cases according to how many followups existed -- this was out of a dozen periods. We noticed, for instance, that cases who Relapsed (thus, "dropping out") showed an increase of symptoms at the both the last period and the one just before it. That was true for both drug-treated and control. - That would be impossible to spot with any analysis I've seen mentioned in this discussion... which has generally assumed few dropouts. And you have only four assessments. I will repeat something I mentioned before: "How much Missing is there?" If there is much of it, you will not have a satisfactory analysis until you do quite a bit to characterize the Missing; it is possible that the Missing could rule out any definitive conclusion /despite/ however-large the apparent effects may be. ("Really effective treatment of the target-effects by the end, but half the Treated group withdrew owing to serious side effects.") -- Rich Ulrich From: SPSSX(r) Discussion <[hidden email]> on behalf of Ecushla <[hidden email]>
Sent: Tuesday, February 21, 2017 9:07 PM To: [hidden email] Subject: Re: Options for Dealing with Missing Data in Mixed ANOVA Rich Ulrich wrote
> What you /properly/ do about missing data has to take into account several > factors, with > > the easiest being, "Why is it missing?" and "How much are we talking > about?" (compared to > > how much total data). > > - Carrying forward the last observation is only reasonable when that > score /does/ represent > > what you would expect their status to be at that time. And when there is > not very much of it. Thanks Rich, I am looking into the reasons for the missing data at the moment. Most is loss of patient contact, and I am unable to get a reason, however there are a couple of cases where the reason for dropout is that they were not happy with the intervention. I think you have confirmed that carrying forward the last observation is not the best approach. Kind regards Ecushla |
Re: Options for Dealing with Missing Data in Mixed ANOVA
|
|
In reply to this post by Bruce Weaver
I don't know the specific examples, but an advantage of Multiple imputation over Multilevel/MIXED is that Multiple imputation can also take into account background variables in handling the missing data, which don't have to be included in the subsequent analysis. My 2014 paper also addresses this issue. By the way, should you want to do a repeated measures ANOVA on multiply imputed datasets you end up in the MIXED procedure anyway because the only way to pool the results of repeated measures ANOVA using my SPSS macro is to carry out the analysis in MIXED. The reason is that GLM doesn't provide covariance matrices of parameter estimates, which are necessary for pooling the results.
-----Original Message----- From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Bruce Weaver Sent: Tuesday, February 21, 2017 7:12 PM To: [hidden email] Subject: Re: Options for Dealing with Missing Data in Mixed ANOVA You say that using the MIXED procedure is beyond your ability at this time. I think you are giving up far too easily on that approach. It sounds like your analysis is structurally identical to one of the examples on this UCLA page: http://www.ats.ucla.edu/stat/spss/seminars/Repeated_Measures/ Scroll down to "Exercise example, model 2 using MIXED Command". In that example, Exercise Type is a between-Ss factor and Time a repeated measures factor. So you have a good model of what your MIXED syntax should look like. Re LOCF, you might find David Streiner's short note on it useful. http://ebmh.bmj.com/content/11/1/3.2 HTH. Brooklyn wrote > Hi, > > I am looking to assess the effect of an intervention (vs control) on a > number of outcomes at 4-time points. I have one between-groups IV > (group) and one within-groups IV (time) and many continuous outcomes > (DV) and am therefore looking to do a Mixed ANOVA. > > I know a linear mixed effects model is better at dealing with missing > data, but I think this is beyond my ability at this point in time. I > have also read a lot that suggests carrying forward the last > observation is never a good idea and can introduce more bias. I am > going to check to make sure the missing data points are not correlated > to demographic variables as well. > > Is carrying forward the last observation (LOCF) for missing time > points suitable in mixed ANOVA or is there another way I could be > dealing with this - other than perhaps running the mixed ANOVA on both > the original dataset and the LOCF dataset? > > Thank you for your help > Brooklyn ----- -- Bruce Weaver [hidden email] http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." NOTE: My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Options-for-Dealing-with-Missing-Data-in-Mixed-ANOVA-tp5733847p5733855.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
|
In reply to this post by Ecushla
While it is true that the way the data set is structured to perform a linear mixed model (LMM) allows one to use "all possible data", that is certainly not the only reason to use LMM. Moreover, linear mixed modeling can be used in conjunction with multiple imputation of missing data. Generally speaking, LMM has more flexibility in the specification of random and residual effects structures than general linear models. It is also worth noting that LMM estimates parameters using maximum likelihood methods. Ryan On Tue, Feb 21, 2017 at 9:54 PM, Ecushla <[hidden email]> wrote:
|
«
Return to SPSSX Discussion
|
1 view|%1 views
| Free forum by Nabble | Edit this page |