Re: Missing Value Analysis

[Feinstein, Zachary]

It's been years since I have looked into the theoretical foundations of
this...

Why are listwise and pairwise deletion methods biased?  I have used a
small variety of missing-value imputation/substitution programs and none
have worked as well as doing mean-substitutions (of course for purely
random missing data) by replacing with means based on finely defined a
priori segments.

Just curious.  Any and all correspondence is welcome.

Zachary
[hidden email]

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
SR Millis
Sent: Tuesday, June 13, 2006 10:30 AM
To: [hidden email]
Subject: Re: Missing Value Analysis

I'm not certain if SPSS has improved their Missing Value Analysis
module, but, at least in previous versions, it was my impresssion that
MVA has had a number of limitations in terms of the methods available.
Have any of these issues been addressed by SPSS?

  --Listwise and pairwise deletion methods are well known to be biased.

  --SPSS's regression imputation method uses a regression model to
impute missing values but the regression parameters are biased because
they are derived using pairwise deletion.

  --SPSS's expectation maximization (EM) method produces aymptotically
unbiased estimates but SPSS's EM implementation is limited to point
estimates (without standard errors) of means, variances, and
covariances. In addition, SPSS's EM can impute values but the values are
imputed WITHOUT residual variation---consequently the analyses that use
these imputed values can be biased.

  You may want to consider the freely available software, IVEware:
Imputation and Variance Estimation Software from the University of
Michigan:

  http://www.isr.umich.edu/src/smp/ive/


  SR Millis



Sibusiso Moyo <[hidden email]> wrote:
  Dear All,

I have a data set that has a lot of missing values for my cases/vars. So
I am considering using MVA in filling up the gaps. But the catch is that
the generated values using Expectation Maximization ought to lie between
0 and 1. So is there a way of forcing this condition onto MVA analysis
in SPSS-14?

Help always appreciated,

Sibusiso.



Scott R Millis, PhD, MEd, ABPP (CN & RP) Professor & Director of
Research Department of Physical Medicine & Rehabilitation Wayne State
University School of Medicine
261 Mack Blvd
Detroit, MI 48201
Email: [hidden email]
Tel: [phone]
Fax: [phone]

*********************************************************
This electronic message may contain information that is confidential
and/or legally privileged. It is intended only for the use of the
individual(s) and entity named as recipients in the message. If you are
not an intended recipient of this message, please notify the sender
immediately and delete the material from any computer. Do not deliver,
distribute or copy this message, and do not disclose its contents or
take any action in reliance on the information it contains. Thank you.

[SR Millis-3]

Regarding pairwise deletion: it will produce parameter estimates that are approximately unbiased in large samples IF the data a mssing completely at random (MCAR)---which doesn't occur very often in most research. I
  if the data are only missing at random (MAR), the estimates may be quite biased---the problem lies with the capacity to obtain consistent estimates of the standard errors---theoretically possible but the formulas are complicated and not implemented in any software that I'm aware of. If addition, it's not uncommon to get correlation or covariance matrices that are positive definite in small samples when using pairwise deletion.

  Listwise deletion does produce valid inferences when data are MCAR. However, it too can produce biased estimates if the data are only MAR.

  Mean substitution isn't a good idea because it reduces variance.

  SR Millis




"Feinstein, Zachary" <[hidden email]> wrote:
    It's been years since I have looked into the theoretical foundations of
this...

Why are listwise and pairwise deletion methods biased? I have used a
small variety of missing-value imputation/substitution programs and none
have worked as well as doing mean-substitutions (of course for purely
random missing data) by replacing with means based on finely defined a
priori segments.



Scott R Millis, PhD, MEd, ABPP (CN & RP)
Professor & Director of Research
Department of Physical Medicine & Rehabilitation
Wayne State University School of Medicine
261 Mack Blvd
Detroit, MI 48201
Email: [hidden email]
Tel: [phone]
Fax: [phone]

*********************************************************
This electronic message may contain information that is confidential and/or legally privileged. It is intended only for the use of the individual(s) and entity named as recipients in the message. If you are not an intended recipient of this message, please notify the sender immediately and delete the material from any computer. Do not deliver, distribute or copy this message, and do not disclose its contents or take any action in reliance on the information it contains. Thank you.