Error in Multiple Imputation PASW 17?

[Joost van Ginkel]

I have tried the Multiple imputation procedure in PASW 17 for analyzing a data set with missing values. The procedure is quite user-friendly. However, when using the procedure I discovered some strange things about the pooled results in the output. The pooled significance tests from a regression model sometimes have p-values that are exactly equal to 1. I find it hard to believe that these results are correct since the p-values of the 5 imputed datasets vary around 0.20. I can imagine that SE's increase due to the extra uncertainty because of the missing data, but I can't imagine that this p-value increases exactly to one. Thus far, I have only seen this occur within the procedure Mixed models so I don't know if it occurs in other procedures as well. For people who want to see what I mean, I can send the data and the syntax file that generated these results, on request.

Best regards,

Joost van Ginkel

**********************************************************************

This email and any files transmitted with it are confidential and

intended solely for the use of the individual or entity to whom they

are addressed. If you have received this email in error please notify

the system manager.

**********************************************************************

 

[Henrik Lolle]

I discovered the same kind of results with Multiple imputation in
ordinary GLM. I have reported it to SPSS but haven't received any
explanation yet.

Best,
Henrik

Quoting "Ginkel, Joost van" <[hidden email]>:

> I have tried the Multiple imputation procedure in PASW 17 for analyzing
> a data set with missing values. The procedure is quite user-friendly.
> However, when using the procedure I discovered some strange things about
> the pooled results in the output. The pooled significance tests from a
> regression model sometimes have p-values that are exactly equal to 1. I
> find it hard to believe that these results are correct since the
> p-values of the 5 imputed datasets vary around 0.20. I can imagine that
> SE's increase due to the extra uncertainty because of the missing data,
> but I can't imagine that this p-value increases exactly to one. Thus
> far, I have only seen this occur within the procedure Mixed models so I
> don't know if it occurs in other procedures as well. For people who want
> to see what I mean, I can send the data and the syntax file that
> generated these results, on request.
>
> Best regards,
>
> Joost van Ginkel
>
> **********************************************************************
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please notify
> the system manager.
> **********************************************************************
>
>

************************************************************
Henrik Lolle
Department of Economics, Politics and Public Administration
Aalborg University
Fibigerstraede 1
9200 Aalborg
Phone: [phone]
************************************************************

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[SPSS Support]

Hi Joost,

 

I can't find any evidence of there being a known problem like this. I'll be happy to investigate things.

 

David Nichols

---

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Ginkel, Joost van
Sent: Wednesday, July 01, 2009 4:02 AM
To: [hidden email]
Subject: [SPSSX-L] Error in Multiple Imputation PASW 17?

I have tried the Multiple imputation procedure in PASW 17 for analyzing a data set with missing values. The procedure is quite user-friendly. However, when using the procedure I discovered some strange things about the pooled results in the output. The pooled significance tests from a regression model sometimes have p-values that are exactly equal to 1. I find it hard to believe that these results are correct since the p-values of the 5 imputed datasets vary around 0.20. I can imagine that SE's increase due to the extra uncertainty because of the missing data, but I can't imagine that this p-value increases exactly to one. Thus far, I have only seen this occur within the procedure Mixed models so I don't know if it occurs in other procedures as well. For people who want to see what I mean, I can send the data and the syntax file that generated these results, on request.

Best regards,

Joost van Ginkel

**********************************************************************

This email and any files transmitted with it are confidential and

intended solely for the use of the individual or entity to whom they

are addressed. If you have received this email in error please notify

the system manager.

**********************************************************************

 

[SPSS Support]

Hi Henrik,

I can't find evidence of our knowing about this in Support. I'll be glad to follow up on it.

David Nichols

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Henrik Lolle
Sent: Wednesday, July 01, 2009 6:49 AM
To: [hidden email]
Subject: Re: [SPSSX-L] Error in Multiple Imputation PASW 17?

I discovered the same kind of results with Multiple imputation in ordinary GLM. I have reported it to SPSS but haven't received any explanation yet.

Best,
Henrik

Quoting "Ginkel, Joost van" <[hidden email]>:

> I have tried the Multiple imputation procedure in PASW 17 for
> analyzing a data set with missing values. The procedure is quite user-friendly.
> However, when using the procedure I discovered some strange things
> about the pooled results in the output. The pooled significance tests
> from a regression model sometimes have p-values that are exactly equal
> to 1. I find it hard to believe that these results are correct since
> the p-values of the 5 imputed datasets vary around 0.20. I can imagine
> that SE's increase due to the extra uncertainty because of the missing
> data, but I can't imagine that this p-value increases exactly to one.
> Thus far, I have only seen this occur within the procedure Mixed
> models so I don't know if it occurs in other procedures as well. For
> people who want to see what I mean, I can send the data and the syntax
> file that generated these results, on request.
>
> Best regards,
>
> Joost van Ginkel
>
> **********************************************************************
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please notify
> the system manager.
> **********************************************************************
>
>

************************************************************
Henrik Lolle
Department of Economics, Politics and Public Administration Aalborg University Fibigerstraede 1 9200 Aalborg
Phone: [phone]
************************************************************

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[SPSS Support]

As a general FYI, Henrik provided me the necessary information and I've confirmed that there are problems with the pooled results when you have situations where there is no between-imputation variance, and a bug report has been filed.

David Nichols

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of SPSS Support
Sent: Wednesday, July 01, 2009 4:12 PM
To: [hidden email]
Subject: Re: [SPSSX-L] Error in Multiple Imputation PASW 17?

Hi Henrik,

I can't find evidence of our knowing about this in Support. I'll be glad to follow up on it.

David Nichols

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Henrik Lolle
Sent: Wednesday, July 01, 2009 6:49 AM
To: [hidden email]
Subject: Re: [SPSSX-L] Error in Multiple Imputation PASW 17?

I discovered the same kind of results with Multiple imputation in ordinary GLM. I have reported it to SPSS but haven't received any explanation yet.

Best,
Henrik

Quoting "Ginkel, Joost van" <[hidden email]>:

> I have tried the Multiple imputation procedure in PASW 17 for
> analyzing a data set with missing values. The procedure is quite user-friendly.
> However, when using the procedure I discovered some strange things
> about the pooled results in the output. The pooled significance tests
> from a regression model sometimes have p-values that are exactly equal
> to 1. I find it hard to believe that these results are correct since
> the p-values of the 5 imputed datasets vary around 0.20. I can imagine
> that SE's increase due to the extra uncertainty because of the missing
> data, but I can't imagine that this p-value increases exactly to one.
> Thus far, I have only seen this occur within the procedure Mixed
> models so I don't know if it occurs in other procedures as well. For
> people who want to see what I mean, I can send the data and the syntax
> file that generated these results, on request.
>
> Best regards,
>
> Joost van Ginkel
>
> **********************************************************************
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please notify
> the system manager.
> **********************************************************************
>
>

************************************************************
Henrik Lolle
Department of Economics, Politics and Public Administration Aalborg University Fibigerstraede 1 9200 Aalborg
Phone: [phone]
************************************************************

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD