Hessian matrix not positive definite
Locked
Hessian matrix not positive definite
 
|
Hello everybody,
I want to run several completely within-subjects two-way ANOVAs with the same independent variables, but with different outcome variables each time. Because I have unbalanced data (some cells missing by design) I'm doing it using Mixed Models. However, in one of these analyses I get a warning message that the Hessian matrix in not positive definite. Aditionally, one of the random effects is zero whereas in the other analyses it is not. I have looked on the internet for possible causes. I got many hits but they only mentioned possible remedies, not any possible explanations. I figured it had to have something to do with some cells having zero variances. However, this turned out not to be the case. Does anyone know what causes this warning? I don't need any advice on possible remedies because I have already figured out that the message will not appear when leaving out the random effect that has been estimated as 0. I would like an explanation for why leaving it out resolves the problem so that I can justify this approach. Any help would be greatly appreciated! Best regards, Joost van Ginkel |
Re: Hessian matrix not positive definite
|
|
Almost surely, the report means that the determinant is zero, not full rank.
=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
A negative covariance matrix would show in inconsistent cross-product matrix. (Does Mixed allow matrix input of correlations? or have the weird Missing option, "use all available data" for separate correlations?) I do not see how a random effect that is approximately zero would have a bad effect. However, if effect is reported as *exactly* zero, that almost never happens by chance, and I would suspect that some hidden redundancy has escaped the other program diagnostics. Setting one element to zero is one way to report/ account for the redundancy, but you would see a different ANOVA if you could drop the other side of the redundancy instead. - Does the warning about the Hessian allow the program to otherwise proceed normally? - Does the analysis without the variable otherwise match the results with the zero-effect? - Are you satisfied with that result? Variables with zero variances certainly cause problems in all regression settings. "Zero variance" for a cell does not, I think. Zero-N for a cell is sometimes recognizable by a program and accounted for by reducing the d.f. where appropriate; another program could simply quit. -- Rich Ulrich > Date: Wed, 24 Jun 2015 02:27:07 -0700 > From: [hidden email] > Subject: Hessian matrix not positive definite > To: [hidden email] > > Hello everybody, > > I want to run several completely within-subjects two-way ANOVAs with the > same independent variables, but with different outcome variables each time. > Because I have unbalanced data (some cells missing by design) I'm doing it > using Mixed Models. However, in one of these analyses I get a warning > message that the Hessian matrix in not positive definite. Aditionally, one > of the random effects is zero whereas in the other analyses it is not. I > have looked on the internet for possible causes. I got many hits but they > only mentioned possible remedies, not any possible explanations. I figured > it had to have something to do with some cells having zero variances. > However, this turned out not to be the case. Does anyone know what causes > this warning? I don't need any advice on possible remedies because I have > already figured out that the message will not appear when leaving out the > random effect that has been estimated as 0. I would like an explanation for > why leaving it out resolves the problem so that I can justify this approach. > Any help would be greatly appreciated! > > Best regards, > > Joost van Ginkel > > > > -- > View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Hessian-matrix-not-positive-definite-tp5729912.html > Sent from the SPSSX Discussion mailing list archive at Nabble.com. > > ===================== > To manage your subscription to SPSSX-L, send a message to > [hidden email] (not to SPSSX-L), with no body text except the > command. To leave the list, send the command > SIGNOFF SPSSX-L > For a list of commands to manage subscriptions, send the command > INFO REFCARD |
Re: Hessian matrix not positive definite
|
|
Would a viable alternative be GLM with a type IV sum of squares method? I’ve never faced this problem but as I was reading this posting I recalled that GLM had
different types of sum-of-squares methods. Also, I think that the Hessian is second derivatives of the likelihood function, which is used in Mixed but not in GLM. Am I correct on this point? Gene Maguin From: SPSSX(r) Discussion [mailto:[hidden email]]
On Behalf Of Rich Ulrich Almost surely, the report means that the determinant is zero, not full rank.
> Date: Wed, 24 Jun 2015 02:27:07 -0700 ===================== To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
|
Re: Hessian matrix not positive definite
|
|
In reply to this post by Rich Ulrich
Dear Rich and Gene, Thank you both for your replies. When I simply remove the random effect that is estimated as (exactly) 0, the results of the other tests and parameters remain
the same. However, doesn’t leaving out the random effect for this specific predictor imply that it will become a between-subjects factor rather than a within-subjects factor? I need to model this predictor as a within-subjects factor because it has been repeatedly
measured. Best regards, Joost From: Rich Ulrich [mailto:[hidden email]]
Almost surely, the report means that the determinant is zero, not full rank.
> Date: Wed, 24 Jun 2015 02:27:07 -0700 |
|
|
Joost, The matrix of second order partial derivatives of the minimization function w.r.t. the parameters of the model is the Hessian matrix. Taking the inverse of the Hessian produces an estimate of the covariance of the parameter estimates. A non-p.d. matrix may occur in mixed models where the variance of the random effects is a component of the function to be minimized. If the variance of the random effect is zero, then there is no variance for the variance estimate, leading to a non-p.d. Hessian matrix. Ryan On Thu, Jun 25, 2015 at 3:32 AM, Ginkel, Joost van <[hidden email]> wrote:
|
| Free forum by Nabble | Edit this page |