significance of a random slope - differences between HLM and MIXED

[carmel]

Hi
I ran an a alysis of multilevel model of student nested within schools. I ran the same model in SPSS MIXED 19 and HLM 6.08 to try and see the differences between the two and check the reliability of estimation (appear in HLM but not in MIXED).
The fixed effects estimations of the coefficions were almost identical, and so was the variance componant of the intercept and residual. T he problem was in the variance componant of the radom slope - the MIXED estimation was lower and insignificant (using WALD Z test) while the HLM variance componant of the slope was higher and significant (using chi-square).
Do you know what can be the reason for the difference? maybe different calculation of the degrees of freedom? We could not find the degrees of freedom used for the WALD Z test in SPSS, nor did we found some explanation to the way the WALD Z of the random slope is calculated in SPSS (in Heck et al. book from 2010).
If you can tell us about the estimation of the degrees of freedom (in the variance componant part) and/or about the reliability of estimating random slopes when the groups are small - we will appreciate it very much

Thank you,
Carmel Blank
Tel Aviv University

[Bruce Weaver]

You'll have a much better chance of  getting some help if you post your MIXED command syntax.  There may be list members who are also familiar with HLM (I am not one of them), so you might want to post the HLM command you used too.  If it's not clear in the HLM command syntax, please indicate which estimation method was used (maximum likelihood or restricted maximum likelihood).  

HTH.

carmel wrote

Hi
I ran an a alysis of multilevel model of student nested within schools. I ran the same model in SPSS MIXED 19 and HLM 6.08 to try and see the differences between the two and check the reliability of estimation (appear in HLM but not in MIXED).
The fixed effects estimations of the coefficions were almost identical, and so was the variance componant of the intercept and residual. T he problem was in the variance componant of the radom slope - the MIXED estimation was lower and insignificant (using WALD Z test) while the HLM variance componant of the slope was higher and significant (using chi-square).
Do you know what can be the reason for the difference? maybe different calculation of the degrees of freedom? We could not find the degrees of freedom used for the WALD Z test in SPSS, nor did we found some explanation to the way the WALD Z of the random slope is calculated in SPSS (in Heck et al. book from 2010).
If you can tell us about the estimation of the degrees of freedom (in the variance componant part) and/or about the reliability of estimating random slopes when the groups are small - we will appreciate it very much

Thank you,
Carmel Blank
Tel Aviv University

[Rich Ulrich]

See below.

> Date: Thu, 3 Nov 2011 06:50:15 -0700

> From: [hidden email]
> Subject: significance of a random slope - differences between HLM and MIXED
> To: [hidden email]
>
> Hi
> I ran an a alysis of multilevel model of student nested within schools. I
> ran the same model in SPSS MIXED 19 and HLM 6.08 to try and see the
> differences between the two and check the reliability of estimation (appear
> in HLM but not in MIXED).
> The fixed effects estimations of the coefficions were almost identical, and
> so was the variance componant of the intercept and residual. T he problem
> was in the variance componant of the radom slope - the MIXED estimation was
> lower and insignificant (using WALD Z test) while the HLM variance componant
> of the slope was higher and significant (using chi-square).
> Do you know what can be the reason for the difference? maybe different

Here is a generic comment, from someone who hasn't been using
either procedure that you mention.  I can't assure you that those two
tests are even *intended*  to test the same hypothesis under the same
assumptions -- so I hope you may consider whether that is true.

One reason that the tests have different names is that they are
computed differently, from different assumptions.

A Wald test, I think, typically creates a standard error term from the
inverse of the covariance matrix.  This is often less robust than taking
a difference in the fit of two models.  If that is how this X^2 test is
defined, it is probably the better test.

--
Rich Ulrich

> calculation of the degrees of freedom? We could not find the degrees of
> freedom used for the WALD Z test in SPSS, nor did we found some explanation
> to the way the WALD Z of the random slope is calculated in SPSS (in Heck et
> al. book from 2010).
> If you can tell us about the estimation of the degrees of freedom (in the
> variance componant part) and/or about the reliability of estimating random
> slopes when the groups are small - we will appreciate it very much
>
> Thank you,
> Carmel Blank
> Tel Aviv University

[Ryan]

The first step you must take is to ensure that you are fitting the same model and employing the same estimation method. Once you've determined that the models and estimation methods are identical, then it's a matter of determining whether the specific random slope test employed is different.

I am not familiar with HLM, but if you provide your MIXED code and you tell us the model you are trying to fit [in words or equation using standard notation], I may be able to tell you whether you are in fact fitting that model in MIXED. 

Ryan

On Thu, Nov 3, 2011 at 10:25 AM, Bruce Weaver <[hidden email]> wrote:

You'll have a much better chance of  getting some help if you post your MIXED
command syntax.  There may be list members who are also familiar with HLM (I
am not one of them), so you might want to post the HLM command you used too.
If it's not clear in the HLM command syntax, please indicate which
estimation method was used (maximum likelihood or restricted maximum
likelihood).

HTH.

carmel wrote:
>
> Hi
> I ran an a alysis of multilevel model of student nested within schools. I
> ran the same model in SPSS MIXED 19 and HLM 6.08 to try and see the
> differences between the two and check the reliability of estimation
> (appear in HLM but not in MIXED).
> The fixed effects estimations of the coefficions were almost identical,
> and so was the variance componant of the intercept and residual. T he
> problem was in the variance componant of the radom slope - the MIXED
> estimation was lower and insignificant (using WALD Z test) while the HLM
> variance componant of the slope was higher and significant (using
> chi-square).
> Do you know what can be the reason for the difference? maybe different
> calculation of the degrees of freedom? We could not find the degrees of
> freedom used for the WALD Z test in SPSS, nor did we found some
> explanation to the way the WALD Z of the random slope is calculated in
> SPSS (in Heck et al. book from 2010).
> If you can tell us about the estimation of the degrees of freedom (in the
> variance componant part) and/or about the reliability of estimating random
> slopes when the groups are small - we will appreciate it very much
>
> Thank you,
> Carmel Blank
> Tel Aviv University
>

-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/significance-of-a-random-slope-differences-between-HLM-and-MIXED-tp4961104p4961199.html

Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD