Can we test a multilevel moderation effect without significant τ of the unconditional model?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Can we test a multilevel moderation effect without significant τ of the unconditional model?

凌楚定

Hello list,

 

Is there anyone here who is expert in HLM? I have a basic question:

 

I am attempting to test a multilevel moderation model, in which the independent and dependent variables are at level 1, while the moderator is at level 2. According to what I learned from the textbook previously, I should run the unconditional model firstly to examine whether the variance component (τ) of the dependent variable’s slope is significant. Unfortunately, it was not significant. It seems that it is not necessary for me to test further. However, I am curious about the result so I did the multilevel moderation test and found it was significant.

 

Now here comes my question: Maybe it is inconsistent with the principle of HLM to test such a moderation test, but how to explain the significant moderation effects since I got the statistic results?

 

I would be grateful if any of you can give me some hopeful suggestions. 

 

Chu-Ding LING
Ph.D. Student of Business Administration
School of Management
Zhejiang University
Reply | Threaded
Open this post in threaded view
|

Re: Can we test a multilevel moderation effect without significant τ of the unconditional model?

Bruce Weaver
Administrator
I don't profess extraordinary expertise in HLM, but I'll jump in with a question and a comment.

Which book are you looking at, and what exactly does it say?  

Now the comment:  Given that you appear to have an a priori interest in that particular cross-level interaction, my gut feeling is that you should go ahead and look at it regardless of significance (or not) of the variance of the random slopes.  I see a parallel here to going ahead with planned orthogonal contrasts (in the ANOVA context) regardless of the significance of the omnibus F-test.  

There...maybe that will provoke the real HLM experts (Ryan, Alex...) to jump in with their comments.  ;-)


凌楚定 wrote
Hello list,
 
Is there anyone here who is expert in HLM? I have a basic question:
 
I am attempting to test a multilevel moderation model, in which the independent and dependent variables are at level 1, while the moderator is at level 2. According to what I learned from the textbook previously, I should run the unconditional model firstly to examine whether the variance component (τ) of the dependent variable’s slope is significant. Unfortunately, it was not significant. It seems that it is not necessary for me to test further. However, I am curious about the result so I did the multilevel moderation test and found it was significant.
 
Now here comes my question: Maybe it is inconsistent with the principle of HLM to test such a moderation test, but how to explain the significant moderation effects since I got the statistic results?
 
I would be grateful if any of you can give me some hopeful suggestions.
 


Chu-Ding LING
Ph.D. Student of Business Administration
School of Management
Zhejiang University
--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

PLEASE NOTE THE FOLLOWING: 
1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above.
2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/).
Reply | Threaded
Open this post in threaded view
|

Re: Can we test a multilevel moderation effect without significant τ of the unconditional model?

Ryan
If there is apriori theory that a cross-level interaction exists between a level-1 variable and level-2 variable, then it is perfectly reasonable to immediately fit a model which includes the interaction.

That said, since the OP couldn't resist fitting both models (I'm shocked! ;-)), we might as well discuss the probable reasons for the results.

The scenario presented by the OP can likely be explained by:

1. A Type I error has been committed on the test of interaction (less likely)

or

2. A Type II error has been committed on the test of slope in the model which excluded the interaction (more likely).

Ryan

p.s. I'm fairly certain this is discussed in one of the widely used and cited multilevel textbooks but I can't recall which one at the moment.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Can we test a multilevel moderation effect without significant τ of the unconditional model?

Ryan
By the way, whenever posting a statistics question to SPSS-L, it is advisable to link it to SPSS in some way. Providing the MIXED syntax for both models would have been a great way to make this question related to SPSS. Moreover, including the syntax would help others learn how to fit such models in SPSS.

Ryan

> On May 18, 2014, at 2:15 PM, GMAIL <[hidden email]> wrote:
>
> If there is apriori theory that a cross-level interaction exists between a level-1 variable and level-2 variable, then it is perfectly reasonable to immediately fit a model which includes the interaction.
>
> That said, since the OP couldn't resist fitting both models (I'm shocked! ;-)), we might as well discuss the probable reasons for the results.
>
> The scenario presented by the OP can likely be explained by:
>
> 1. A Type I error has been committed on the test of interaction (less likely)
>
> or
>
> 2. A Type II error has been committed on the test of slope in the model which excluded the interaction (more likely).
>
> Ryan
>
> p.s. I'm fairly certain this is discussed in one of the widely used and cited multilevel textbooks but I can't recall which one at the moment.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: [mpluser] Can we test a multilevel moderation effect without significant τ of the unconditional model?

凌楚定
In reply to this post by 凌楚定

Sorry for the delayed reply. Thanks for all of your helpful suggestions! Especially I would like to thank Bruce; it is so kind of you to response timely every time. 




Chu-Ding Ling


2014-05-28 9:25 GMT+08:00 凌楚定 <[hidden email]>:
抱歉,回复迟了。问题已解决,谢谢您!





Chu-Ding Ling


2014-05-19 18:28 GMT+08:00 ya <[hidden email]>:

哈喽阿,
 
你的意思是,
 
1. 你的unconditional model是指的null model还是带level-1 covariate的model?
一般情况下,你需要先做一个null model,看level-2是否有显著影响,说白了,就是先检验是否有必要考虑multilevel structure,通过这个模型,你可以得到ICC。然后,根据你做的领域里大家一般认可的一个值的大小来决定ICC是否足够大,如果够大了,就需要考虑multilevel。
 
2. 在null model的基础上,你才开始纳入level-1的covariate,有了covariate,才会有slope的问题。然后接着是level-2的covariate,也就是你的moderator。你后来得到的显著的结果,说明的是moderator的slope显著,所以调节作用应该是显著的。而且这个跟你null model给出的level-2是否有显著影响应该是不矛盾的。所以,貌似是你的第一个分析出了问题?
 
3. 关于这方面的内容,中文的书可以看张雷的那本多层线性回归。英文的资料可以google Kristopher Preacher.
 
4. 提问的话,最好是贴上你采用的程序(或相应的报错),这样大家比较容易看出存在的问题来:)
 
good luck
 
 

ya
 
发件人: [hidden email]
发送时间: <a href="tel:2014-05-18%C2%A007" value="+862014051807" target="_blank">2014-05-18 07:42
收件人: [hidden email]
主题: [mpluser] Can we test a multilevel moderation effect without significant τ of the unconditional model?

Hello list,

 

Is there anyone here who is expert in HLM? I have a basic question:

 

I am attempting to test a multilevel moderation model, in which the independent and dependent variables are at level 1, while the moderator is at level 2. According to what I learned from the textbook previously, I should run the unconditional model firstly to examine whether the variance component (τ) of the dependent variable’s slope is significant. Unfortunately, it was not significant. It seems that it is not necessary for me to test further. However, I am curious about the result so I did the multilevel moderation test and found it was significant.

 

Now here comes my question: Maybe it is inconsistent with the principle of HLM to test such a moderation test, but how to explain the significant moderation effects since I got the statistic results?

 

I would be grateful if any of you can give me some hopeful suggestions. 

 

Chu-Ding LING
Ph.D. Student of Business Administration
School of Management
Zhejiang University

--
-- 本小组推荐:
http://www.statmodel.com/
http://quantpsy.org/
http://www.kthau.com/?page_id=81
http://www.caldar.org/html/2010%20Speakers/mcardle.html
http://courses.nus.edu.sg/course/psycwlm/internet/
(4 TEACHING------4.2 Some notes from previous workshops)
http://blog.sina.com.cn/u/2142257021
http://lavaan.ugent.be/
http://personality-project.org/r/r.guide.html#Intro
 
 
You received this message because you are subscribed to the Google
Groups "Mplus学习小组" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/mpluser?hl=zh-TW
---
You received this message because you are subscribed to the Google Groups "Mplus学习小组" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
-- 本小组推荐:
http://www.statmodel.com/
http://quantpsy.org/
http://www.kthau.com/?page_id=81
http://www.caldar.org/html/2010%20Speakers/mcardle.html
http://courses.nus.edu.sg/course/psycwlm/internet/
(4 TEACHING------4.2 Some notes from previous workshops)
http://blog.sina.com.cn/u/2142257021
http://lavaan.ugent.be/
http://personality-project.org/r/r.guide.html#Intro
 
 
You received this message because you are subscribed to the Google
Groups "Mplus学习小组" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/mpluser?hl=zh-TW
---
You received this message because you are subscribed to the Google Groups "Mplus学习小组" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.