Significant interaction- reporting statistics

[lken]

Hello everyone,

I have a few follow-up questions regarding the suggestion of doing a binomial logistic regression GLM in SPSS.

I have a categorical predictor variable (3 levels... trying to find the proportion difference between three groups), and multiple continuos covariates.

One of my covariates is showing a significant interaction term with my categorical predictor.

How do I find out how differently it effects each group? How do I report this, and given the significant interaction, what statistics do I use to display that the groups are different?

Thanks,
lken

[Bruce Weaver]

It sounds like your primary interest is in the Group variable, and that you want to see how differences among the groups differ, depending on the covariate that interacts with Group.  If so, you might find it more convenient to run your model via GENLIN (rather than LOGISTIC REGRESSION).  The EMMEANS sub-command can be used to display fitted values of Y at selected combinations of Group and the covariate.  I don't have SPSS on this machine, so the following is untested.  But I think you want syntax something like this:

GENLIN Y (REFERENCE=FIRST)
       BY Group { other categorical variables } (order = descending)
       WITH X1 { other covariates }
  /MODEL   Group X1 Group*X1 { other terms }
   INTERCEPT=YES
   DISTRIBUTION=BINOMIAL LINK=LOGIT
  /EMMEANS TABLES=Group CONTROL=X1(value1) SCALE=TRANSFORMED
  /EMMEANS TABLES=Group CONTROL=X1(value2) SCALE=TRANSFORMED
  /EMMEANS TABLES=Group CONTROL=X1(value3) SCALE=TRANSFORMED
  /MISSING CLASSMISSING=EXCLUDE
  /PRINT CPS DESCRIPTIVES MODELINFO FIT SUMMARY SOLUTION (EXPONENTIATED).

Replace value1, value2 and value3 with 3 selected values of X1, the covariate.  You could also use OMS to send the fitted values to another dataset, and then plot them to show graphically the nature of the interaction.

HTH.

lken wrote

Hello everyone,

I have a few follow-up questions regarding the suggestion of doing a binomial logistic regression GLM in SPSS.

I have a categorical predictor variable (3 levels... trying to find the proportion difference between three groups), and multiple continuos covariates.

One of my covariates is showing a significant interaction term with my categorical predictor.

How do I find out how differently it effects each group? How do I report this, and given the significant interaction, what statistics do I use to display that the groups are different?

Thanks,
lken

[lken]

Can I use the GENLIN if my response variable is binary?

You are correct, I care mostly about the differences between groups. I just wanted to add in covariates so my results are not bias.

In the first test I did, age is normally distributed and there is NO significant interaction between group and age. I fitted a logistic model, then asked for Tukey's HSD post hoc tests to find where the differences in proportion are between groups. Here is what my results looks like:

The fitted logistic model with groups as a predictor and age as a covariate was significant (X2=18.511, df=3, p=<0.001). Age was a significant covariate (X2=7.357, df=1, p=0.007), with every year increase in age leading to a EXP(B) 1.084 (CI %95=1.023-1.150) chance of a tree falling. When Age is held at a constant 25.17cm, the group 1 has an adjusted mean proportion of fallen trees of approximately 69%, significantly more than the means for the group2 and group3 (p=0.048, p =0.003). The adjusted mean proportion of fallen trees in Group2 and Group3 are 38% and 32% and do not significantly differ (p=0.631).

I would like to make a simular write-up for my other test, but there is an significant interation between age and group in that one. I am not sure if I can still accept my Tukey's HSD post hocs and report those means.

I'm going to look into the GLM you proposed for when there are significant interactions, but does this seem like a valid start? Would you be able to submit something like this to a journal? It will obviously be gone over thoroughly by my supervisor before it is submitted, but I would like to hand in something solid before he goes over it.

[lken]

Just another note. I am using the GLM in SPSS with a binary logistic link. I can not get the Nagelkerte R value in this test. Instead I am using a omnibus test for model fit. I also do not get statistics like "We predicted 70% of the outcomes correctly" etc..

I chose to do it this way because in GLM I am able to get post hoc tests for proportion means.

It seemed more valid than running a logistic regression and then changing the reference group to get differences between groups, what do you guys think?

[lken]

Hi Bruce,

The syntax you game me is working well. I'm essentially getting the same output as I was by using the window instead of the syntax method, exept I am able to control what the constants are.

I end up with just density as a significant covariate, and it has a significant interaction with group. I asked for EMMEANS for three different densities. (200,400,600 trees per hectare). Here is one of the outputs:

                Estimates
                        95% Wald Confidence Interval
BEC Mean Std. Error Lower Upper
SBS -.990 .091        -1.168 -.812
SBPS -1.931 .580        -3.067 -.795
IDF -1.136 .613       -2.337 .066
Covariates appearing in the model are fixed at the following values: density=400.000000

I am confused because I thought it was giving me a value for the mean proportion of trees down when density is held at 400. Why are the values negative, and why are some of them above 0? I am also unsure as to how I would use this method to see the difference in how density effects the proportion down bewteen the three groups.

[Bruce Weaver]

For the syntax I posted, the estimated marginal means would be log-odds, not proportions.  The nice thing about using log-odds is that things that are linear look linear when you plot them.  If you plot odds or predicted probabilities or whatever, that will not be so.

HTH.

lken wrote

Hi Bruce,

The syntax you game me is working well. I'm essentially getting the same output as I was by using the window instead of the syntax method, exept I am able to control what the constants are.

I end up with just density as a significant covariate, and it has a significant interaction with group. I asked for EMMEANS for three different densities. (200,400,600 trees per hectare). Here is one of the outputs:

                Estimates
                        95% Wald Confidence Interval
BEC Mean Std. Error Lower Upper
SBS -.990 .091        -1.168 -.812
SBPS -1.931 .580        -3.067 -.795
IDF -1.136 .613       -2.337 .066
Covariates appearing in the model are fixed at the following values: density=400.000000

I am confused because I thought it was giving me a value for the mean proportion of trees down when density is held at 400. Why are the values negative, and why are some of them above 0? I am also unsure as to how I would use this method to see the difference in how density effects the proportion down bewteen the three groups.

[lken]

OK, that makes more sense.

So potentially, I could plug in a bunch of different values for density, then plot the log odds on a graph to show, for example, that increasing density in group 1 decreases the chance of a tree falling, while in group two it also decreases the probability just not as strongly.

Because I have a significant interaction term in this case, it is not valid to ask for Tukey's HSD. Could I just change the reference group and report log-odds? For example group 1 is the reference, I would report log-odds and the p value. Then change the reference group and report log-odds with their p values? The p-values in this case signify that the groups are different correct?

[Bruce Weaver]

AFAIK, you can't get Tukey's HSD via GENLIN.  Do you mean LSD?  

Look up EMMEANS under GENLIN, and pay attention to the COMPARE and CONTRAST options.  One of the options is pairwise contrasts.  It sounds like that may be what you want.

lken wrote

OK, that makes more sense.

So potentially, I could plug in a bunch of different values for density, then plot the log odds on a graph to show, for example, that increasing density in group 1 decreases the chance of a tree falling, while in group two it also decreases the probability just not as strongly.

Because I have a significant interaction term in this case, it is not valid to ask for Tukey's HSD. Could I just change the reference group and report log-odds? For example group 1 is the reference, I would report log-odds and the p value. Then change the reference group and report log-odds with their p values? The p-values in this case signify that the groups are different correct?

[lken]

Yes, pairwise contrasts is what I meant. I am reporting those for my study, while the covariate is held at a constant.

When my covariate has an interaction term, are those pairwise contrasts still valid?

[Bruce Weaver]

If I follow, you're getting the pairwise among your 3 groups at 3 different values of the covariate (200, 400 and 600).  I don't see any problem, other than inflation of the alpha.  But you could address that with a Bonferroni correction.

lken wrote

Yes, pairwise contrasts is what I meant. I am reporting those for my study, while the covariate is held at a constant.

When my covariate has an interaction term, are those pairwise contrasts still valid?

[Tierney, Mary Lou]

I am PTOB today Friday 13 April returning to the office on Tuesday April 17.  I will not have access to email while I am away from the office.

 

For assistance with the MITRE Innovation Program please contact the [hidden email]

 

For assistance with Project/Portfolio Pages or Discover, please contact the [hidden email]

 

For assistance with other Innovation Zone sites, such as CI&T InZone, please contact [hidden email].

 

 

Regards,

Mary Lou

[lken]

If the means are the log odds, then what are they being compared to? I have three groups, and each group has a value. I thought one should be the reference group and not have a value.

[Bruce Weaver]

You are thinking of a table of coefficients, where there are k-1 coefficients for k groups.  Those k-1 coefficients represent differences between the k-1 group means and the mean for the 1 reference group.  But your table of EMMEANS is not giving you coefficients--it is giving you means (although it would probably be more accurate to say fitted values in this case).  That's why there's a value for each group.  

HTH.

lken wrote

If the means are the log odds, then what are they being compared to? I have three groups, and each group has a value. I thought one should be the reference group and not have a value.

[lken]

Sorry about this, I've never done a binary regression before, let alone had to deal with interactions. I've got another load of questions

Here is my test so far:
Density shows a significant interaction with group (p<0.001), so I can not just report the mean differences in proportion between groups for one value of density.

I asked for estimated marginal means for my groups for three values of density (the mean, the mean minus STD, the mean plus STD).

Group 1 and 2 start with negative values, and continue to lower. Group 3 starts with a positive value, and gets higher. Interesting results, though I am still not sure what the mean is actually refering to. You said log ods, but then I got confused with coefficients and you said it was the mean... the mean of the log odds? If it is a log odds, doesn't that mean there must be a comparison because it's a ratio?

It appears that at least group 3 is reacting differently to density than my other groups. How exactly can I get a p-value to say it is significant?

[Bruce Weaver]

Think of fitted values rather than means.  The values you are seeing in the EMMEANS output give the fitted value of the log-odds of the outcome variable being equal to 1 for that combination of explanatory variables.  

In your table of coefficients, the B-value for the Intercept gives the log-odds of the outcome variable being equal to 1 when all explanatory variables are equal to 0 (or their reference categories for categorical variables).  

I suggest you look for James Jaccard's book on interactions in logistic regression -- it's one of those little green Sage monographs, and is quite good.  It may even be available via Google Books.

HTH.

lken wrote

Sorry about this, I've never done a binary regression before, let alone had to deal with interactions. I've got another load of questions

Here is my test so far:
Density shows a significant interaction with group (p<0.001), so I can not just report the mean differences in proportion between groups for one value of density.

I asked for estimated marginal means for my groups for three values of density (the mean, the mean minus STD, the mean plus STD).

Group 1 and 2 start with negative values, and continue to lower. Group 3 starts with a positive value, and gets higher. Interesting results, though I am still not sure what the mean is actually refering to. You said log ods, but then I got confused with coefficients and you said it was the mean... the mean of the log odds? If it is a log odds, doesn't that mean there must be a comparison because it's a ratio?

It appears that at least group 3 is reacting differently to density than my other groups. How exactly can I get a p-value to say it is significant?

[lken]

Aha! This series of books would have been really useful at the start of my analysis. They were in a different section that I was used to in our library. Thanks very much for the reference.

I think I get it now- my estimated marginal means are of the log odds of my outcome variable being 1. The groups are not being compared to each other in any way. In other words, if my value is 0.266 for group 1, there is a 26% chance that a a dead tree will be fallen (coded 1) in group 1 if the DBH is (whatever value I entered in).

Does that sound about right? Then I can graph the odds for different values of my covariate to see how it interacts differently with my three groups.

If this is right, can I just ignore the fact that the estimated marginal means are negative... or does being negative mean something important?

[Bruce Weaver]

Where did that 26% come from?

Log-odds(Y=1) = 0.266
Odds(Y=1) = Exp(0.266) = 1.305
P(Y=1) = 1.305 / (1.305 + 1) = 0.566.

Here's a nice note by Steve Simon that might be helpful.

   http://www.childrensmercy.org/stats/model/logist_concepts.aspx

HTH.

lken wrote

Aha! This series of books would have been really useful at the start of my analysis. They were in a different section that I was used to in our library. Thanks very much for the reference.

I think I get it now- my estimated marginal means are of the log odds of my outcome variable being 1. The groups are not being compared to each other in any way. In other words, if my value is 0.266 for group 1, there is a 26% chance that a a dead tree will be fallen (coded 1) in group 1 if the DBH is (whatever value I entered in).

Does that sound about right? Then I can graph the odds for different values of my covariate to see how it interacts differently with my three groups.

If this is right, can I just ignore the fact that the estimated marginal means are negative... or does being negative mean something important?

[Ryan]

I don't have time to elaborate much except to strongly advise the OP to write out the function; i.e.,

logit[pr(y=1)] = b0 + b1*x1 + b2*x2 + b3*x1*x2

Note: If any of the predictors are categorical, use indicator coding. With more than 2 levels of the categorical predictor, the equation must be expanded accordingly.

Plug in the estimated coefficients (b1, b2, b3), set the x values at whatever is desired, and then apply the inverse logit link function to obtain the predicted probabilities.

Only then will a deeper understanding of logistic regression be achieved, IMO.

Ryan

On Apr 16, 2012, at 8:46 PM, Bruce Weaver <[hidden email]> wrote:

> Where did that 26% come from?
>
> Log-odds(Y=1) = 0.266
> Odds(Y=1) = Exp(0.266) = 1.305
> P(Y=1) = 1.305 / (1.305 + 1) = 0.566.
>
> Here's a nice note by Steve Simon that might be helpful.
>
>   http://www.childrensmercy.org/stats/model/logist_concepts.aspx
>
> HTH.
>
>
>
> lken wrote
>>
>> Aha! This series of books would have been really useful at the start of my
>> analysis. They were in a different section that I was used to in our
>> library. Thanks very much for the reference.
>>
>> I think I get it now- my estimated marginal means are of the log odds of
>> my outcome variable being 1. The groups are not being compared to each
>> other in any way. In other words, if my value is 0.266 for group 1, there
>> is a 26% chance that a a dead tree will be fallen (coded 1) in group 1 if
>> the DBH is (whatever value I entered in).
>>
>> Does that sound about right? Then I can graph the odds for different
>> values of my covariate to see how it interacts differently with my three
>> groups.
>>
>> If this is right, can I just ignore the fact that the estimated marginal
>> means are negative... or does being negative mean something important?
>>
>
>
> -----
> --
> Bruce Weaver
> [hidden email]
> http://sites.google.com/a/lakeheadu.ca/bweaver/
>
> "When all else fails, RTFM."
>
> NOTE: My Hotmail account is not monitored regularly.
> To send me an e-mail, please use the address shown above.
>
> --
> View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Significant-interaction-reporting-statistics-tp5634009p5645271.html
> Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD