GENLINMIXED EMMEANS don't agree with descriptives

[Beckstead, Jason-2]

Hello all,

 

I am using GENLINMIXED procedure to analyze binary responses from a 3 x 3 within-subjects design. Each subject provided 9 responses (coded 0, 1). I am running a multilevel logistic regression model specifying a binomial distribution and logit link function. I include a random intercept to account for the within-subjects data structure. The procedure executes fine without any errors or warnings.

When I request the marginal means (and 95% C.I.s) with the /EMMEANS subcommand, however, the values provided by GENLINMIXED do not equal the values I get when I run MEANS procedure on the same data.

 

Any help reconciling would be appreciated.

Jason

 

 

_____________________________________________________________

Jason W. Beckstead, Ph.D.       

  Professor/Quantitative Methodologist

  University of South Florida College of Public Health

  13201 Bruce B. Downs Blvd., MDC56, Tampa, FL 33612, USA

  Statistical Editor, International Journal of Nursing Studies

  phone: [phone]  fax: [phone]          

  personal website:  http://personal.health.usf.edu/jbeckste/

  International Journal of Nursing Studies  http://www.elsevier.com/ijns

  NOTE new email: [hidden email]

 

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Bruce Weaver]

Hello Jason.  If you provide a small illustrative data set (via DATA LIST)
along with your GENLINMIXED and MEANS commands, others might find it easier
to help you figure out what's going on.  (Folks who do not wish to take the
time to generate such an example themselves may be willing to help if you
provide it!)  

Bruce



Beckstead, Jason-2 wrote

> Hello all,
>
> I am using GENLINMIXED procedure to analyze binary responses from a 3 x 3
> within-subjects design. Each subject provided 9 responses (coded 0, 1). I
> am running a multilevel logistic regression model specifying a binomial
> distribution and logit link function. I include a random intercept to
> account for the within-subjects data structure. The procedure executes
> fine without any errors or warnings.
> When I request the marginal means (and 95% C.I.s) with the /EMMEANS
> subcommand, however, the values provided by GENLINMIXED do not equal the
> values I get when I run MEANS procedure on the same data.
>
> Any help reconciling would be appreciated.
> Jason

-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Beckstead, Jason-2]

Thanks Bruce. Below is small data and syntax.


* Encoding: UTF-8.
*-9 within-subjects conditions (3 x 3; AC x WT) & binary choice response.
*-below is hypothetical data from N=6 subjects.
*-At issue: estimated means do not agree with observed means.
DATA LIST FREE /SUB AC WT CHOICE.
BEGIN DATA.
1 1 1 0
1 1 2 0
1 1 3 0
1 2 1 0
1 2 2 1
1 2 3 1
1 3 1 1
1 3 2 1
1 3 3 1
2 1 1 0
2 1 2 0
2 1 3 1
2 2 1 0
2 2 2 0
2 2 3 1
2 3 1 0
2 3 2 1
2 3 3 1
3 1 1 0
3 1 2 1
3 1 3 0
3 2 1 1
3 2 2 1
3 2 3 0
3 3 1 1
3 3 2 1
3 3 3 0
4 1 1 0
4 1 2 0
4 1 3 0
4 2 1 1
4 2 2 1
4 2 3 1
4 3 1 1
4 3 2 1
4 3 3 1
5 1 1 0
5 1 2 0
5 1 3 0
5 2 1 0
5 2 2 0
5 2 3 0
5 3 1 0
5 3 2 0
5 3 3 0
6 1 1 1
6 1 2 1
6 1 3 1
6 2 1 1
6 2 2 1
6 2 3 1
6 3 1 1
6 3 2 1
6 3 3 1
END DATA.

VARIABLE LEVEL SUB AC WT (NOMINAL).
GENLINMIXED /DATA_STRUCTURE SUBJECTS=SUB
 /FIELDS TARGET=CHOICE TRIALS=NONE OFFSET=NONE
 /TARGET_OPTIONS DISTRIBUTION=BINOMIAL LINK=LOGIT REFERENCE=0
 /FIXED EFFECTS= AC WT AC*WT USE_INTERCEPT=TRUE
 /RANDOM USE_INTERCEPT=TRUE SUBJECTS=SUB
 /EMMEANS TABLES=AC*WT
 /BUILD_OPTIONS TARGET_CATEGORY_ORDER=aSCENDING INPUTS_CATEGORY_ORDER=aSCENDING.

MEANS CHOICE BY AC BY WT.

_____________________________________________________________
Jason W. Beckstead, Ph.D.       
  Professor/Quantitative Methodologist
  University of South Florida College of Public Health
  13201 Bruce B. Downs Blvd., MDC56, Tampa, FL 33612, USA
  Statistical Editor, International Journal of Nursing Studies
  phone: [phone]  fax: [phone]          
  personal website:  http://personal.health.usf.edu/jbeckste/
  International Journal of Nursing Studies  http://www.elsevier.com/ijns
  NOTE new email: [hidden email]

-----Original Message-----
From: SPSSX(r) Discussion <[hidden email]> On Behalf Of Bruce Weaver
Sent: Wednesday, February 24, 2021 2:18 PM
To: [hidden email]
Subject: Re: GENLINMIXED EMMEANS don't agree with descriptives

Hello Jason.  If you provide a small illustrative data set (via DATA LIST) along with your GENLINMIXED and MEANS commands, others might find it easier to help you figure out what's going on.  (Folks who do not wish to take the time to generate such an example themselves may be willing to help if you provide it!)

Bruce



Beckstead, Jason-2 wrote

> Hello all,
>
> I am using GENLINMIXED procedure to analyze binary responses from a 3
> x 3 within-subjects design. Each subject provided 9 responses (coded
> 0, 1). I am running a multilevel logistic regression model specifying
> a binomial distribution and logit link function. I include a random
> intercept to account for the within-subjects data structure. The
> procedure executes fine without any errors or warnings.
> When I request the marginal means (and 95% C.I.s) with the /EMMEANS
> subcommand, however, the values provided by GENLINMIXED do not equal
> the values I get when I run MEANS procedure on the same data.
>
> Any help reconciling would be appreciated.
> Jason

-----
--
Bruce Weaver
[hidden email]
https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsites.google.com%2Fa%2Flakeheadu.ca%2Fbweaver%2F&amp;data=04%7C01%7Cjbeckste%40USF.EDU%7Cdbecac2f949745e6fef308d8d8f8f66b%7C741bf7dee2e546df8d6782607df9deaa%7C0%7C0%7C637497911082647083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=e329noDG7J7AWC6ep8QLIeGID7lCmcNUQt2KRhomOmQ%3D&amp;reserved=0

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspssx-discussion.1045642.n5.nabble.com%2F&amp;data=04%7C01%7Cjbeckste%40USF.EDU%7Cdbecac2f949745e6fef308d8d8f8f66b%7C741bf7dee2e546df8d6782607df9deaa%7C0%7C0%7C637497911082647083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=wp3j%2BV8IEoM3Q2z6vWIRyTCDejaBy84hJkf5wzybRyk%3D&amp;reserved=0

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD [EXTERNAL EMAIL] DO NOT CLICK links or attachments unless you recognize the sender and know the content is safe.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

Thanks for providing that example.  I am not surprised that the EMMEANS do
not match the MEANS, given that you are using a binomial error distribution
and a logit link.  If you change those to normal error distribution and
identity link (as shown below), the EMMEANS do match the MEANS.  (If this
"risk difference" kind of model makes sense, you'll probably want to use the
ROBUST covariance estimator too, as I have done below.)  


VARIABLE LEVEL CHOICE (SCALE).
GENLINMIXED /DATA_STRUCTURE SUBJECTS=SUB
 /FIELDS TARGET=CHOICE TRIALS=NONE OFFSET=NONE
 /TARGET_OPTIONS DISTRIBUTION=NORMAL LINK=IDENTITY REFERENCE=0
 /FIXED EFFECTS= AC WT AC*WT USE_INTERCEPT=TRUE
 /RANDOM USE_INTERCEPT=TRUE SUBJECTS=SUB
 /EMMEANS TABLES=AC*WT
 /BUILD_OPTIONS TARGET_CATEGORY_ORDER=aSCENDING
INPUTS_CATEGORY_ORDER=aSCENDING COVB = ROBUST.

MEANS CHOICE BY AC BY WT.



Beckstead, Jason-2 wrote

> Thanks Bruce. Below is small data and syntax.
>
>
> * Encoding: UTF-8.
> *-9 within-subjects conditions (3 x 3; AC x WT) & binary choice response.
> *-below is hypothetical data from N=6 subjects.
> *-At issue: estimated means do not agree with observed means.
> DATA LIST FREE /SUB AC WT CHOICE.
> BEGIN DATA.
> 1 1 1 0
> 1 1 2 0
> 1 1 3 0
> 1 2 1 0
> 1 2 2 1
> 1 2 3 1
> 1 3 1 1
> 1 3 2 1
> 1 3 3 1
> 2 1 1 0
> 2 1 2 0
> 2 1 3 1
> 2 2 1 0
> 2 2 2 0
> 2 2 3 1
> 2 3 1 0
> 2 3 2 1
> 2 3 3 1
> 3 1 1 0
> 3 1 2 1
> 3 1 3 0
> 3 2 1 1
> 3 2 2 1
> 3 2 3 0
> 3 3 1 1
> 3 3 2 1
> 3 3 3 0
> 4 1 1 0
> 4 1 2 0
> 4 1 3 0
> 4 2 1 1
> 4 2 2 1
> 4 2 3 1
> 4 3 1 1
> 4 3 2 1
> 4 3 3 1
> 5 1 1 0
> 5 1 2 0
> 5 1 3 0
> 5 2 1 0
> 5 2 2 0
> 5 2 3 0
> 5 3 1 0
> 5 3 2 0
> 5 3 3 0
> 6 1 1 1
> 6 1 2 1
> 6 1 3 1
> 6 2 1 1
> 6 2 2 1
> 6 2 3 1
> 6 3 1 1
> 6 3 2 1
> 6 3 3 1
> END DATA.
>
> VARIABLE LEVEL SUB AC WT (NOMINAL).
> GENLINMIXED /DATA_STRUCTURE SUBJECTS=SUB
>  /FIELDS TARGET=CHOICE TRIALS=NONE OFFSET=NONE
>  /TARGET_OPTIONS DISTRIBUTION=BINOMIAL LINK=LOGIT REFERENCE=0
>  /FIXED EFFECTS= AC WT AC*WT USE_INTERCEPT=TRUE
>  /RANDOM USE_INTERCEPT=TRUE SUBJECTS=SUB
>  /EMMEANS TABLES=AC*WT
>  /BUILD_OPTIONS TARGET_CATEGORY_ORDER=aSCENDING
> INPUTS_CATEGORY_ORDER=aSCENDING.
>
> MEANS CHOICE BY AC BY WT.

-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Beckstead, Jason-2]

Bruce,
Thanks for your reply. One reason I am using binomial dist & logit link function is that the DV (Choice) is binary and so logistic regression (not linear regression) is appropriate.
Another reason is that I want the 95% CIs for the cell means (which are proportions).

Treating the DV as scale, as you suggest, gives CIs that are incorrect; they exceed the range (0 to 1) allowed for proportions. (as shown in the output from your suggested syntax)

SO, I guess my main question is what is GENLINMIXED actually doing as it calculates the estimated means, standard errors, & CIs using binomial dist & logit link function?
I have reviewed the algorithms documentation for GENLINMIXED, but haven't been able to make much sense of it.

My hunch is that when GENLINMIXED calculates EMMEANS, that some Ss are given more weight than others.

Any thoughts?
Jason

_____________________________________________________________
Jason W. Beckstead, Ph.D.       
  Professor/Quantitative Methodologist
  University of South Florida College of Public Health
  13201 Bruce B. Downs Blvd., MDC56, Tampa, FL 33612, USA
  Statistical Editor, International Journal of Nursing Studies
  phone: [phone]  fax: [phone]          
  International Journal of Nursing Studies  http://www.elsevier.com/ijns
  NOTE new email: [hidden email]

-----Original Message-----
From: SPSSX(r) Discussion <[hidden email]> On Behalf Of Bruce Weaver
Sent: Friday, February 26, 2021 8:45 PM
To: [hidden email]
Subject: Re: GENLINMIXED EMMEANS don't agree with descriptives

Thanks for providing that example.  I am not surprised that the EMMEANS do not match the MEANS, given that you are using a binomial error distribution and a logit link.  If you change those to normal error distribution and identity link (as shown below), the EMMEANS do match the MEANS.  (If this "risk difference" kind of model makes sense, you'll probably want to use the ROBUST covariance estimator too, as I have done below.)


VARIABLE LEVEL CHOICE (SCALE).
GENLINMIXED /DATA_STRUCTURE SUBJECTS=SUB  /FIELDS TARGET=CHOICE TRIALS=NONE OFFSET=NONE  /TARGET_OPTIONS DISTRIBUTION=NORMAL LINK=IDENTITY REFERENCE=0  /FIXED EFFECTS= AC WT AC*WT USE_INTERCEPT=TRUE  /RANDOM USE_INTERCEPT=TRUE SUBJECTS=SUB  /EMMEANS TABLES=AC*WT  /BUILD_OPTIONS TARGET_CATEGORY_ORDER=aSCENDING INPUTS_CATEGORY_ORDER=aSCENDING COVB = ROBUST.

MEANS CHOICE BY AC BY WT.



Beckstead, Jason-2 wrote

> Thanks Bruce. Below is small data and syntax.
>
>
> * Encoding: UTF-8.
> *-9 within-subjects conditions (3 x 3; AC x WT) & binary choice response.
> *-below is hypothetical data from N=6 subjects.
> *-At issue: estimated means do not agree with observed means.
> DATA LIST FREE /SUB AC WT CHOICE.
> BEGIN DATA.
> 1 1 1 0
> 1 1 2 0
> 1 1 3 0
> 1 2 1 0
> 1 2 2 1
> 1 2 3 1
> 1 3 1 1
> 1 3 2 1
> 1 3 3 1
> 2 1 1 0
> 2 1 2 0
> 2 1 3 1
> 2 2 1 0
> 2 2 2 0
> 2 2 3 1
> 2 3 1 0
> 2 3 2 1
> 2 3 3 1
> 3 1 1 0
> 3 1 2 1
> 3 1 3 0
> 3 2 1 1
> 3 2 2 1
> 3 2 3 0
> 3 3 1 1
> 3 3 2 1
> 3 3 3 0
> 4 1 1 0
> 4 1 2 0
> 4 1 3 0
> 4 2 1 1
> 4 2 2 1
> 4 2 3 1
> 4 3 1 1
> 4 3 2 1
> 4 3 3 1
> 5 1 1 0
> 5 1 2 0
> 5 1 3 0
> 5 2 1 0
> 5 2 2 0
> 5 2 3 0
> 5 3 1 0
> 5 3 2 0
> 5 3 3 0
> 6 1 1 1
> 6 1 2 1
> 6 1 3 1
> 6 2 1 1
> 6 2 2 1
> 6 2 3 1
> 6 3 1 1
> 6 3 2 1
> 6 3 3 1
> END DATA.
>
> VARIABLE LEVEL SUB AC WT (NOMINAL).
> GENLINMIXED /DATA_STRUCTURE SUBJECTS=SUB  /FIELDS TARGET=CHOICE
> TRIALS=NONE OFFSET=NONE  /TARGET_OPTIONS DISTRIBUTION=BINOMIAL
> LINK=LOGIT REFERENCE=0  /FIXED EFFECTS= AC WT AC*WT USE_INTERCEPT=TRUE  
> /RANDOM USE_INTERCEPT=TRUE SUBJECTS=SUB  /EMMEANS TABLES=AC*WT  
> /BUILD_OPTIONS TARGET_CATEGORY_ORDER=aSCENDING
> INPUTS_CATEGORY_ORDER=aSCENDING.
>
> MEANS CHOICE BY AC BY WT.

-----
--
Bruce Weaver
[hidden email]
https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsites.google.com%2Fa%2Flakeheadu.ca%2Fbweaver%2F&amp;data=04%7C01%7Cjbeckste%40USF.EDU%7C878a650f35ca4810f58d08d8dac14b99%7C741bf7dee2e546df8d6782607df9deaa%7C0%7C0%7C637499871024726099%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=WbEFDlOLyruVkJ35luD4Db7txqhE%2BL%2FIuka9osK6PgI%3D&amp;reserved=0

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspssx-discussion.1045642.n5.nabble.com%2F&amp;data=04%7C01%7Cjbeckste%40USF.EDU%7C878a650f35ca4810f58d08d8dac14b99%7C741bf7dee2e546df8d6782607df9deaa%7C0%7C0%7C637499871024726099%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Cy9fTQlhxgHGTtp4aK16lJ%2FHlytq8mfX6mVj0pGuJtM%3D&amp;reserved=0

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD [EXTERNAL EMAIL] DO NOT CLICK links or attachments unless you recognize the sender and know the content is safe.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Andy W]

Bruce, those should only change the standard error of the means, not the mean
estimate itself (at least that is the case for data with only discrete dummy
variables like here).

Here I am not sure how SPSS treats the random effect in EMMEANS (sets it to
0 I imagine). If you don't estimate random effects the two procedures match.

MEANS CHOICE BY AC BY WT.

GENLINMIXED /DATA_STRUCTURE SUBJECTS=SUB
 /FIELDS TARGET=CHOICE TRIALS=NONE OFFSET=NONE
 /TARGET_OPTIONS DISTRIBUTION=BINOMIAL LINK=LOGIT REFERENCE=0
 /FIXED EFFECTS= AC WT AC*WT USE_INTERCEPT=TRUE
 /EMMEANS TABLES=AC*WT
 /BUILD_OPTIONS TARGET_CATEGORY_ORDER=aSCENDING
INPUTS_CATEGORY_ORDER=aSCENDING.






-----
Andy W
[hidden email]
http://andrewpwheeler.wordpress.com/
--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

"If you don't estimate random effects the two procedures match."

Aha.  Good observation, Andy.  However, to one of Jason's points, the limits
of some CIs are < 0 or > 1.  Using the small dataset Jason provided, and
excluding random effects yields these EMMEANS:


Estimates
AC WT Mean Std. Error 95% Confidence Interval
                                Lower Upper
1 1 .167 .152 -.140 .473
        2 .333 .192 -.054 .721
        3 .333 .192 -.054 .721
2 1 .500 .204 .089 .911
        2 .667 .192 .279 1.054
        3 .667 .192 .279 1.054
3 1 .667 .192 .279 1.054
        2 .833 .152 .527 1.140
        3 .667 .192 .279 1.054



Andy W wrote

> Bruce, those should only change the standard error of the means, not the
> mean
> estimate itself (at least that is the case for data with only discrete
> dummy
> variables like here).
>
> Here I am not sure how SPSS treats the random effect in EMMEANS (sets it
> to
> 0 I imagine). If you don't estimate random effects the two procedures
> match.
>
> MEANS CHOICE BY AC BY WT.
>
> GENLINMIXED /DATA_STRUCTURE SUBJECTS=SUB
>  /FIELDS TARGET=CHOICE TRIALS=NONE OFFSET=NONE
>  /TARGET_OPTIONS DISTRIBUTION=BINOMIAL LINK=LOGIT REFERENCE=0
>  /FIXED EFFECTS= AC WT AC*WT USE_INTERCEPT=TRUE
>  /EMMEANS TABLES=AC*WT
>  /BUILD_OPTIONS TARGET_CATEGORY_ORDER=aSCENDING
> INPUTS_CATEGORY_ORDER=aSCENDING.
>
>
>
>
>
>
> -----
> Andy W

> apwheele@

> http://andrewpwheeler.wordpress.com/
> --
> Sent from: http://spssx-discussion.1045642.n5.nabble.com/
>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Beckstead, Jason-2]

Hi guys,
Thanks for your responses regarding GENLINMIXED. I believe that including the random intercepts in the model is necessary to account for the nested structure of the data (9 binary responses per subject) and which should adjust the standard errors for nonindependence in the data.
Also, because the DV is binary the binomial distribution with the logit link function is appropriate for conducting logistic regression. I think that specifying both of these options is necessary to obtain proper estimates of the 95% Cis for the cell means (which are proportions).
I also note that the estimated grand mean (also a proportion) does not equal that of the entire sample (as calculated using MEANS or DESCRIPTIVES).
At this point I am quite uneasy trusting the results from GENLINMIXED. Does anyone know of another way to estimate a multilevel logistic regression in SPSS.
Thanks,
Jason



_____________________________________________________________
Jason W. Beckstead, Ph.D.       
  Professor/Quantitative Methodologist
  University of South Florida College of Public Health
  13201 Bruce B. Downs Blvd., MDC56, Tampa, FL 33612, USA
  Statistical Editor, International Journal of Nursing Studies
  phone: [phone]  fax: [phone]          
  personal website:  http://personal.health.usf.edu/jbeckste/
  International Journal of Nursing Studies  http://www.elsevier.com/ijns
  NOTE new email: [hidden email]

-----Original Message-----
From: SPSSX(r) Discussion <[hidden email]> On Behalf Of Bruce Weaver
Sent: Saturday, February 27, 2021 9:28 PM
To: [hidden email]
Subject: Re: GENLINMIXED EMMEANS don't agree with descriptives

"If you don't estimate random effects the two procedures match."

Aha.  Good observation, Andy.  However, to one of Jason's points, the limits of some CIs are < 0 or > 1.  Using the small dataset Jason provided, and excluding random effects yields these EMMEANS:


Estimates
AC      WT      Mean    Std. Error      95% Confidence Interval
                                Lower   Upper
1       1       .167    .152    -.140   .473
        2       .333    .192    -.054   .721
        3       .333    .192    -.054   .721
2       1       .500    .204     .089   .911
        2       .667    .192     .279   1.054
        3       .667    .192     .279   1.054
3       1       .667    .192     .279   1.054
        2       .833    .152     .527   1.140
        3       .667    .192     .279   1.054



Andy W wrote

> Bruce, those should only change the standard error of the means, not
> the mean estimate itself (at least that is the case for data with only
> discrete dummy variables like here).
>
> Here I am not sure how SPSS treats the random effect in EMMEANS (sets
> it to
> 0 I imagine). If you don't estimate random effects the two procedures
> match.
>
> MEANS CHOICE BY AC BY WT.
>
> GENLINMIXED /DATA_STRUCTURE SUBJECTS=SUB  /FIELDS TARGET=CHOICE
> TRIALS=NONE OFFSET=NONE  /TARGET_OPTIONS DISTRIBUTION=BINOMIAL
> LINK=LOGIT REFERENCE=0  /FIXED EFFECTS= AC WT AC*WT USE_INTERCEPT=TRUE  
> /EMMEANS TABLES=AC*WT  /BUILD_OPTIONS TARGET_CATEGORY_ORDER=aSCENDING
> INPUTS_CATEGORY_ORDER=aSCENDING.
>
>
>
>
>
>
> -----
> Andy W

> apwheele@

> https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fandre
> wpwheeler.wordpress.com%2F&amp;data=04%7C01%7Cjbeckste%40usf.edu%7C621
> d0b29623148ab8e2308d8db907101%7C741bf7dee2e546df8d6782607df9deaa%7C0%7
> C1%7C637500760713804811%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLC
> JQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=BsiihZ8u
> %2Bvs99Yn5W%2B%2FLGLWYdpjD5HGFNRVskWwdwaU%3D&amp;reserved=0
> --
> Sent from:
> https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspssx
> -discussion.1045642.n5.nabble.com%2F&amp;data=04%7C01%7Cjbeckste%40usf
> .edu%7C621d0b29623148ab8e2308d8db907101%7C741bf7dee2e546df8d6782607df9
> deaa%7C0%7C1%7C637500760713814813%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4w
> LjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdat
> a=bGpI65uazTdwGqNAEdZWPfsbaEVsfwo3lfHVgpF5yYM%3D&amp;reserved=0
>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the command. To leave the
> list, send the command SIGNOFF SPSSX-L For a list of commands to
> manage subscriptions, send the command INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsites.google.com%2Fa%2Flakeheadu.ca%2Fbweaver%2F&amp;data=04%7C01%7Cjbeckste%40usf.edu%7C621d0b29623148ab8e2308d8db907101%7C741bf7dee2e546df8d6782607df9deaa%7C0%7C1%7C637500760713814813%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=LsDDhOHSsX0KMac9chrGOCCzgcK24RBXQZKZhyxRnm4%3D&amp;reserved=0

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspssx-discussion.1045642.n5.nabble.com%2F&amp;data=04%7C01%7Cjbeckste%40usf.edu%7C621d0b29623148ab8e2308d8db907101%7C741bf7dee2e546df8d6782607df9deaa%7C0%7C1%7C637500760713814813%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=bGpI65uazTdwGqNAEdZWPfsbaEVsfwo3lfHVgpF5yYM%3D&amp;reserved=0

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD [EXTERNAL EMAIL] DO NOT CLICK links or attachments unless you recognize the sender and know the content is safe.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

Jason, I tried analyzing your sample dataset via Stata, and the same issue
arose there.  I.e., to get the predicted probabilities to match the ordinary
means, I had to omit the random intercept term from the model (using Stata's
-melogit- command).  So whatever is going on is not peculiar to GENLINMIXED.  

I'll go back to something I mentioned before:  If you are willing to model
the risk difference rather than the odds ratio, you can get GENLINMIXED to
give you predicted probabilities that match ordinary means.  But you can
also get predicted probabilities (or CI limits) that fall outside the range
0 to 1.  This is unavoidable, I think.  E.g., see this nice set of slides:

https://si.biostat.washington.edu/sites/default/files/modules/2018_SISG_4_4_0.pdf

HTH.


Beckstead, Jason-2 wrote

> Hi guys,
> Thanks for your responses regarding GENLINMIXED. I believe that including
> the random intercepts in the model is necessary to account for the nested
> structure of the data (9 binary responses per subject) and which should
> adjust the standard errors for nonindependence in the data.
> Also, because the DV is binary the binomial distribution with the logit
> link function is appropriate for conducting logistic regression. I think
> that specifying both of these options is necessary to obtain proper
> estimates of the 95% Cis for the cell means (which are proportions).
> I also note that the estimated grand mean (also a proportion) does not
> equal that of the entire sample (as calculated using MEANS or
> DESCRIPTIVES).
> At this point I am quite uneasy trusting the results from GENLINMIXED.
> Does anyone know of another way to estimate a multilevel logistic
> regression in SPSS.
> Thanks,
> Jason

-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Andy W]

Well, it does not mean the model is wrong Jason, we are just talking about
different ways of summarizing the same data.

I was actually wrong in guessing it set the random intercepts to 0, it
appears to just calculate them through. See an excel spreadsheet I did
quickly when printing off the random intercepts in this example,

https://www.dropbox.com/s/cu3mbgpof1klech/EMMEANS_Example.xlsx?dl=0

You could calculate the EMMEANS yourself, I show how to set the random
intercepts to 0 overall (which gives you your expected behavior Jason of
marginal = estimated). I don't know offhand how folks do the standard error
estimates for EMMEANS though.

If folks from SPSS pay attention here, this suggests it would be useful to
add an option to /EMMEANS_OPTIONS to set the random effects to particular
values, e.g.

/EMMEANS_OPTIONS SUBJECT=0

or something like that. Currently only supports setting the fixed effects to
particular values.



-----
Andy W
[hidden email]
http://andrewpwheeler.wordpress.com/
--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Beckstead, Jason-2]

Thanks Andy, your excel sheet is helpful.
How did you get GENLINMIXED to print (or save) the intercepts for each subject?
Jason

_____________________________________________________________
Jason W. Beckstead, Ph.D.       
  Professor/Quantitative Methodologist
  University of South Florida College of Public Health
  13201 Bruce B. Downs Blvd., MDC56, Tampa, FL 33612, USA
  Statistical Editor, International Journal of Nursing Studies
  phone: [phone]  fax: [phone]          
  International Journal of Nursing Studies  http://www.elsevier.com/ijns
  NOTE new email: [hidden email]

-----Original Message-----
From: SPSSX(r) Discussion <[hidden email]> On Behalf Of Andy W
Sent: Tuesday, March 2, 2021 11:02 AM
To: [hidden email]
Subject: Re: GENLINMIXED EMMEANS don't agree with descriptives

Well, it does not mean the model is wrong Jason, we are just talking about different ways of summarizing the same data.

I was actually wrong in guessing it set the random intercepts to 0, it appears to just calculate them through. See an excel spreadsheet I did quickly when printing off the random intercepts in this example,

https://www.dropbox.com/s/cu3mbgpof1klech/EMMEANS_Example.xlsx?dl=0

You could calculate the EMMEANS yourself, I show how to set the random intercepts to 0 overall (which gives you your expected behavior Jason of marginal = estimated). I don't know offhand how folks do the standard error estimates for EMMEANS though.

If folks from SPSS pay attention here, this suggests it would be useful to add an option to /EMMEANS_OPTIONS to set the random effects to particular values, e.g.

/EMMEANS_OPTIONS SUBJECT=0

or something like that. Currently only supports setting the fixed effects to particular values.



-----
Andy W
[hidden email]
https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fandrewpwheeler.wordpress.com%2F&amp;data=04%7C01%7Cjbeckste%40usf.edu%7C94111d85910f4a21058c08d8dd94889b%7C741bf7dee2e546df8d6782607df9deaa%7C0%7C1%7C637502977303426429%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=fI6VCBITK9LTJk1cAKFA9NkrElo5emMBYY3KGEBWuqI%3D&amp;reserved=0
--
Sent from: https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspssx-discussion.1045642.n5.nabble.com%2F&amp;data=04%7C01%7Cjbeckste%40usf.edu%7C94111d85910f4a21058c08d8dd94889b%7C741bf7dee2e546df8d6782607df9deaa%7C0%7C1%7C637502977303426429%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=vvDiRrvbIcnj9yDHLr1voIz9DDnN0NVaDXO3Dlr2Wn4%3D&amp;reserved=0

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD [EXTERNAL EMAIL] DO NOT CLICK links or attachments unless you recognize the sender and know the content is safe.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Beckstead, Jason-2]

Thanks Andy & Bruce,
I ran the same sample data in SAS and get the same estimated means.
I was so distracted by the difference in observed vs estimated means that I failed to notice another quirk with GENLINMIXED,
it gets the F values correct (compared to SAS) but the df2 values (and thus the p values) are not correct for the two main effects (df2 should=10) and the interaction (df2 should=20). It uses df2=45 for all tests.
Does anyone know how to change this? I have already tried the various options for DF_METHOD, but it does not change the df2 used in the three omnibus tests.
Jason
_____________________________________________________________
Jason W. Beckstead, Ph.D.       
  Professor/Quantitative Methodologist
  University of South Florida College of Public Health
  13201 Bruce B. Downs Blvd., MDC56, Tampa, FL 33612, USA
  Statistical Editor, International Journal of Nursing Studies
  phone: [phone]  fax: [phone]          
  International Journal of Nursing Studies  http://www.elsevier.com/ijns
  NOTE new email: [hidden email]

-----Original Message-----
From: SPSSX(r) Discussion <[hidden email]> On Behalf Of Andy W
Sent: Tuesday, March 2, 2021 11:02 AM
To: [hidden email]
Subject: Re: GENLINMIXED EMMEANS don't agree with descriptives

Well, it does not mean the model is wrong Jason, we are just talking about different ways of summarizing the same data.

I was actually wrong in guessing it set the random intercepts to 0, it appears to just calculate them through. See an excel spreadsheet I did quickly when printing off the random intercepts in this example,

https://www.dropbox.com/s/cu3mbgpof1klech/EMMEANS_Example.xlsx?dl=0

You could calculate the EMMEANS yourself, I show how to set the random intercepts to 0 overall (which gives you your expected behavior Jason of marginal = estimated). I don't know offhand how folks do the standard error estimates for EMMEANS though.

If folks from SPSS pay attention here, this suggests it would be useful to add an option to /EMMEANS_OPTIONS to set the random effects to particular values, e.g.

/EMMEANS_OPTIONS SUBJECT=0

or something like that. Currently only supports setting the fixed effects to particular values.



-----
Andy W
[hidden email]
https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fandrewpwheeler.wordpress.com%2F&amp;data=04%7C01%7Cjbeckste%40usf.edu%7C94111d85910f4a21058c08d8dd94889b%7C741bf7dee2e546df8d6782607df9deaa%7C0%7C1%7C637502977303426429%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=fI6VCBITK9LTJk1cAKFA9NkrElo5emMBYY3KGEBWuqI%3D&amp;reserved=0
--
Sent from: https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspssx-discussion.1045642.n5.nabble.com%2F&amp;data=04%7C01%7Cjbeckste%40usf.edu%7C94111d85910f4a21058c08d8dd94889b%7C741bf7dee2e546df8d6782607df9deaa%7C0%7C1%7C637502977303426429%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=vvDiRrvbIcnj9yDHLr1voIz9DDnN0NVaDXO3Dlr2Wn4%3D&amp;reserved=0

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD [EXTERNAL EMAIL] DO NOT CLICK links or attachments unless you recognize the sender and know the content is safe.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD