Obtaining means and effect sizes from a multiple regression run

[Anton-24]

Dear SPSS Listers,

It's been ages since my last statistics course, so please bear with me as I
struggle to figure out this multiple regression issue.

My main regression run includes a primary independent variable I'll call
"group" (coded 0 & 1) along with other independent variables.  I am trying
to predict test score, my primary dependent variable.  In order to get the
mean test score for the two groups, I have been dumping the unstandardized
betas along with the independent variable means (and the constant) into to
Excel, substituting 0 and 1 for the group means, and letting the spreadsheet
program do the rest.  Is there any way for SPSS to do some of this for me?

I also need to compute an effect size for group.  Is it okay to take the
unstandardized beta for group and simply divide by the control group's SD
obtained from a basic descriptives procedure, or do I need to somehow obtain
a SD directly from the regression run.

I really appreciate any guidance.

Anton

[statisticsdoc]

Stephen Brand
www.statisticsdoc.com

Anton,

Are you trying to compute the mean test score for the two groups, or the
mean __predicted__ test score for the two groups?  What you are doing in the
spreadsheet will generate predicted scores.  Another option, if your
interest lies in predicted scores, is to have SPSS save predicted scores
(see SAVE option under the regression command), and then compute means on
the predicted scores for the groups.  If you want to know the actual test
scores for the groups, just run Means.

What sort of effect size are you interested in?  If you want to know the
unique contribution of group membership, adjusting for the other variables
in the model, then compare the r-squared of the regression model with and
without the group membership effect.  The increment in the r-squared
indicates how much additional variance in the dependent variable is
accounted for by the inclusion of group membership, over and above the
variance explained by the other independent variables in the regression
model.  Please note that the size of the unique effect size is meaningful
only in the context of the other variables in the regression model.  If you
have a different set of independent variables, the unique contribution of
group membership will likely be different.

HTH,

Stephen Brand


For personalized and professional consultation in statistics and research
design, visit
www.statisticsdoc.com


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
Anton
Sent: Sunday, January 14, 2007 6:37 PM
To: [hidden email]
Subject: Obtaining means and effect sizes from a multiple regression run


Dear SPSS Listers,

It's been ages since my last statistics course, so please bear with me as I
struggle to figure out this multiple regression issue.

My main regression run includes a primary independent variable I'll call
"group" (coded 0 & 1) along with other independent variables.  I am trying
to predict test score, my primary dependent variable.  In order to get the
mean test score for the two groups, I have been dumping the unstandardized
betas along with the independent variable means (and the constant) into to
Excel, substituting 0 and 1 for the group means, and letting the spreadsheet
program do the rest.  Is there any way for SPSS to do some of this for me?

I also need to compute an effect size for group.  Is it okay to take the
unstandardized beta for group and simply divide by the control group's SD
obtained from a basic descriptives procedure, or do I need to somehow obtain
a SD directly from the regression run.

I really appreciate any guidance.

Anton

[Anton-24]

Thanks to Stephen Brand for clarifying my original post.

I am interested in obtaining the two means that correspond to the effect
attributed to "group" (in a sense, treatment vs. control) when the other
independent variables are "controlled for," by setting their values to their
own respective means.  For example, the unstandardized beta for group is
equal to 9.00 in one of the runs.  I have taken the unstandarized betas for
the other independent variables and their corresponding means from the
descriptive portion of the multiple regression procedure and plopped them
into Excel.  I then set group equal to zero, then group equal to one, to
obtain the mean values for each group.  This procedure yields means for
group (21.50 & 30.50) whose difference is equal to the unstandardized effect
for group (9.00).  These are the means I think I should report.  If I use
the SAVE option under the regression command, the resultant means are not
21.50 & 30.50, nor is the difference between them equal to 9.00.

I'd also like to generate some measure of effect size, rather than unique
contribution of group membership.  In a couple of the regression
discontinuity papers I have read, the authors seem to taken the
unstandardized beta for group and divided it by some standard deviation
measure.  I assume that the SD is simply coming from a descriptives
procedure and not generated via the regression run.

So... I guess this boils down to two or more questions:

(1) Am I using the correct procedure to generate the means, and do I have
the correct interpretation of the unstandardized beta?

(2) Can I use the unstandardized beta and some measure of the SD to estimate
an effect size.  By which procedure should I generate the SD?

I really appreciate any guidance.

Anton




On Sun, 14 Jan 2007 21:14:25 -0500, Statisticsdoc <[hidden email]> wrote:

>Stephen Brand
>www.statisticsdoc.com
>
>Anton,
>
>Are you trying to compute the mean test score for the two groups, or the
>mean __predicted__ test score for the two groups?  What you are doing in the
>spreadsheet will generate predicted scores.  Another option, if your
>interest lies in predicted scores, is to have SPSS save predicted scores
>(see SAVE option under the regression command), and then compute means on
>the predicted scores for the groups.  If you want to know the actual test
>scores for the groups, just run Means.
>
>What sort of effect size are you interested in?  If you want to know the
>unique contribution of group membership, adjusting for the other variables
>in the model, then compare the r-squared of the regression model with and
>without the group membership effect.  The increment in the r-squared
>indicates how much additional variance in the dependent variable is
>accounted for by the inclusion of group membership, over and above the
>variance explained by the other independent variables in the regression
>model.  Please note that the size of the unique effect size is meaningful
>only in the context of the other variables in the regression model.  If you
>have a different set of independent variables, the unique contribution of
>group membership will likely be different.
>
>HTH,
>
>Stephen Brand
>
>
>For personalized and professional consultation in statistics and research
>design, visit
>www.statisticsdoc.com
>
>
>-----Original Message-----
>From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
>Anton
>Sent: Sunday, January 14, 2007 6:37 PM
>To: [hidden email]
>Subject: Obtaining means and effect sizes from a multiple regression run
>
>
>Dear SPSS Listers,
>
>It's been ages since my last statistics course, so please bear with me as I
>struggle to figure out this multiple regression issue.
>
>My main regression run includes a primary independent variable I'll call
>"group" (coded 0 & 1) along with other independent variables.  I am trying
>to predict test score, my primary dependent variable.  In order to get the
>mean test score for the two groups, I have been dumping the unstandardized
>betas along with the independent variable means (and the constant) into to
>Excel, substituting 0 and 1 for the group means, and letting the spreadsheet
>program do the rest.  Is there any way for SPSS to do some of this for me?
>
>I also need to compute an effect size for group.  Is it okay to take the
>unstandardized beta for group and simply divide by the control group's SD
>obtained from a basic descriptives procedure, or do I need to somehow obtain
>a SD directly from the regression run.
>
>I really appreciate any guidance.
>
>Anton

[statisticsdoc]

Anton,

Thank you for clarifying the question.

In the context of the regression discontinuity design, it is an accepted
practice to interpret the unstandardized beta in terms of the pooled within
group standard deviation (if the variances are homogeneous between groups).
If I can throw in another five cents or more of advice, I would suggest that
you check on work by Hedges and Olkin (1985) dealing with potential biases
that can affect this design.  It is also critically important that the
regression model be correctly specified in terms of potential quadratic and
higher order terms, and interactions.

Best Wishes,

Stephen Brand

For personalized and professional consultation in statistics and research
design, visit
www.statisticsdoc.com


-----Original Message-----
From: Anton [mailto:[hidden email]]
Sent: Monday, January 15, 2007 7:00 PM
To: [hidden email]; Statisticsdoc Consulting
Cc: Anton
Subject: Re: Obtaining means and effect sizes from a multiple regression
run


Thanks to Stephen Brand for clarifying my original post.

I am interested in obtaining the two means that correspond to the effect
attributed to "group" (in a sense, treatment vs. control) when the other
independent variables are "controlled for," by setting their values to their
own respective means.  For example, the unstandardized beta for group is
equal to 9.00 in one of the runs.  I have taken the unstandarized betas for
the other independent variables and their corresponding means from the
descriptive portion of the multiple regression procedure and plopped them
into Excel.  I then set group equal to zero, then group equal to one, to
obtain the mean values for each group.  This procedure yields means for
group (21.50 & 30.50) whose difference is equal to the unstandardized effect
for group (9.00).  These are the means I think I should report.  If I use
the SAVE option under the regression command, the resultant means are not
21.50 & 30.50, nor is the difference between them equal to 9.00.

I'd also like to generate some measure of effect size, rather than unique
contribution of group membership.  In a couple of the regression
discontinuity papers I have read, the authors seem to taken the
unstandardized beta for group and divided it by some standard deviation
measure.  I assume that the SD is simply coming from a descriptives
procedure and not generated via the regression run.

So... I guess this boils down to two or more questions:

(1) Am I using the correct procedure to generate the means, and do I have
the correct interpretation of the unstandardized beta?

(2) Can I use the unstandardized beta and some measure of the SD to estimate
an effect size.  By which procedure should I generate the SD?

I really appreciate any guidance.

Anton




On Sun, 14 Jan 2007 21:14:25 -0500, Statisticsdoc <[hidden email]>
wrote:

>Stephen Brand
>www.statisticsdoc.com
>
>Anton,
>
>Are you trying to compute the mean test score for the two groups, or the
>mean __predicted__ test score for the two groups?  What you are doing in
the

>spreadsheet will generate predicted scores.  Another option, if your
>interest lies in predicted scores, is to have SPSS save predicted scores
>(see SAVE option under the regression command), and then compute means on
>the predicted scores for the groups.  If you want to know the actual test
>scores for the groups, just run Means.
>
>What sort of effect size are you interested in?  If you want to know the
>unique contribution of group membership, adjusting for the other variables
>in the model, then compare the r-squared of the regression model with and
>without the group membership effect.  The increment in the r-squared
>indicates how much additional variance in the dependent variable is
>accounted for by the inclusion of group membership, over and above the
>variance explained by the other independent variables in the regression
>model.  Please note that the size of the unique effect size is meaningful
>only in the context of the other variables in the regression model.  If you
>have a different set of independent variables, the unique contribution of
>group membership will likely be different.
>
>HTH,
>
>Stephen Brand
>
>
>For personalized and professional consultation in statistics and research
>design, visit
>www.statisticsdoc.com
>
>
>-----Original Message-----
>From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
>Anton
>Sent: Sunday, January 14, 2007 6:37 PM
>To: [hidden email]
>Subject: Obtaining means and effect sizes from a multiple regression run
>
>
>Dear SPSS Listers,
>
>It's been ages since my last statistics course, so please bear with me as I
>struggle to figure out this multiple regression issue.
>
>My main regression run includes a primary independent variable I'll call
>"group" (coded 0 & 1) along with other independent variables.  I am trying
>to predict test score, my primary dependent variable.  In order to get the
>mean test score for the two groups, I have been dumping the unstandardized
>betas along with the independent variable means (and the constant) into to
>Excel, substituting 0 and 1 for the group means, and letting the

spreadsheet
>program do the rest.  Is there any way for SPSS to do some of this for me?
>
>I also need to compute an effect size for group.  Is it okay to take the
>unstandardized beta for group and simply divide by the control group's SD
>obtained from a basic descriptives procedure, or do I need to somehow
obtain
>a SD directly from the regression run.
>
>I really appreciate any guidance.
>
>Anton