Difference Score as a dependent variable

[Laura2014]

Hi,

I am working on a study in which we test the effect of a behavioral variable X on a difference score variable Y. The difference score is build out of two distinct factors (Ya - Yb). However I know scholars such as Edwards strongly recommend to not just compute in SPPS the dependent variable as Ya-Yb.  
I found several alternatives for using difference scores such as polynomial regression. But from my understanding, techniques such as polynomial regression can only be used when the difference score is your independent variable.

Besides testing the effect of X on Ya and Yb separatly, I was wondering whether anyone has other suggestions to address the difference score? (which can be performed in SPSS or AMOS)?

Thanks!

Laura

[Art Kendall]

Does the design include a pre-test and a post-test?  Are you looking as a difference in change question?
If so one approach is to use a stepped (NOT stepwise) ordinary regression approach with the pre-test as a predictor, so that the DV on the second step is effectively the residual. Another is a repeated measures ANOVA. (ANCOVA would not allow you to test the interaction of the pre-test with the other predictors.)

You can use many instances of the general linear model -- ordinary regression, ANOVA, etc. depending on what your other predictors are. List members would need more details about your design, number of cases, etc. to be more specific.

I don't have the citations at hand, but someone on this list most likely does.
On 4/16/2014 7:11 PM, Laura2014 [via SPSSX Discussion] wrote:

Hi,

I am working on a study in which we test the effect of a behavioral variable X on a difference score variable Y. The difference score is build out of two distinct factors (Ya - Yb). However I know scholars such as Edwards strongly recommend to not just compute in SPPS the dependent variable as Ya-Yb.  
I found several alternatives for using difference scores such as polynomial regression. But from my understanding, techniques such as polynomial regression can only be used when the difference score is your independent variable.

Besides testing the effect of X on Ya and Yb separatly, I was wondering whether anyone has other suggestions to address the difference score? (which can be performed in SPSS or AMOS)?

Thanks!

Laura

---

If you reply to this email, your message will be added to the discussion below:

http://spssx-discussion.1045642.n5.nabble.com/Difference-Score-as-a-dependent-variable-tp5725496.html

To start a new topic under SPSSX Discussion, email [hidden email]
To unsubscribe from SPSSX Discussion, click here.
NAML

[Andy W]

I agree with Art in that much more detail is needed to give useful advice. IMO general "how should I model this" questions aren't a great fit for a list-serve like this - it should be based on the consultation of domain experts in how you set up your study and your models. This is hard to give advice given the limited context of a list-serve. An email can't replace an academic advisor.

I don't feel too guilty throw pointing towards references. One of my favorites on the subject is

Allison, Paul D. 1990. Change scores as dependent variables in regression analysis. Sociological Methodology 20:93-114. Public PDF version.

The question comes up alot on the cross validated forum, here are a few questions relevant to the discussion (the second I have a quick run down of the Allison article):

 - Best practice when analysing pre-post treatment-control designs
 - Is it valid to include a baseline measure as control variable when testing the effect of an independent variable on change scores?

Given your goal I would say ignore the Edward's article. The fundamental problems with using change scores as independent variables are different. (I have a basic 100 word recap of the fundamental argument in the Edward's paper in this question on Cross Validated, Is it valid to use a difference score as an independent variable in a regression analysis.) The citation for the Edward's article for those interested is

Edwards, Jeffrey R. 2001. Ten difference score myths. Organizational Research Methods 4(3):265-287. Public PDF.

[Laura2014]

Thank you all for the information!
We conducted a cross-sectional survey which tests the influence of increased media use on an objectified self-concept in 244 respondents. The difference score variable "objectified self-concept" reflects whether someone considers his/her own appearance as more important than his/her physical competence. The scale includes 12items: 5 items on level of importance attached to appearance and 7 items on level of importance attached to competence. Objectified self-concept is now calculated by subtracting mean score of appearance from mean score of competence.
So the independent variable in our cross-sectional study is frequency of media use and our dependent variable is objectified self-concept = a difference score variable.
Because the design is cross-sectional (and not experimental), I do not have a pretest score which all the techniques for handling difference scores I know refer to?
Thanks!

[Bruce Weaver]

Laura, thanks for providing more info.  One more question though:  How is "frequency of media use" measured?  

Laura2014 wrote

Thank you all for the information!
We conducted a cross-sectional survey which tests the influence of increased media use on an objectified self-concept in 244 respondents. The difference score variable "objectified self-concept" reflects whether someone considers his/her own appearance as more important than his/her physical competence. The scale includes 12items: 5 items on level of importance attached to appearance and 7 items on level of importance attached to competence. Objectified self-concept is now calculated by subtracting mean score of appearance from mean score of competence.
So the independent variable in our cross-sectional study is frequency of media use and our dependent variable is objectified self-concept = a difference score variable.
Because the design is cross-sectional (and not experimental), I do not have a pretest score which all the techniques for handling difference scores I know refer to?
Thanks!

[Laura2014]

The media variable is "watching reality TV" and is measured by asking respondents to indicate on a 5-point scale (ranging from never to every day), how often they watch reality TV?
Thanks!

[Bruce Weaver]

Thanks for clarifying.  Following up on Art K's earlier post, One option would be a 5 (between-Ss) x 2 (Within-Ss) mixed design ANOVA.  The "reality TV" question gives you 5 independent groups.  The 2 paired scores are the means of the appearance and competence items respectively.  The model term of most interest to you is the interaction term.  The F-test for the interaction tests the null hypothesis that the difference between appearance & competence means is the same for all 5 "reality TV" groups.  Given that the groups are ordered, the default polynomial contrasts you get (with GLM) would be appropriate for the main effect of Group and for the interaction term.  

Note that if you ran a one-way ANOVA (with the 5 groups) using the difference scores as the outcome variable, the F-test for Group in that one-way ANOVA would be identical to the F-test for the interaction in the mixed design ANOVA described earlier.  This makes sense, because the null hypothesis for the one-way ANOVA is that the differences are not dependent on group.

HTH.

Laura2014 wrote

The media variable is "watching reality TV" and is measured by asking respondents to indicate on a 5-point scale (ranging from never to every day), how often they watch reality TV?
Thanks!

[Rich Ulrich]

If you want to know how Ya and Yb relate to X, you can look at the
canonical correlation equations; potentially, these give you coefficients
that go in both directions.  The weights for Ya and Yb, predicting X,
will show whether the best *statistical* combination is a sum or a
difference, in what proportions.  If you only want to see the F-test, you
can put this up as the simple multiple regression, placing X in the usual
position of the IV.   This gives a good test, because the multiple R says
everything there is to know about the magnitude of that inter-relation.

For the matrix-notated, Y= X + E, and if you look at the regression analog,
you have the two Y terms being predicted by several X terms.  This is
sometimes called "multivariate multiple regression".  If you solve, instead,
what I described above, X= Y + E, you get slightly different coefficients. An
example of MMR using GLM is at
www.ats.ucla.edu/stat/spss/whatstat/whatstat.htm
You might try MR for the test, and then MMR in each direction, to figure
out what is being reported by MMR.

Canonical correlation is the generic term that can include MANOVA and
multiple-group discriminant function, with multiple regression and
ANOVA as simple cases that have a single variable (1 d.f.) as outcome.



CONFUSION.
In my subscription to the list, I see 3 threads following this original post.

On Nabble, I see several posts by the OP that are not yet "accepted by the list."

In the discussion, I see comments directed to "change scores,"
which are a specific and largely irrelevant sort of difference.  The critiques
of change scores are seldom applicable to simple composite scores that
happen to be differences.

I see "media use" which turns out to be "frequency of watching
Reality TV" ... which seems to be an odd choice of labeling, since
I would have assumed that this particular term invoked something
about "Facebook, etc.,  participation."

I see the use of "self-concept" in the description of the variables.

Self-concept has always been a tricky and nasty category to play with. 
This study applies the term to *views* of what is important, rather than
to self-regard with its own components.  My conclusion is that the study's
conclusions will be hard for readers to accept, whatever they are, owing to 
loose conceptualization of the terms.  And loose relation of the terms to
the actual items collected.

--
Rich Ulrich

> Date: Wed, 16 Apr 2014 16:11:17 -0700

> From: [hidden email]
> Subject: Difference Score as a dependent variable
> To: [hidden email]
>
> Hi,
>
> I am working on a study in which we test the effect of a behavioral variable
> X on a difference score variable Y. The difference score is build out of two
> distinct factors (Ya - Yb). However I know scholars such as Edwards strongly
> recommend to not just compute in SPPS the dependent variable as Ya-Yb.
> I found several alternatives for using difference scores such as polynomial
> regression. But from my understanding, techniques such as polynomial
> regression can only be used when the difference score is your independent
> variable.
>
> Besides testing the effect of X on Ya and Yb separatly, I was wondering
> whether anyone has other suggestions to address the difference score? (which
> can be performed in SPSS or AMOS)?
>
> Thanks!
>
> Laura
>
>
>
> --
> View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Difference-Score-as-a-dependent-variable-tp5725496.html
> Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD

[Laura2014]

Thank you! This is really helpful!