I recall learning a while back that when comparing two groups, Welch's T is
based upon df using Satterthwaite; SPSS MIXED employs a Satterthwaite
approximation to calculate df all the time, AFAIA. While it would be nice
if the SPSS MIXED procedure offered other methods for calculating df,
I'm not terribly concerned because the Satterthwaite approximation is
intended to handle both balanced and unbalanced designs. But I digress.
As was discussed previously in this thread, with only two groups, Welch and
B-F are equivalent, so as I demonstrated in my previous post, the results
all around are identical (Welch=B-F=MIXED). I was not surprised because
both (Welch and MIXED) claim to use the same method for calculating df, at
least in name.
@Bruce: You actually demonstrated that they were equivalent in a
previous discussion on SPSS-L:
http://spssx-discussion.1045642.n5.nabble.com/Solving-heterogeneous-variance-using-mixed-linear-models-td5652270.html#a5654365However, we never examined if they would be *exactly* the same when we went
beyond two groups. As a mixed modeler, frankly, it was not something of
interest to me. Once I figured out how to allow for heterogeneous variances
across independent groups via MIXED, I didn't look back.
HOWEVER, to be certain there isn't anything strange going on with SPSS
MIXED, I just decided to employ the same model using SAS:
proc mixed data=mydata;
class group;
model y = group / s ddfm=satterthwaite;
repeated / group=group;
run;
The df, F, and p are identical to those produced by the SPSS MIXED
procedure. Now we can rule out that there is something unique to the SPSS
mixed procedure. By the way, the parameterization above is standard
practice among SAS mixed modelers to test for inequalities in means across
independent groups with unequal variances. Unfortunately, the current
version of SPSS does not offer the GROUP= option, so I had to devise a
different way to obtain the same parameterization, as I showed previously.
Bottom line: When more than two groups are being compared, I do not know
how the B-F on inequality in means test is calculating df.
@Bruce: I would *expect* Welch's df to be fairly close to the MIXED df,
perhaps even closer to the MIXED df than the B-F df. Looking at your
example (I can't find the data file), could you tell us what Welch's df is?
Ryan
On Thu, May 16, 2013 at 10:46 AM, Bruce Weaver <
[hidden email]>wrote:
> Earlier this morning, before I saw Ryan's example below, I was working on
> one
> of my own in an attempt to understand the use of /REPEATED when there are
> no
> repeated measures. (That was a new one to me -- thanks for pointing it
> out,
> Ryan.)
>
> For some reason, the denominator df from MIXED for my example do not equal
> the B-F df from ONEWAY. I can't see where I'm doing anything differently
> than in Ryan's example. Can anyone spot what the problem/difference is?
> (Maybe I should ask first if anyone can duplicate the results I get. I'm
> using v20 with patch running under Windoze 7.)
>
>
> * Modify the path below if necessary.
> GET FILE='C:\SPSSdata\1991 U.S. General Social Survey.sav'.
>
> ONEWAY age BY race
> /STATISTICS DESCRIPTIVES BROWNFORSYTHE WELCH
> .
>
> * Replicate the standard equal variances F-test via MIXED.
> MIXED age BY race
> /FIXED=race | SSTYPE(3)
> /PRINT=SOLUTION
> .
>
> * Now allow unequal variances via /REPEATED sub-command.
> COMPUTE subject = $casenum.
> MIXED age BY race
> /FIXED=race | SSTYPE(3)
> /PRINT=SOLUTION
> /REPEATED=race | SUBJECT(subject) COVTYPE(DIAG)
> .
>
> * B-F df2 from ONEWAY = 217.762 .
> * df2 from MIXED = 98.658 .
> * Why are they not the same?.
>
>
>
> Ryan Black wrote
> > Diana,
> >
> > This is not a "clever ploy." Your understanding of the diagonal
> > covariance
> > matrix I specified is incorrect. The model I specified tests for mean
> > differences between k independent groups, while allowing for
> > group-specific
> > variances. Below is an illustration. Compare the test statistic, df, and
> > p-value from B-F and WELCH's tests to the same statistics produced by the
> > MIXED procedure. They are identical. Further, look at the covariance
> > parameters estimated from the MIXED model. How do those group-specific
> > variances compare to the group-specific variances calculated by EXAMINE
> > VARIABLES? I'll tell you. They are identical.
> >
> > Ryan
> >
> >
> > *Generate data.
> > set seed 98788978.
> > new file.
> > input program.
> > loop subject= 1 to 100.
> > if (subject>=1 or subject<=50) group=1.
> > if (subject>50) group=2.
> > compute y = 50*(group=1) + 45*(group=2) +
> > sqrt(10)*rv.normal(0,1)*(group=1) + sqrt(20)*rv.normal(0,1)*(group=2).
> > end case.
> > end loop.
> > end file.
> > end input program.
> > execute.
> >
> >
> > EXAMINE VARIABLES=y BY group
> > /PLOT NONE
> > /STATISTICS DESCRIPTIVES
> > /CINTERVAL 95
> > /MISSING LISTWISE
> > /NOTOTAL.
> >
> >
> > ONEWAY y BY group
> > /STATISTICS BROWNFORSYTHE WELCH.
> >
> >
> > MIXED y BY group
> > /FIXED=group | SSTYPE(3)
> > /REPEATED=group | SUBJECT(subject) COVTYPE(DIAG).
> >
> >
> > On Thu, May 16, 2013 at 8:27 AM, Kornbrot, Diana
> > <
>
> > d.e.kornbrot@.ac
>
> > >wrote:
> >
> >> This is a clever ploy of Ryan’s, I also thought of it
> >> BUT it only works if the predictor is repeated
> >> Obviously, if all predictors are between there is no covariance matirx
> to
> >> calculate
> >> There could be unequal variances, but that option is not available
> >> Best
> >> Diana
> >>
> >>
> >>
> >> On 16/05/2013 00:34, "Ryan Black" <
>
> > ryan.andrew.black@
>
> > > wrote:
> >>
> >> Let's not leave the MIXED procedure out of this conversation! :-)
> >>
> >> The MIXED procedure can be used to test differences in means between k
> >> groups while accounting for heterogeneous variances, as shown below:
> >>
> >> MIXED y BY group
> >> /FIXED=group | SSTYPE(3)
> >> /PRINT=SOLUTION
> >> /REPEATED=group | SUBJECT(subject) COVTYPE(DIAG).
> >>
> >> Ryan
> >>
> >>
> >>
> >> On Wed, May 15, 2013 at 5:35 PM, Bruce Weaver <
>
> > bruce.weaver@
>
> > >
> >> wrote:
> >>
> >> If you wish to replicate Mike's analysis, run the following:
> >>
> >> MATRIX DATA VARIABLES=Race ROWTYPE_ RT_diff /FACTORS=Race.
> >> BEGIN DATA
> >> 0 N 5
> >> 1 N 172
> >> 0 MEAN 202.8
> >> 1 MEAN 194.1919
> >> 0 STDDEV 99.64788
> >> 1 STDDEV 359.55507
> >> END DATA.
> >>
> >> ONEWAY RT_diff BY Race
> >> /MATRIX=IN(*)
> >> /STATISTICS DESCRIPTIVES BROWNFORSYTHE WELCH
> >> .
> >>
> >> --------------------------------------------------------------
> >>
> >> p.s. - Jon, the example of using MATRIX DATA with ONEWAY in the v20 FM
> >> shows
> >> ONEWAY specifying the range of values for the Group variable. I.e.,
> >>
> >> ONEWAY WELL BY EDUC*(1,6)* /MATRIX=IN(*).
> >>
> >> When I tried that with the example shown above, it ran, but I got a
> >> warning
> >> message:
> >>
> >> *Warnings*
> >> Range specifications are no longer honored. All values have been used.
> To
> >> select a range of values, use the FILTER subcommand.
> >>
> >> It's not a big deal, but maybe something to add to the list of edits for
> >> the
> >> FM.
> >>
> >>
> >>
> >> Mike Palij wrote
> >> > Okay, this is getting out of hand. Ryan Black below cites a source
> for
> >> > a Brown-Forsythe test of VARIANCES which does not appear in SPSS.
> >> > Just check any manual on algorithms. Black's source provides
> >> equivalent
> >> > information to that Glass & Hopkins (3e) provide in their section
> 16.10
> >> in
> >> > their chapter on "inferences about variances".
> >> >
> >> > Since Diana Kornbrot has not been forthcoming with additional details,
> >> > let me use some of my own data that seem to emulate her situation.
> >> > First, here are the descriptive statistics for a Reaction Time (RT)
> >> > difference
> >> > for the two groups in the variable "race.2": (Total N=177)
> >> >
> >> > Report
> >> >
> >> > RT_diff
> >> >
> >> > race.2 Mean N Std. Deviation
> >> >
> >> > 0.00 202.8000 5 99.64788
> >> >
> >> > 1.00 194.1919 172 359.55507
> >> >
> >> > Total 194.4350 177 354.73208
> >> >
> >> >
> >> > The standard deviation of Group1 is about 3.5 times that of Group 0.
> >> >
> >> > Next, the ANOVA table:
> >> >
> >> > ANOVA
> >> >
> >> > RT_diff
> >> >
> >> > ___________________SS____df____Mean Square____F_____Sig.
> >> >
> >> > Between_______360.034_____1_______360.034___.003____.958
> >> >
> >> > Within___22146573.469___175______26551.848
> >> >
> >> > Total_22146933.503 176
> >> >
> >> > Finally, the Robust Tests of Equality of Means:
> >> >
> >> > Robust Tests of Equality of Means
> >> >
> >> > RT_diff
> >> >
> >> > ______________Statistica____df1_____df2___Sig.
> >> >
> >> > Welch___________.027_________1____7.575___.874
> >> >
> >> > Brown-Forsythe__.027_________1____7.575___.874
> >> >
> >> > a. Asymptotically F distributed.
> >> >
> >> >
> >> > Note that the df-Within from the ANOVA is 172 while df2 from
> >> > the Robust Means tests is 7.575 for both the Welch and Brown-Forsythe
> >> > tests. This, I believe, replicates the Kornbrot situation.
> >> >
> >> > -Mike Palij
> >> > New York University
> >>
> >> > mp26@
> >>
> >> >
> >> > ----- Original Message -----
> >> > From: Ryan Black
> >> > To:
> >>
> >> > SPSSX-L@.UGA
> >>
> >> >
> >> > Sent: Wednesday, May 15, 2013 3:31 PM
> >> > Subject: Re: Brown_Forsythe error in 1-way ANOVA?
> >> >
> >> >
> >> > No time to examine closely, but this textbook demonstrates what
> >> appears
> >> > to be a fairly straightforward way for conducting the B-F test in
> SPSS:
> >> >
> >> >
> >> >
> >>
>
http://books.google.com/books?id=a-9m55d_uE0C&pg=PA226&lpg=PA226&dq=brown-forsythe+variance+test+spss&source=bl&ots=8DiDbHL4-D&sig=DBsbbzKMfDq8o6uScTphzJ1ZOak&hl=en&sa=X&ei=7t-TUaWqMvfe4APK8YCQBA&ved=0CD0Q6AEwAw#v=onepage&q=brown-forsythe%20variance%20test%20spss&f=false> >> >
> >> > The author (Andrew Hayes) goes on to state that this test is
> >> technically
> >> > a test on "equality of dispersion."
> >> >
> >> > HTH.
> >> >
> >> > Ryan
> >> >
> >> >
> >> >
> >> >
> >> > On Wed, May 15, 2013 at 2:25 PM, Bruce Weaver <
> >>
> >> > bruce.weaver@
> >>
> >> > > wrote:
> >> >
> >> > Diana, if you use ONEWAY's /MATRIX=OUT sub-command, you could
> share
> >> > the data
> >> > in a way that allows others to run the analysis for themselves
> >> (using
> >> > /MATRIX = IN) to see what's going on. For example:
> >> >
> >> > ONEWAY Y BY Group
> >> > /STATISTICS BROWNFORSYTHE WELCH
> >> > /MATRIX=OUT(*) .
> >> > LIST.
> >> >
> >> > The resulting file has variables ROWTYPE_, Group, VARNAME_ and Y;
> >> and
> >> > the
> >> > row types are MEAN, STDDEV and N. (So if you post the data to let
> >> > others
> >> > run the analysis, you are posting summary statistics for the
> groups
> >> > only,
> >> > not raw data.)
> >> >
> >> > To run the same analysis with that dataset (rather than the raw
> >> data)
> >> > as
> >> > input:
> >> >
> >> > ONEWAY Y BY Group
> >> > /STATISTICS BROWNFORSYTHE WELCH
> >> > /MATRIX=IN(*) .
> >> >
> >> > HTH.
> >> >
> >> >
> >> >
> >> > Kornbrot, Diana wrote
> >> >
> >> > > Dear SPSS experts
> >> > > Please help with this problem
> >> > >
> >> > > Most sources give same denominator df for Brown-Forsyth as for
> >> equal
> >> > > variance ANOVA
> >> > > Not SPSS, no idea how 170 for equal variance got reduced to 6.97
> >> > > Furthermore, the Brown-Forsyth significance given by SPSS does
> >> not
> >> > agree
> >> > > with either that corresponding to its own, now with that from
> df2
> >> =
> >> > N - k
> >> > > In this study, n1 =7, n2 = 165and there are 2 groups, K=2
> >> > > Here are results:
> >> > > F df1
> >> df2
> >> > > sigSPSS sigdfA sigdfB
> >> > > predictor Equal variance .697 1 170 . 4051
> >> > .4051
> >> > > predictor Brown-Forsythe .909 1 6.697 .3736
> >> > .3418
> >> > > .3773
> >> > >
> >> > > sigSPSS is significance in SPSS output
> >> > > SigdfA is significance using df = n1+n2 - k =7+165-2=170
> >> > > SigdfB is significance using df in SPSS Brown-Forsyth
> >> > >
> >> > > Any ideas?
> >> > > Best
> >> > > Diana
> >> > >
> >> > > NB I know this is very unbalanced design, but problem occurs
> with
> >> > more
> >> > > balanced designs.
> >> > > Nbt2 cited df is NOT the harmonic mean of n1, n2
> >> > > ________________________________
> >> > > Emeritus Professor Diana Kornbrot
> >> > > email:
> >> >
> >> >
> >> > > d.e.kornbrot@.ac
> >> >
> >> >
> >> > > web:
http://dianakornbrot.wordpress.com/> >> > > Work
> >> > > Department of Psychology
> >> > > School of Life and Medical Sciences
> >> > > University of Hertfordshire
> >> > > College Lane, Hatfield, Hertfordshire AL10 9AB, UK
> >> > > voice: +44 (0) 170 728
> >> 4626<tel:%2B44%20%280%29%20170%20728%204626>
> >> > > Home
> >> > > 19 Elmhurst Avenue
> >> > > London N2 0LT, UK
> >> > > voice: +44 (0) 208 444
> >> 2081<tel:%2B44%20%280%29%20208%20444%202081>
> >> > > mobile: +44 (0) 740 318
> >> 1612<tel:%2B44%20%280%29%20740%20318%201612>
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > -----
> >> > --
> >> > Bruce Weaver
> >> >
> >>
> >> > bweaver@
> >>
> >> >
http://sites.google.com/a/lakeheadu.ca/bweaver/> >> >
> >> > "When all else fails, RTFM."
> >> >
> >> > NOTE: My Hotmail account is not monitored regularly.
> >> > To send me an e-mail, please use the address shown above.
> >> >
> >> > --
> >> > View this message in context:
> >> >
> >>
>
http://spssx-discussion.1045642.n5.nabble.com/Brown-Forsythe-error-in-1-way-ANOVA-tp5720253p5720264.html> >> > Sent from the SPSSX Discussion mailing list archive at Nabble.com.
> >> >
> >> > =====================
> >> > To manage your subscription to SPSSX-L, send a message to
> >> >
> >>
> >> > LISTSERV@.UGA
> >>
> >> > (not to SPSSX-L), with no body text except the
> >> > command. To leave the list, send the command
> >> > SIGNOFF SPSSX-L
> >> > For a list of commands to manage subscriptions, send the command
> >> > INFO REFCARD
> >>
> >>
> >>
> >>
> >>
> >> -----
> >> --
> >> Bruce Weaver
> >>
>
> > bweaver@
>
> >>
http://sites.google.com/a/lakeheadu.ca/bweaver/> >>
> >> "When all else fails, RTFM."
> >>
> >> NOTE: My Hotmail account is not monitored regularly.
> >> To send me an e-mail, please use the address shown above.
> >>
> >> --
> >> View this message in context:
> >>
>
http://spssx-discussion.1045642.n5.nabble.com/Brown-Forsythe-error-in-1-way-ANOVA-tp5720253p5720269.html> >> Sent from the SPSSX Discussion mailing list archive at Nabble.com.
> >>
> >> =====================
> >> To manage your subscription to SPSSX-L, send a message to
> >>
>
> > LISTSERV@.UGA
>
> > (not to SPSSX-L), with no body text except the
> >> command. To leave the list, send the command
> >> SIGNOFF SPSSX-L
> >> For a list of commands to manage subscriptions, send the command
> >> INFO REFCARD
> >>
> >>
> >>
> >>
> >> ------------------------------
> >> Emeritus Professor Diana Kornbrot
> >> email:
>
> > d.e.kornbrot@.ac
>
> >>
> >> web:
http://dianakornbrot.wordpress.com/> >> *Work
> >> *Department of Psychology
> >> School of Life and Medical Sciences
> >> University of Hertfordshire
> >> College Lane, Hatfield, Hertfordshire AL10 9AB, UK
> >> voice: +44 (0) 170 728 4626
> >> *Home
> >> *19 Elmhurst Avenue
> >> London N2 0LT, UK
> >> voice: +44 (0) 208 444 2081
> >> mobile: +44 (0) 740 318 1612
> >>
> >>
> >>
>
>
>
>
>
> -----
> --
> Bruce Weaver
>
[hidden email]>
http://sites.google.com/a/lakeheadu.ca/bweaver/>
> "When all else fails, RTFM."
>
> NOTE: My Hotmail account is not monitored regularly.
> To send me an e-mail, please use the address shown above.
>
> --
> View this message in context:
>
http://spssx-discussion.1045642.n5.nabble.com/Brown-Forsythe-error-in-1-way-ANOVA-tp5720253p5720289.html> Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
>
[hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>