Levene test with large sample

[Salbod]

Good Evening Everyone,



I hope someone will guide me around this simple problem that fell in front
of me today.



Which t test do I want to report?



I have two groups:



Group1   n= 1,512       M=2.5668           SD1=.80339

Group2   n=    273       M=2.3187           SD2=.96877



Levene's Test for Equality of Variance F= 27.531, p<.0001.



1) Equal Variances Assumed t(1783) = 4.542, p <.001.



2) Equal Variances NOT Assumed t(342.79) = 3.991.





The large N has created a statistically significance difference out of a
trivial difference; furthermore, the variances between the groups are
statistically different.

Can Levene's result be ignored because of the large sample or is it
recommended that I report t test #2?



I would appreciate any suggestions and, if possible, references on this
problem.



I thank you in advance.



Stephen Salbod, Pace University

[Hector Maletta]

        The main point, I think, is that your conclusion stands either under
the equal variances or the unequal variances hypotheses. This is, of course,
in part due to the fact that your sample is large. And that is a good thing.

        Now, to report your results you may choose one hypothesis or another
about the equality of variances (both leading to the same substantive
conclusion). Since (at your sample size) the variances observed in the
groups lend credence to the different-variances hypothesis, i.e. the
difference in variances is statistically significant, you may want to use
that one for reporting purposes.

        Finding different variances is not altogether surprising. I imagine
your results do not come from any randomized experimental design, but from
some observational study, so it would be mere fluke that your groups show
equal variances. One should rather EXPECT different variances (as one would
rather expect different group sizes) for the various groups.

        Hector

        -----Mensaje original-----
De: SPSSX(r) Discussion [mailto:[hidden email]] En nombre de
Stephen Salbod
Enviado el: 23 March 2007 23:04
Para: [hidden email]
Asunto: Levene test with large sample

        Good Evening Everyone,



        I hope someone will guide me around this simple problem that fell in
front
        of me today.



        Which t test do I want to report?



        I have two groups:



        Group1   n= 1,512       M=2.5668           SD1=.80339

        Group2   n=    273       M=2.3187           SD2=.96877



        Levene's Test for Equality of Variance F= 27.531, p<.0001.



        1) Equal Variances Assumed t(1783) = 4.542, p <.001.



        2) Equal Variances NOT Assumed t(342.79) = 3.991.





        The large N has created a statistically significance difference out
of a
        trivial difference; furthermore, the variances between the groups
are
        statistically different.

        Can Levene's result be ignored because of the large sample or is it
        recommended that I report t test #2?



        I would appreciate any suggestions and, if possible, references on
this
        problem.



        I thank you in advance.



        Stephen Salbod, Pace University

[Ornelas, Fermin]

I think you should report the F-test. Having said that, if you are
concerned with the test results; you should also rely on the traditional
plots of your residuals against the predictors and the fitted response
variable. If the plots show the typical pattern of non constant
variances then the data validates the results of the test. But if these
plots contradict the test results you may want to also add a comment to
your results.

I am coming to the conclusion that the plots tell the story better.
Another possibility for test results to contradict your plots could be
an improper model specification or omission of an important predictor.

Fermin Ornelas, Ph.D.
Management Analyst III, AZ DES
Tel: [phone]
E-mail: [hidden email]


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Stephen Salbod
Sent: Friday, March 23, 2007 3:04 PM
To: [hidden email]
Subject: Levene test with large sample

Good Evening Everyone,



I hope someone will guide me around this simple problem that fell in
front
of me today.



Which t test do I want to report?



I have two groups:



Group1   n= 1,512       M=2.5668           SD1=.80339

Group2   n=    273       M=2.3187           SD2=.96877



Levene's Test for Equality of Variance F= 27.531, p<.0001.



1) Equal Variances Assumed t(1783) = 4.542, p <.001.



2) Equal Variances NOT Assumed t(342.79) = 3.991.





The large N has created a statistically significance difference out of a
trivial difference; furthermore, the variances between the groups are
statistically different.

Can Levene's result be ignored because of the large sample or is it
recommended that I report t test #2?



I would appreciate any suggestions and, if possible, references on this
problem.



I thank you in advance.



Stephen Salbod, Pace University

NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR
CONFIDENTIAL information and is intended only for the use of the
specific individual(s) to whom it is addressed.  It may contain
information that is privileged and confidential under state and federal
law.  This information may be used or disclosed only in accordance with
law, and you may be subject to penalties under law for improper use or
further disclosure of the information in this e-mail and its
attachments. If you have received this e-mail in error, please
immediately notify the person named above by reply e-mail, and then
delete the original e-mail.  Thank you.

[Art Kendall]

I would recommend using the separate variances estimate to report.

The observed difference is not not readily attributable to random
variation among samples from a common pop. i.e. the difference is
"statistically significant".

However, you characterize the difference as trivial, this should also be
reported.

It is difficult to comment further without more information about the
study design, definitions of the DV and IV, and the decision this
analysis was dine in support of.

Art Kendall
Social Research Consultants


Stephen Salbod wrote:

> Good Evening Everyone,
>
>
>
> I hope someone will guide me around this simple problem that fell in front
> of me today.
>
>
>
> Which t test do I want to report?
>
>
>
> I have two groups:
>
>
>
> Group1   n= 1,512       M=2.5668           SD1=.80339
>
> Group2   n=    273       M=2.3187           SD2=.96877
>
>
>
> Levene's Test for Equality of Variance F= 27.531, p<.0001.
>
>
>
> 1) Equal Variances Assumed t(1783) = 4.542, p <.001.
>
>
>
> 2) Equal Variances NOT Assumed t(342.79) = 3.991.
>
>
>
>
>
> The large N has created a statistically significance difference out of a
> trivial difference; furthermore, the variances between the groups are
> statistically different.
>
> Can Levene's result be ignored because of the large sample or is it
> recommended that I report t test #2?
>
>
>
> I would appreciate any suggestions and, if possible, references on this
> problem.
>
>
>
> I thank you in advance.
>
>
>
> Stephen Salbod, Pace University
>
>
>

[Richard Ristow]

At 06:03 PM 3/23/2007, Stephen Salbod wrote:

>Which t test do I want to report?
>
>I have two groups:
>
>Group1   n= 1,512     M=2.5668     SD1=.80339
>Group2   n=   273     M=2.3187     SD2=.96877
>
>Levene's Test for Equality of Variance F= 27.531, p<.0001.
>
>1) Equal Variances Assumed t(1783) = 4.542, p <.001.
>2) Equal Variances NOT Assumed t(342.79) = 3.991.
>
>Can Levene's result be ignored because of the large sample or is it
>recommended that I report t test #2?

As others have noted, the simplest answer is, the test says different
variances, you test assuming different variances. So far, so good,
especially as it doesn't change anything of note.

Now, you wrote,

>The large N has created a statistically significance difference out of
>a trivial difference

That's the more interesting thing going on here: N is large enough that
the old-school 'whether' test (i.e., for a significant difference)
needs to be replaced by 'how much' tests (i.e., confidence intervals).
Perhaps something like that should be done regarding Levene's test:
variances within a certain ratio of each other may be treated as equal,
even if 'significantly' different.

In any case, simply reporting the t-test isn't illuminating, for the
same reason. More like the following (based on generated data whose
population parameters match the estimates you posted):

a. Yes, the between-groups effect is strongly significant (p<.001)
b. However, that is not much of the observed variance (R^2=.011)
c. The best estimate of the difference between groups is
.242 (95% CI, .137 to .346). Whether that's of practical significance,
depends on the study.

SPSS 15 draft output, heavily edited to shorten lines:


UNIANOVA
   OBSERVE  BY GROUP
   /METHOD = SSTYPE(3)
   /INTERCEPT = INCLUDE
   /PRINT = DESCRIPTIVE OPOWER PARAMETER HOMOGENEITY
   /CRITERIA = ALPHA(.05)
   /DESIGN = GROUP .


Univariate Analysis of Variance
|-----------------------------|---------------------------|
|Output Created               |26-MAR-2007 18:27:30       |
|-----------------------------|---------------------------|
[Salbod]

Between-Subjects Factors [suppressed - see descriptives]

Descriptive Statistics
Dependent Variable: OBSERVE
|-----|------|--------------|----|
|GROUP|Mean  |Std. Deviation|N   |
|-----|------|--------------|----|
|1    |2.5724|.77670        |1512|
|2    |2.3308|.96286        |273 |
|-----|------|--------------|----|
|Total|2.5355|.81232        |1785|
|-----|------|--------------|----|

Levene's Test of Equality of Error Variances(a)
Dependent Variable: OBSERVE
|------|---|----|----|
|F     |df1|df2 |Sig.|
|------|---|----|----|
|18.996|1  |1783|.000|
|------|---|----|----|
Tests the null hypothesis that the error variance of the
dependent variable is equal across groups.
a Design: Intercept+GROUP

Tests of Between-Subjects Effects
Dependent Variable: OBSERVE
|----------|-----------|----|--------|--------|----|--------|---------|
|Source    |Type III   |df  |Mean    |F       |Sig.|Noncent.|Observed |
|          |Sum of     |    |Square  |        |    |Param.  |Power(a) |
|          |Squares    |    |        |        |    |        |         |
|----------|-----------|----|--------|--------|----|--------|---------|
|Corrected |13.498(b)  |1   |13.498  |20.682  |.000|20.682  |.995     |
|Model     |           |    |        |        |    |        |         |
|----------|-----------|----|--------|--------|----|--------|---------|
|Intercept |5559.515   |1   |5559.515|8518.133|.000|8518.133|1.000    |
|----------|-----------|----|--------|--------|----|--------|---------|
|GROUP     |13.498     |1   |13.498  |20.682  |.000|20.682  |.995     |
|----------|-----------|----|--------|--------|----|--------|---------|
|Error     |1163.707   |1783|.653    |        |    |        |         |
|----------|-----------|----|--------|--------|----|--------|---------|
|Total     |12652.135  |1785|        |        |    |        |         |
|----------|-----------|----|--------|--------|----|--------|---------|
|Corrected |1177.206   |1784|        |        |    |        |         |
|Total     |           |    |        |        |    |        |         |
|----------|-----------|----|--------|--------|----|--------|---------|
a Computed using alpha = .05
b R Squared = .011 (Adjusted R Squared = .011)



Parameter Estimates
Dependent Variable: OBSERVE
|---------|-----------|------|-------|------|----------------|----------|---------|
|Parameter|B          |Std.  |t      |Sig.  |95%
Confidence  |Noncent.  |Observed |
|         |           |Error |       |      |Interval        |Parameter
|Power(a) |
|
|-----------|------|-------|------|--------|-------|----------|---------|
|         |           |      |       |      |Lower   |Upper  |
|         |
|         |           |      |       |      |Bound   |Bound  |
|         |
|---------|-----------|------|-------|------|--------|-------|----------|---------|
|Intercept|2.331      |.049  |47.669
|.000  |2.235   |2.427  |47.669    |1.000    |
|---------|-----------|------|-------|------|--------|-------|----------|---------|
|[GROUP=1]|.242       |.053  |4.548  |.000  |.137    |.346   |4.548
|.995     |
|---------|-----------|------|-------|------|--------|-------|----------|---------|
|[GROUP=2]|0(b)       |.     |.      |.     |.       |.      |.
|.        |
|---------|-----------|------|-------|------|--------|-------|----------|---------|
a Computed using alpha = .05
b This parameter is set to zero because it is redundant.



===================
APPENDIX: Test data
===================
Since this is randomly-generated data, a separate run will generally
yield somewhat different results. Yeah, I could have put in a SET SEED.

NEW FILE.
INPUT PROGRAM.
.  NUMERIC SAMPLE   (F3)
           /GROUP    (F2)
           /OBSERVE  (F5.2).
.  LEAVE   SAMPLE
            GROUP.

*  Characteristics of the generated sample  ......... .
.  COMPUTE #N1    = 1512       /* N,        group 1 */.
.  COMPUTE #MEAN1 =    2.5668  /* Mean,     group 1 */.
.  COMPUTE #SD1   =     .80339 /* Std dev., group 1 */.
.  COMPUTE #N2    =  273       /* N,        group 2 */.
.  COMPUTE #MEAN2 =    2.3187  /* Mean,     group 2 */.
.  COMPUTE #SD2   =     .96877 /* Std dev., group 2 */.

.  LOOP   SAMPLE   = 1 TO 1.
.     COMPUTE     GROUP   = 1.
.     LOOP #OBSNUM = 1 TO #N1.
.        COMPUTE  OBSERVE = RV.NORMAL(#MEAN1,#SD1).
.        END CASE.
.     END LOOP.

.     COMPUTE     GROUP   = 2.
.     LOOP #OBSNUM = 1 TO #N2.
.        COMPUTE  OBSERVE = RV.NORMAL(#MEAN2,#SD2).
.        END CASE.
.     END LOOP.
.  END LOOP.


END FILE.
END INPUT PROGRAM.
DATASET NAME Salbod WINDOW=Front.