SPSSX Discussion

Comparing mean of squares for many group configurations

Classic

List

Threaded

16 messages Options

Empi

Comparing mean of squares for many group configurations

Dear all,

I’d like to compare the between- and within-group mean of squares for a
continuous dependent variable. However, I face the following problem: what I
need to do is to compare one specific group (the reference group) with all
remaining other groups. However, the categorical grouping variable/the
factor comprises of many groups (k=25). This means that I would need to
conduct e.g. 24 separate ANOVAS. Possible, but not very effective. I spent
the last afternoon trying to apply ‚do repeat‘ and ‚loop‘ commands, but
didn’t achieve any progress. Thus, I am curious how you would approach this
problem?
Best,
E.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

spss.giesel@yahoo.de

Re: Comparing mean of squares for many group configurations

Maybe you show the code you tried?

Mario Giesel
Munich, Germany
---

Am Freitag, 23. Oktober 2020, 17:19:30 MESZ hat Empi <[hidden email]> Folgendes geschrieben:

Dear all,

I’d like to compare the between- and within-group mean of squares for a

continuous dependent variable. However, I face the following problem: what I

need to do is to compare one specific group (the reference group) with all

remaining other groups. However, the categorical grouping variable/the

factor comprises of many groups (k=25). This means that I would need to

conduct e.g. 24 separate ANOVAS. Possible, but not very effective. I spent

the last afternoon trying to apply ‚do repeat‘ and ‚loop‘ commands, but

didn’t achieve any progress. Thus, I am curious how you would approach this

problem?

Best,

Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================

To manage your subscription to SPSSX-L, send a message to

[hidden email] (not to SPSSX-L), with no body text except the

command. To leave the list, send the command

SIGNOFF SPSSX-L

For a list of commands to manage subscriptions, send the command

INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

Art Kendall

Re: Comparing mean of squares for many group configurations

In reply to this post by Empi

I am not sure what your question is.
there is one IV -- "Group" with 25 values.
There is one DV.

Are you asking whether group 1 differs from ANY of the 24? Whether group 1
differs from EACH AND EVERY of the other? Whether group1 differs from the
set of 24 others? You have multiple continuous variables ==> What
distinguishes group 1 (discriminat type of question)?

A substantive description of your grouping variable and your DV(s) might
help list members help you.

-----
Art Kendall
Social Research Consultants
--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Art Kendall
Social Research Consultants

Jon Peck

Re: Comparing mean of squares for many group configurations

In reply to this post by Empi

Maybe use the ONEWAY or UNIANOVA post-hoc tests for this (LSD doesn't do a multiple test correction IIRC).

Or, if you want to create a bunch of ANOVAs against dichotomies formed from each category against just one reference category,

a small dose of Python code could enumerate the categories, do a transformation, and run the ANOVA in a loop.

You can't put procedures inside a DO REPEAT or LOOP, because those are for transformations, but the Python code could easily handle a loop.

On Fri, Oct 23, 2020 at 9:19 AM Empi <[hidden email]> wrote:

Dear all,

I’d like to compare the between- and within-group mean of squares for a
continuous dependent variable. However, I face the following problem: what I
need to do is to compare one specific group (the reference group) with all
remaining other groups. However, the categorical grouping variable/the
factor comprises of many groups (k=25). This means that I would need to
conduct e.g. 24 separate ANOVAS. Possible, but not very effective. I spent
the last afternoon trying to apply ‚do repeat‘ and ‚loop‘ commands, but
didn’t achieve any progress. Thus, I am curious how you would approach this
problem?
Best,
E.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Jon K Peck
[hidden email]

Mike

Re: Comparing mean of squares for many group configurations

It would seem that the OP wants to do Dunnett's test -- see Wikipedia for

one description of the test: https://en.wikipedia.org/wiki/Dunnett's_test

Other sources provide additional info (search Google scholar for refs).

Basically, one defines a reference group/category and this is compared

with all other groups/categories.

UNIANOVA allows one to do this test with a specification in /POSTHOC.

One could also specify paired comparisons between the reference group

and the other groups and used an adjusted p-value per test (i.e., say

usine alpha-overall = .05 and dividing by the number of tests, e.g., .05/24;

this would be an ordinary Bonferroni correction for the number of tests --

given the large number of tests, the per comparison alpha would be

very small and one would probably use the Sidak correction to reduce

the penalty).

The LSD or Least Significant Difference test is just multiple t-tests

each at a conventional alpha (sat, .05). It has been referred to as

a "protected t-test" procedure because one requires a significant

ANOVA F to use this test and, theoretically, a significant F value

implies that at least one comparison between/among means (i.e.,

simple or complex comparisons) is statistically significant, however,

it may not be the one that one is interested in.

-Mike Palij

New York University

[hidden email]

On Fri, Oct 23, 2020 at 11:39 AM Jon Peck <[hidden email]> wrote:

Maybe use the ONEWAY or UNIANOVA post-hoc tests for this (LSD doesn't do a multiple test correction IIRC).

Or, if you want to create a bunch of ANOVAs against dichotomies formed from each category against just one reference category,
a small dose of Python code could enumerate the categories, do a transformation, and run the ANOVA in a loop.

You can't put procedures inside a DO REPEAT or LOOP, because those are for transformations, but the Python code could easily handle a loop.

On Fri, Oct 23, 2020 at 9:19 AM Empi <[hidden email]> wrote:
Dear all,

I’d like to compare the between- and within-group mean of squares for a
continuous dependent variable. However, I face the following problem: what I
need to do is to compare one specific group (the reference group) with all
remaining other groups. However, the categorical grouping variable/the
factor comprises of many groups (k=25). This means that I would need to
conduct e.g. 24 separate ANOVAS. Possible, but not very effective. I spent
the last afternoon trying to apply ‚do repeat‘ and ‚loop‘ commands, but
didn’t achieve any progress. Thus, I am curious how you would approach this
problem?
Best,
E.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

Jon Peck

Re: Comparing mean of squares for many group configurations

I would add that while the Unianova post hoc dialog only supports First and Last as the reference category, in syntax you can specify any category. But be careful, the category specification, n, is not the value of the reference category but the nth level. See the CSR if in doubt.

On Fri, Oct 23, 2020 at 10:45 AM Michael Palij <[hidden email]> wrote:

It would seem that the OP wants to do Dunnett's test -- see Wikipedia for
one description of the test: https://en.wikipedia.org/wiki/Dunnett's_test
Other sources provide additional info (search Google scholar for refs).
Basically, one defines a reference group/category and this is compared
with all other groups/categories.

UNIANOVA allows one to do this test with a specification in /POSTHOC.
One could also specify paired comparisons between the reference group
and the other groups and used an adjusted p-value per test (i.e., say
usine alpha-overall = .05 and dividing by the number of tests, e.g., .05/24;
this would be an ordinary Bonferroni correction for the number of tests --
given the large number of tests, the per comparison alpha would be
very small and one would probably use the Sidak correction to reduce
the penalty).

The LSD or Least Significant Difference test is just multiple t-tests
each at a conventional alpha (sat, .05). It has been referred to as
a "protected t-test" procedure because one requires a significant
ANOVA F to use this test and, theoretically, a significant F value
implies that at least one comparison between/among means (i.e.,
simple or complex comparisons) is statistically significant, however,
it may not be the one that one is interested in.

-Mike Palij
New York University
[hidden email]

On Fri, Oct 23, 2020 at 11:39 AM Jon Peck <[hidden email]> wrote:
Maybe use the ONEWAY or UNIANOVA post-hoc tests for this (LSD doesn't do a multiple test correction IIRC).

Or, if you want to create a bunch of ANOVAs against dichotomies formed from each category against just one reference category,
a small dose of Python code could enumerate the categories, do a transformation, and run the ANOVA in a loop.

You can't put procedures inside a DO REPEAT or LOOP, because those are for transformations, but the Python code could easily handle a loop.

On Fri, Oct 23, 2020 at 9:19 AM Empi <[hidden email]> wrote:
Dear all,

I’d like to compare the between- and within-group mean of squares for a
continuous dependent variable. However, I face the following problem: what I
need to do is to compare one specific group (the reference group) with all
remaining other groups. However, the categorical grouping variable/the
factor comprises of many groups (k=25). This means that I would need to
conduct e.g. 24 separate ANOVAS. Possible, but not very effective. I spent
the last afternoon trying to apply ‚do repeat‘ and ‚loop‘ commands, but
didn’t achieve any progress. Thus, I am curious how you would approach this
problem?
Best,
E.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

Jon K Peck
[hidden email]

Rich Ulrich

Re: Comparing mean of squares for many group configurations

In reply to this post by Empi

As Jon points out, Dunnett's test is designed for comparing one

control group to all the others in an experimental design. However,

in an experimental design designed for good power, one would

create all comparison groups with the same N and the control

group with a larger N.

Further - the design assumption for Dunnett's is that those N's are

equal. Your description gives no reason for us to assume that.

Given your data, I would do two things. First, I would get the LSD test as

a test to report, with the assumption that all this is written up as

"exploratory," so that there is little concern for "controlling for multiple

tests." Second, I would compute a standardized score, as a T-score, for

the other group means, in order to show the effect size. T is computed

to set the mean to 50 and SD to 10 for the control.

The T is computed using mean Xbar and SD from the controls, as

Tscore= 10* (X-Xbar)/SD + 50.

Then I could list results as (Group), (p-value), (T-score) (N) --

Group p T N

4 0.0001 70 80

18 0.0003 73 62

...

(My guesses, above, do not accurately reflect that statistical formula

connecting T, N, and p. But I do show a smaller T in the first line than

in the second, which can happen because of the larger N. )

Rich Ulrich

From: SPSSX(r) Discussion <[hidden email]> on behalf of Empi <[hidden email]>
Sent: Friday, October 23, 2020 11:19 AM
To: [hidden email] <[hidden email]>
Subject: Comparing mean of squares for many group configurations

Jon Peck

Re: Comparing mean of squares for many group configurations

Wikipedia states the assumptions for the test as

The analysis considers the case where the results of the experiment are numerical, and the experiment is performed to compare p treatments with a control group. The results can be summarized as a set of

(p+1)

calculated means of the sets of observations,

({\bar {X_{{0}}}},...,{\bar {X_{{p}}}})

, while

({\bar {X_{{1}}}},...,{\bar {X_{{p}}}})

are referring to the treatment and

{\bar {X_{{0}}}}

is referring to the control set of observations, and

s

is an independent estimate of the common standard deviation of all

p+1

sets of observations. All

{\bar {X_{{i}}}}

of the

p+1

sets of observations are assumed to be independently and normally distributed with a common variance

\sigma ^{2}

and means

\mu_{i}

. There is also an assumption that there is an available estimate

s^{2}

for

\sigma ^{2}

It doesn't seem to require equal N's, and the formulas accommodate different sizes, whatever the virtue of equal N's. What am I missing?

On Fri, Oct 23, 2020 at 11:54 AM Rich Ulrich <[hidden email]> wrote:

As Jon points out, Dunnett's test is designed for comparing one

control group to all the others in an experimental design. However,

in an experimental design designed for good power, one would

create all comparison groups with the same N and the control

group with a larger N.

Further - the design assumption for Dunnett's is that those N's are

equal. Your description gives no reason for us to assume that.

Given your data, I would do two things. First, I would get the LSD test as

a test to report, with the assumption that all this is written up as

"exploratory," so that there is little concern for "controlling for multiple

tests." Second, I would compute a standardized score, as a T-score, for

the other group means, in order to show the effect size. T is computed

to set the mean to 50 and SD to 10 for the control.

The T is computed using mean Xbar and SD from the controls, as

Tscore= 10* (X-Xbar)/SD + 50.

Then I could list results as (Group), (p-value), (T-score) (N) --

Group      p       T    N

4      0.0001 70 80

18     0.0003 73   62

...

(My guesses, above, do not accurately reflect that statistical formula

connecting T, N, and p. But I do show a smaller T in the first line than

in the second, which can happen because of the larger N. )

--

Rich Ulrich

From: SPSSX(r) Discussion <[hidden email]> on behalf of Empi <[hidden email]>
Sent: Friday, October 23, 2020 11:19 AM
To: [hidden email] <[hidden email]>
Subject: Comparing mean of squares for many group configurations

Dear all,

I’d like to compare the between- and within-group mean of squares for a
continuous dependent variable. However, I face the following problem: what I
need to do is to compare one specific group (the reference group) with all
remaining other groups. However, the categorical grouping variable/the
factor comprises of many groups (k=25). This means that I would need to
conduct e.g. 24 separate ANOVAS. Possible, but not very effective. I spent
the last afternoon trying to apply ‚do repeat‘ and ‚loop‘ commands, but
didn’t achieve any progress. Thus, I am curious how you would approach this
problem?
Best,
E.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

Jon K Peck
[hidden email]

Mike

Re: Comparing mean of squares for many group configurations

In reply to this post by Jon Peck

I just checked the manual (for SPSS ver 22 -- if you have a different ver, double-check) and

there are 3 versions of Dunnett's test: (1) two-tailed test, (2) one-tailed test for differences

larger than the control group, and (3) one-tailed test for difference smaller than the control

group. Here is description from the manual:

DUNNETT(refcat). Dunnett’s two-tailed t test. Each level of the factor is compared to a reference category.
A reference category can be specified in parentheses. The default reference category is the last category.
This keyword must be spelled out in full.

DUNNETTL(refcat). Dunnett’s one-tailed t test. This test indicates whether the mean at any level (except
the reference category) of the factor is smaller than that of the reference category. A reference category can
be specified in parentheses. The default reference category is the last category. This keyword must be
spelled out in full.

DUNNETTR(refcat). Dunnett’s one-tailed t test. This test indicates whether the mean at any level (except
the reference category) of the factor is larger than that of the reference category. A reference category can
be specified in parentheses. The default reference category is the last category. This keyword must be
spelled out in full.

************************************

The choice of which test to use depends upon one's knowledge about the direction

of the difference. If one is interested in either direction, the use the two-tailed version (DUNNETT).

If one is interested in only differences reflecting means greater than the control, then

use DunnettR. If one is interested in only difference reflecting means smaller than the

control, then use DunnettL. One should be able to provide a reasonable rationale for

one's choice.

-Mike Palij

New York University

[hidden email]

On Fri, Oct 23, 2020 at 1:05 PM Jon Peck <[hidden email]> wrote:

I would add that while the Unianova post hoc dialog only supports First and Last as the reference category, in syntax you can specify any category. But be careful, the category specification, n, is not the value of the reference category but the nth level. See the CSR if in doubt.

On Fri, Oct 23, 2020 at 10:45 AM Michael Palij <[hidden email]> wrote:
It would seem that the OP wants to do Dunnett's test -- see Wikipedia for
one description of the test: https://en.wikipedia.org/wiki/Dunnett's_test
Other sources provide additional info (search Google scholar for refs).
Basically, one defines a reference group/category and this is compared
with all other groups/categories.

UNIANOVA allows one to do this test with a specification in /POSTHOC.
One could also specify paired comparisons between the reference group
and the other groups and used an adjusted p-value per test (i.e., say
usine alpha-overall = .05 and dividing by the number of tests, e.g., .05/24;
this would be an ordinary Bonferroni correction for the number of tests --
given the large number of tests, the per comparison alpha would be
very small and one would probably use the Sidak correction to reduce
the penalty).

The LSD or Least Significant Difference test is just multiple t-tests
each at a conventional alpha (sat, .05). It has been referred to as
a "protected t-test" procedure because one requires a significant
ANOVA F to use this test and, theoretically, a significant F value
implies that at least one comparison between/among means (i.e.,
simple or complex comparisons) is statistically significant, however,
it may not be the one that one is interested in.

-Mike Palij
New York University
[hidden email]

On Fri, Oct 23, 2020 at 11:39 AM Jon Peck <[hidden email]> wrote:
Maybe use the ONEWAY or UNIANOVA post-hoc tests for this (LSD doesn't do a multiple test correction IIRC).

Or, if you want to create a bunch of ANOVAs against dichotomies formed from each category against just one reference category,
a small dose of Python code could enumerate the categories, do a transformation, and run the ANOVA in a loop.

You can't put procedures inside a DO REPEAT or LOOP, because those are for transformations, but the Python code could easily handle a loop.

On Fri, Oct 23, 2020 at 9:19 AM Empi <[hidden email]> wrote:
Dear all,

I’d like to compare the between- and within-group mean of squares for a
continuous dependent variable. However, I face the following problem: what I
need to do is to compare one specific group (the reference group) with all
remaining other groups. However, the categorical grouping variable/the
factor comprises of many groups (k=25). This means that I would need to
conduct e.g. 24 separate ANOVAS. Possible, but not very effective. I spent
the last afternoon trying to apply ‚do repeat‘ and ‚loop‘ commands, but
didn’t achieve any progress. Thus, I am curious how you would approach this
problem?
Best,
E.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

Mike

Re: Comparing mean of squares for many group configurations

In reply to this post by Jon Peck

My go-to text is Kirk's Experimental Design (4th Ed) where his Chapter 5 is devoted to

Multiple Comparison Tests. His section 5.3 covers "Dunnett's Multiple Comparison test"

and here are some key points:

(1) (Kirk uses "p" to refer to the number of groups) Dunnett's procedure is applicable

to any set of p-1 a priori nonorthogonal contrasts for which the p-1 correlation between

the ***contrasts*** (emphasis added) are equal to 0.50, which occurs when p-1 experimental

group means is compared to a control group ***and the sample sizes are equal****** (emphasis

added).. Kirk provides examples of these points on page 177-178

(2) All assumptions of the test are provided on page 179 and are consistent with the

assumptions of doing the ordinary t-test. This includes the assumption of homogeneity

of variances among groups; A reference for Dunnett (1964; Biometrics, vol 20,pp 482-491)

provides info on how to modify the test to allow for heterogeneity. If the sample sizes are not

equal this implies that the correlations between contrasts is not 0.50 and one should

check out Hochberg & Tamhane ("Multiple Comparison Procedures", 1987, pp 140-144)

which provides tables for when the correlation is not equal to 0.50.

I assume that SPSS implicitly assumes equal sample sizes for the test but perhaps

a python add-on can be written (Hi Jon!) to take into account both heterogeneity of

variance and/or unequal sample sizes. BTW, IIRC, the Hochberg & Tamhane book

is pretty good but I am willing to bet that there may be a more recent text that

adds recent multiple comparison procedures.

-Mike Palij

New York University

[hidden email]

On Fri, Oct 23, 2020 at 2:04 PM Jon Peck <[hidden email]> wrote:

Wikipedia states the assumptions for the test as

The analysis considers the case where the results of the experiment are numerical, and the experiment is performed to compare p treatments with a control group. The results can be summarized as a set of {\displaystyle (p+1)} $(p+1)$ calculated means of the sets of observations, $({\bar {X_{{0}}}},...,{\bar {X_{{p}}}})$ , while $({\bar {X_{{1}}}},...,{\bar {X_{{p}}}})$ are referring to the treatment and {\displaystyle {\bar {X_{0}}}} ${\bar {X_{{0}}}}$ is referring to the control set of observations, and {\displaystyle s} $s$ is an independent estimate of the common standard deviation of all {\displaystyle p+1} $p+1$ sets of observations. All {\displaystyle {\bar {X_{i}}}} ${\bar {X_{{i}}}}$ of the {\displaystyle p+1} $p+1$ sets of observations are assumed to be independently and normally distributed with a common variance {\displaystyle \sigma ^{2}} $\sigma ^{2}$ and means {\displaystyle \mu _{i}} $\mu_{i}$ . There is also an assumption that there is an available estimate {\displaystyle s^{2}} $s^{2}$ for {\displaystyle \sigma ^{2}} $\sigma ^{2}$ .

It doesn't seem to require equal N's, and the formulas accommodate different sizes, whatever the virtue of equal N's. What am I missing?

On Fri, Oct 23, 2020 at 11:54 AM Rich Ulrich <[hidden email]> wrote:

As Jon points out, Dunnett's test is designed for comparing one

control group to all the others in an experimental design. However,

in an experimental design designed for good power, one would

create all comparison groups with the same N and the control

group with a larger N.

Further - the design assumption for Dunnett's is that those N's are

equal. Your description gives no reason for us to assume that.

Given your data, I would do two things. First, I would get the LSD test as

a test to report, with the assumption that all this is written up as

"exploratory," so that there is little concern for "controlling for multiple

tests." Second, I would compute a standardized score, as a T-score, for

the other group means, in order to show the effect size. T is computed

to set the mean to 50 and SD to 10 for the control.

The T is computed using mean Xbar and SD from the controls, as

Tscore= 10* (X-Xbar)/SD + 50.

Then I could list results as (Group), (p-value), (T-score) (N) --

Group      p       T    N

4      0.0001 70 80

18     0.0003 73   62

...

(My guesses, above, do not accurately reflect that statistical formula

connecting T, N, and p. But I do show a smaller T in the first line than

in the second, which can happen because of the larger N. )

--

Rich Ulrich

From: SPSSX(r) Discussion <[hidden email]> on behalf of Empi <[hidden email]>
Sent: Friday, October 23, 2020 11:19 AM
To: [hidden email] <[hidden email]>
Subject: Comparing mean of squares for many group configurations

Dear all,

I’d like to compare the between- and within-group mean of squares for a
continuous dependent variable. However, I face the following problem: what I
need to do is to compare one specific group (the reference group) with all
remaining other groups. However, the categorical grouping variable/the
factor comprises of many groups (k=25). This means that I would need to
conduct e.g. 24 separate ANOVAS. Possible, but not very effective. I spent
the last afternoon trying to apply ‚do repeat‘ and ‚loop‘ commands, but
didn’t achieve any progress. Thus, I am curious how you would approach this
problem?
Best,
E.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

Empi

Re: Comparing mean of squares for many group configurations

In reply to this post by Art Kendall

Dear Art,

sorry for being imprecise - in fact, I'm interested in all 24 comparisons.
Let's make it a bit simpler and assume there are just 4 groups: a, b, c & d.
Let group a be the reference group. I am interested in the between-group
variance and total variance (as indicated via the mean squares output) for

'Group a' vs. 'Group b',
'Group a' vs. 'Group c' &
'Group a' vs. 'Group d'.

The aim is to use the between- and total variance to compute the 'cultural
fixation index' suggested by Muthukrishna et al. (2020), see
https://journals.sagepub.com/doi/full/10.1177/0956797620916782
p. 682, which is the between-variance/total variance ratio.

Best,
E.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Rich Ulrich

Re: Comparing mean of squares for many group configurations

In reply to this post by Jon Peck

Jon,

Okay, thanks. I missed that. Dunnett's tables do not require equal

Ns (according to Wiki, and that sounds justified by the write-up).

The procedure, it says, does require finding the tabled-up "p"

coefficients (which take into account multiple tests) from a list

of the groups in order of t-value, rather than in order of the

means. I do not remember ever noticing that Dunnett's was

different this way from other corrections, but my lifetime use of

Dunnett's can be counted on the fingers of one hand. And never

with awkward conditions.

So, Dunnett's is pretty sound to use with varying Ns. But, be aware

that you may see results that appear paradoxical.

With varying Ns, the t-values may give a different order than the

means, which other "follow-up" tests use. To the extent that

t-values have to be larger for tiny Ns to get the same p, the list

may give a different order than if it were listed by separate p-values.

(I suspect that this affects the accuracy of the table when applied

to unequal Ns that include some tiny Ns, but no one will care.)

Rich Ulrich

PS. I read mail in Outlook. For others who may do the same, I note

that Outlook gave me a thoroughly mangled version of the Wiki

quote - writing out "Xbar" and "sigma" across multiple lines. However

those were all fine in the text when I clicked on the dots after "Reply".

I've seen that before, where mangled text is just fine in the Reply-version.

So this PS may help someone.

Rich Ulrich

From: Jon Peck <[hidden email]>
Sent: Friday, October 23, 2020 2:03 PM
To: Rich Ulrich <[hidden email]>
Cc: SPSS List <[hidden email]>
Subject: Re: [SPSSX-L] Comparing mean of squares for many group configurations

Wikipedia states the assumptions for the test as

(p+1)

calculated means of the sets of observations,

({\bar {X_{{0}}}},...,{\bar {X_{{p}}}})

, while

({\bar {X_{{1}}}},...,{\bar {X_{{p}}}})

are referring to the treatment and

{\bar {X_{{0}}}}

is referring to the control set of observations, and

s

is an independent estimate of the common standard deviation of all

p+1

sets of observations. All

{\bar {X_{{i}}}}

of the

p+1

sets of observations are assumed to be independently and normally distributed with a common variance

\sigma ^{2}

and means

\mu_{i}

. There is also an assumption that there is an available estimate

s^{2}

for

\sigma ^{2}

It doesn't seem to require equal N's, and the formulas accommodate different sizes, whatever the virtue of equal N's. What am I missing?

On Fri, Oct 23, 2020 at 11:54 AM Rich Ulrich <[hidden email]> wrote:

As Jon points out, Dunnett's test is designed for comparing one

control group to all the others in an experimental design. However,

in an experimental design designed for good power, one would

create all comparison groups with the same N and the control

group with a larger N.

Further - the design assumption for Dunnett's is that those N's are

equal. Your description gives no reason for us to assume that.

Given your data, I would do two things. First, I would get the LSD test as

a test to report, with the assumption that all this is written up as

"exploratory," so that there is little concern for "controlling for multiple

tests." Second, I would compute a standardized score, as a T-score, for

the other group means, in order to show the effect size. T is computed

to set the mean to 50 and SD to 10 for the control.

The T is computed using mean Xbar and SD from the controls, as

Tscore= 10* (X-Xbar)/SD + 50.

Then I could list results as (Group), (p-value), (T-score) (N) --

Group      p       T    N

4      0.0001 70 80

18     0.0003 73   62

...

(My guesses, above, do not accurately reflect that statistical formula

connecting T, N, and p. But I do show a smaller T in the first line than

in the second, which can happen because of the larger N. )

--

Rich Ulrich

From: SPSSX(r) Discussion <[hidden email]> on behalf of Empi <[hidden email]>
Sent: Friday, October 23, 2020 11:19 AM
To: [hidden email] <[hidden email]>
Subject: Comparing mean of squares for many group configurations

Dear all,

I’d like to compare the between- and within-group mean of squares for a
continuous dependent variable. However, I face the following problem: what I
need to do is to compare one specific group (the reference group) with all
remaining other groups. However, the categorical grouping variable/the
factor comprises of many groups (k=25). This means that I would need to
conduct e.g. 24 separate ANOVAS. Possible, but not very effective. I spent
the last afternoon trying to apply ‚do repeat‘ and ‚loop‘ commands, but
didn’t achieve any progress. Thus, I am curious how you would approach this
problem?
Best,
E.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

Jon K Peck
[hidden email]

Bruce Weaver

Re: Comparing mean of squares for many group configurations

Administrator

In reply to this post by Mike

Agreed. When the intent is to compare each treatment group to a
predesignated control group (or reference group, in this case), Dunnett's
test is the one.

But why are there so many groups? I cannot help but wonder if there are
"groups of groups", so to speak. If there are, perhaps some other contrasts
might be more useful than each group vs the reference group.

HTH.

Mike wrote
> It would seem that the OP wants to do Dunnett's test -- see Wikipedia for
> one description of the test: https://en.wikipedia.org/wiki/Dunnett's_test
> Other sources provide additional info (search Google scholar for refs).
> Basically, one defines a reference group/category and this is compared
> with all other groups/categories.
>
> --- snip ---

-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

PLEASE NOTE THE FOLLOWING:
1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above.
2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/).

Jon Peck

Re: Comparing mean of squares for many group configurations

In reply to this post by Rich Ulrich

Your lifetime Dunnett usage is slightly greater than mine, but I can think of a case or two where it would have been useful.

I tried to get the Dunnett JASA article from 1955, but even though I am an ASA life member (and still alive), the site refused to let me see the article. I had this problem recently with an old American Statistician article, too. I think I can still get it, although I guess that's academic [sic] now. Old stuff used to be accessible via JSTOR, but apparently ASA membership doesn't provide that access anymore. And I threw out my old journals (not quite that old!), since I would have online access to all this stuff.

On Fri, Oct 23, 2020 at 1:03 PM Rich Ulrich <[hidden email]> wrote:

Jon,

Okay, thanks. I missed that. Dunnett's tables do not require equal

Ns (according to Wiki, and that sounds justified by the write-up).

The procedure, it says, does require finding the tabled-up "p"

coefficients (which take into account multiple tests) from a list

of the groups in order of t-value, rather than in order of the

means. I do not remember ever noticing that Dunnett's was

different this way from other corrections, but my lifetime use of

Dunnett's can be counted on the fingers of one hand. And never

with awkward conditions.

So, Dunnett's is pretty sound to use with varying Ns. But, be aware

that you may see results that appear paradoxical.

With varying Ns, the t-values may give a different order than the

means, which other "follow-up" tests use. To the extent that

t-values have to be larger for tiny Ns to get the same p, the list

may give a different order than if it were listed by separate p-values.

(I suspect that this affects the accuracy of the table when applied

to unequal Ns that include some tiny Ns, but no one will care.)

--

Rich Ulrich

PS. I read mail in Outlook. For others who may do the same, I note

that Outlook gave me a thoroughly mangled version of the Wiki

quote - writing out "Xbar" and "sigma" across multiple lines. However

those were all fine in the text when I clicked on the dots after "Reply".

I've seen that before, where mangled text is just fine in the Reply-version.

So this PS may help someone.

--

Rich Ulrich

From: Jon Peck <[hidden email]>
Sent: Friday, October 23, 2020 2:03 PM
To: Rich Ulrich <[hidden email]>
Cc: SPSS List <[hidden email]>
Subject: Re: [SPSSX-L] Comparing mean of squares for many group configurations

Wikipedia states the assumptions for the test as

The analysis considers the case where the results of the experiment are numerical, and the experiment is performed to compare p treatments with a control group. The results can be summarized as a set of {\displaystyle (p+1)} $(p+1)$ calculated means of the sets of observations, $({\bar {X_{{0}}}},...,{\bar {X_{{p}}}})$ , while $({\bar {X_{{1}}}},...,{\bar {X_{{p}}}})$ are referring to the treatment and {\displaystyle {\bar {X_{0}}}} ${\bar {X_{{0}}}}$ is referring to the control set of observations, and {\displaystyle s} $s$ is an independent estimate of the common standard deviation of all {\displaystyle p+1} $p+1$ sets of observations. All {\displaystyle {\bar {X_{i}}}} ${\bar {X_{{i}}}}$ of the {\displaystyle p+1} $p+1$ sets of observations are assumed to be independently and normally distributed with a common variance {\displaystyle \sigma ^{2}} $\sigma ^{2}$ and means {\displaystyle \mu _{i}} $\mu_{i}$ . There is also an assumption that there is an available estimate {\displaystyle s^{2}} $s^{2}$ for {\displaystyle \sigma ^{2}} $\sigma ^{2}$ .

It doesn't seem to require equal N's, and the formulas accommodate different sizes, whatever the virtue of equal N's. What am I missing?

On Fri, Oct 23, 2020 at 11:54 AM Rich Ulrich <[hidden email]> wrote:

As Jon points out, Dunnett's test is designed for comparing one

control group to all the others in an experimental design. However,

in an experimental design designed for good power, one would

create all comparison groups with the same N and the control

group with a larger N.

Further - the design assumption for Dunnett's is that those N's are

equal. Your description gives no reason for us to assume that.

Given your data, I would do two things. First, I would get the LSD test as

a test to report, with the assumption that all this is written up as

"exploratory," so that there is little concern for "controlling for multiple

tests." Second, I would compute a standardized score, as a T-score, for

the other group means, in order to show the effect size. T is computed

to set the mean to 50 and SD to 10 for the control.

The T is computed using mean Xbar and SD from the controls, as

Tscore= 10* (X-Xbar)/SD + 50.

Then I could list results as (Group), (p-value), (T-score) (N) --

Group      p       T    N

4      0.0001 70 80

18     0.0003 73   62

...

(My guesses, above, do not accurately reflect that statistical formula

connecting T, N, and p. But I do show a smaller T in the first line than

in the second, which can happen because of the larger N. )

--

Rich Ulrich

From: SPSSX(r) Discussion <[hidden email]> on behalf of Empi <[hidden email]>
Sent: Friday, October 23, 2020 11:19 AM
To: [hidden email] <[hidden email]>
Subject: Comparing mean of squares for many group configurations

Dear all,

I’d like to compare the between- and within-group mean of squares for a
continuous dependent variable. However, I face the following problem: what I
need to do is to compare one specific group (the reference group) with all
remaining other groups. However, the categorical grouping variable/the
factor comprises of many groups (k=25). This means that I would need to
conduct e.g. 24 separate ANOVAS. Possible, but not very effective. I spent
the last afternoon trying to apply ‚do repeat‘ and ‚loop‘ commands, but
didn’t achieve any progress. Thus, I am curious how you would approach this
problem?
Best,
E.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

--

Jon K Peck
[hidden email]

Jon K Peck
[hidden email]

Mike

Re: Comparing mean of squares for many group configurations

In reply to this post by Bruce Weaver

I too wondered why there were 25 groups. I also wondered whether these were 25

qualitatively different conditions or 25 quantitatively different levels (e.g., drug dosage).

If the latter, then other types of analysis might be warranted.

Also, Kirk points out two alternative forms of analysis that can be done with

the Dunnett's test:

(1) A Critical Difference based on the Dunnett statistic (Kirk p178-179). Just

as the LSD or Least Significant Difference test is the minimum difference between

two means to achieve a significant difference -- the numerator of the critical t statistic,

the Dunnett critical difference is the minimum difference between the control and

an experimental group that has to be exceeded to claim a significant difference (usual

assumptions apply).

(2) For fans of Confidence Intervals, a simultaneous CI can be calculated involving

the control group mean (Kirk p179).

As with most situations, the person asking a question may think that their question

is extremely simple until the person trying to answer the question probes for context,

assumptions, and other information to make sense of the question.

-Mike Palij

New York University

[hidden email]

On Fri, Oct 23, 2020 at 3:37 PM Bruce Weaver <[hidden email]> wrote:

Agreed. When the intent is to compare each treatment group to a
predesignated control group (or reference group, in this case), Dunnett's
test is the one.

But why are there so many groups? I cannot help but wonder if there are
"groups of groups", so to speak. If there are, perhaps some other contrasts
might be more useful than each group vs the reference group.

HTH.

Mike wrote
> It would seem that the OP wants to do Dunnett's test -- see Wikipedia for
> one description of the test: https://urldefense.proofpoint.com/v2/url?u=https-3A__en.wikipedia.org_wiki_Dunnett-27s-5Ftest&d=DwICAg&c=slrrB7dE8n7gBJbeO0g-IQ&r=A8kXUln5f-BYIUaapBvbXA&m=flWPLb1x4nzAQS74hX1RVEYr7Plc-6vlxnc4bLjHan0&s=rXzOxay07vn9LfHZ3IheDqmAeESZ4lOUB7NLtTrDZDU&e=
> Other sources provide additional info (search Google scholar for refs).
> Basically, one defines a reference group/category and this is compared
> with all other groups/categories.
>
> --- snip ---

-----
--
Bruce Weaver
[hidden email]
https://urldefense.proofpoint.com/v2/url?u=http-3A__sites.google.com_a_lakeheadu.ca_bweaver_&d=DwICAg&c=slrrB7dE8n7gBJbeO0g-IQ&r=A8kXUln5f-BYIUaapBvbXA&m=flWPLb1x4nzAQS74hX1RVEYr7Plc-6vlxnc4bLjHan0&s=XgzlbqSgFLGmBofGCJiEaBqmqDQiz3yRCwVrjwAJVi8&e=

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: https://urldefense.proofpoint.com/v2/url?u=http-3A__spssx-2Ddiscussion.1045642.n5.nabble.com_&d=DwICAg&c=slrrB7dE8n7gBJbeO0g-IQ&r=A8kXUln5f-BYIUaapBvbXA&m=flWPLb1x4nzAQS74hX1RVEYr7Plc-6vlxnc4bLjHan0&s=8AztxxhLVuORS4J6mSvJMAe7L3w8eJNH7nmO93Q2-G0&e=

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Rich Ulrich

Re: Comparing mean of squares for many group configurations

In reply to this post by Empi

E.,

Your response at 2:56 PM seems to have arrived late at my

list -- I did not see it until now. And it has not been addressed

by the later answers I have seen by other people. So: You do

want to get numbers for each of 24 countries. Or the like.

I clicked on the article you cited, in order to figure out what you

are looking for. Maybe if I were familiar with the "genetic" index

that it is modeled after, I could make sense of it; but, as it is, I can't.

It seems to be combining doing tests or getting measures across

multiple dimensions and (somehow) combining those results. My

curiosity ran out before I figured it out.

Rich Ulrich

From: SPSSX(r) Discussion <[hidden email]> on behalf of Empi <[hidden email]>
Sent: Friday, October 23, 2020 2:56 PM
To: [hidden email] <[hidden email]>
Subject: Re: Comparing mean of squares for many group configurations