ANOVA Statistical Significance, where does "d" value come from?

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

ANOVA Statistical Significance, where does "d" value come from?

William Peck
5 months into learning SPSS / Statistics and need a hopefully quick primer on one statistic from ANOVA.

I am duplicating a report from last year, which worked great for this year's data. SPSS ran great and I got the output I expected, but need help in interpreting last year's report.

ANOVA output from last year is

Fall CALC1  grades
                  N Mean        Std. Deviation
Class of 2020 125 2.0720 1.01740
Class of 2021 149 2.6644  .96989
Total                274 2.3942 1.03320


Fall CALC1 grades
                         SS        df Mean Square F Sig.
Between Groups 23.857 1 23.857 24.252 .000 ***
Within Groups 267.573 272 .984
Total                291.431 273

*** The footnote on this data says "There is a statistically significant and moderately sized (d = .60) difference in Calculus I grades between 2020 and 2021. "

My question is where do you get "d = .60" (because I don't see it anywhere). If I can find that, at least I'll have a reference for the comment, then I'll study ANOVA myself and post questions as needed.

(The statistician who did last year's report is nowhere to be found ...)

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: ANOVA Statistical Significance, where does "d" value come from?

Kirill Orlov
It must be Cohen's d effect size. See Wikipedia.


===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: ANOVA Statistical Significance, where does "d" value come from?

Ware, William B
In reply to this post by William Peck
I would bet that it is Cohen's d... The difference between the means is approximately .59.  The pooled standard deviation is a little less than 1.00, so the ratio is approximately .6 or so..

Bill
 
William B. Ware, Professor Emeritus
Educational Psychology, Measurement, and Evaluation
Learning Sciences and Psychological Studies
University of North Carolina at Chapel Hill
McMichael Term Professor of Education, 2011-2013
Adjunct Professor, School of Social Work
Academy of Distinguished Teaching Scholars at UNC-Chapel Hill, Charter Member
EMAIL: [hidden email]



-----Original Message-----
From: SPSSX(r) Discussion <[hidden email]> On Behalf Of William Peck
Sent: Wednesday, January 30, 2019 11:01 AM
To: [hidden email]
Subject: ANOVA Statistical Significance, where does "d" value come from?

5 months into learning SPSS / Statistics and need a hopefully quick primer on one statistic from ANOVA.

I am duplicating a report from last year, which worked great for this year's data. SPSS ran great and I got the output I expected, but need help in interpreting last year's report.

ANOVA output from last year is

Fall CALC1  grades
                  N Mean        Std. Deviation
Class of 2020 125 2.0720 1.01740
Class of 2021 149 2.6644  .96989
Total                274 2.3942 1.03320


Fall CALC1 grades
                         SS        df Mean Square F Sig.
Between Groups 23.857 1 23.857 24.252 .000 ***
Within Groups 267.573 272 .984
Total                291.431 273

*** The footnote on this data says "There is a statistically significant and moderately sized (d = .60) difference in Calculus I grades between 2020 and 2021. "

My question is where do you get "d = .60" (because I don't see it anywhere). If I can find that, at least I'll have a reference for the comment, then I'll study ANOVA myself and post questions as needed.

(The statistician who did last year's report is nowhere to be found ...)

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: ANOVA Statistical Significance, where does "d" value come from?

bdates

He probably calculated it by hand. SPSS produces the partial eta squared as the measure of effect size, if the option is checked when preparing the analysis.


Brian

From: SPSSX(r) Discussion <[hidden email]> on behalf of Ware, William B <[hidden email]>
Sent: Wednesday, January 30, 2019 11:12:02 AM
To: [hidden email]
Subject: Re: ANOVA Statistical Significance, where does "d" value come from?
 
I would bet that it is Cohen's d... The difference between the means is approximately .59.  The pooled standard deviation is a little less than 1.00, so the ratio is approximately .6 or so..

Bill
 
William B. Ware, Professor Emeritus
Educational Psychology, Measurement, and Evaluation
Learning Sciences and Psychological Studies
University of North Carolina at Chapel Hill
McMichael Term Professor of Education, 2011-2013
Adjunct Professor, School of Social Work
Academy of Distinguished Teaching Scholars at UNC-Chapel Hill, Charter Member
EMAIL: [hidden email]



-----Original Message-----
From: SPSSX(r) Discussion <[hidden email]> On Behalf Of William Peck
Sent: Wednesday, January 30, 2019 11:01 AM
To: [hidden email]
Subject: ANOVA Statistical Significance, where does "d" value come from?

5 months into learning SPSS / Statistics and need a hopefully quick primer on one statistic from ANOVA.

I am duplicating a report from last year, which worked great for this year's data. SPSS ran great and I got the output I expected, but need help in interpreting last year's report.

ANOVA output from last year is

Fall CALC1  grades
                         N       Mean            Std. Deviation
Class of 2020   125     2.0720  1.01740
Class of 2021   149     2.6644    .96989
Total                   274      2.3942  1.03320


Fall CALC1 grades
                         SS              df       Mean Square     F       Sig.
Between Groups  23.857  1       23.857  24.252  .000 ***
Within Groups   267.573 272     .984           
Total                   291.431  273                    

*** The footnote on this data says "There is a statistically significant and moderately sized (d = .60) difference in Calculus I grades between 2020 and 2021. "

My question is where do you get "d = .60" (because I don't see it anywhere). If I can find that, at least I'll have a reference for the comment, then I'll study ANOVA myself and post questions as needed.

(The statistician who did last year's report is nowhere to be found ...)

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: ANOVA Statistical Significance, where does "d" value come from?

Mike
In reply to this post by William Peck
Note that, in general, there are two types of effect size measures:
(1) Percentage of variance accounted for (e.g., eta-square, R-square, etc;
historically, SPSS has peovided only this type of effect size measure).
and
(2) Difference(s) between means (e.g., d, g, f, etc.)

Because of the critical role effect size measures play in statistical power
analysis, one nice reference for effect size measures and power analysis
is Jack Cohen's 1992 article "A Power Primer"; see:

Note2:  For a one-way independent groups ANOVA, Cohen recommends
the effect size measure "f".  However, in your example you only have two
groups (which could have been analyzed by independent groups t-test),
though f can be calculated, d is a simpler and more familiar measure.
Cohen's Table 1 provides a listing of effect size measures, their formulas,
and guidelines for the interpretation of the magnitude of an ES.

HTH.

-Mike Palij
Nrw York University


On Wed, Jan 30, 2019 at 11:01 AM William Peck <[hidden email]> wrote:
5 months into learning SPSS / Statistics and need a hopefully quick primer on one statistic from ANOVA.

I am duplicating a report from last year, which worked great for this year's data. SPSS ran great and I got the output I expected, but need help in interpreting last year's report.

ANOVA output from last year is

Fall CALC1  grades
                        N       Mean            Std. Deviation
Class of 2020   125     2.0720  1.01740
Class of 2021   149     2.6644    .96989
Total                   274     2.3942  1.03320


Fall CALC1 grades
                         SS             df      Mean Square     F       Sig.
Between Groups  23.857  1       23.857  24.252  .000 ***
Within Groups   267.573 272     .984           
Total                   291.431 273                     

*** The footnote on this data says "There is a statistically significant and moderately sized (d = .60) difference in Calculus I grades between 2020 and 2021. "

My question is where do you get "d = .60" (because I don't see it anywhere). If I can find that, at least I'll have a reference for the comment, then I'll study ANOVA myself and post questions as needed.

(The statistician who did last year's report is nowhere to be found ...)

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: ANOVA Statistical Significance, where does "d" value come from?

Jon Peck
And while the t test output doesn't include d, it is easy to add it to the output using the STATS  TABLE CALC extension command as explained in my chapter (18) in McCormick and Salcedo, SPSS Statistics for Data Analysis and Visualization .  I can send the code to anyone who wants it.

On Wed, Jan 30, 2019 at 10:27 AM Michael Palij <[hidden email]> wrote:
Note that, in general, there are two types of effect size measures:
(1) Percentage of variance accounted for (e.g., eta-square, R-square, etc;
historically, SPSS has peovided only this type of effect size measure).
and
(2) Difference(s) between means (e.g., d, g, f, etc.)

Because of the critical role effect size measures play in statistical power
analysis, one nice reference for effect size measures and power analysis
is Jack Cohen's 1992 article "A Power Primer"; see:

Note2:  For a one-way independent groups ANOVA, Cohen recommends
the effect size measure "f".  However, in your example you only have two
groups (which could have been analyzed by independent groups t-test),
though f can be calculated, d is a simpler and more familiar measure.
Cohen's Table 1 provides a listing of effect size measures, their formulas,
and guidelines for the interpretation of the magnitude of an ES.

HTH.

-Mike Palij
Nrw York University


On Wed, Jan 30, 2019 at 11:01 AM William Peck <[hidden email]> wrote:
5 months into learning SPSS / Statistics and need a hopefully quick primer on one statistic from ANOVA.

I am duplicating a report from last year, which worked great for this year's data. SPSS ran great and I got the output I expected, but need help in interpreting last year's report.

ANOVA output from last year is

Fall CALC1  grades
                        N       Mean            Std. Deviation
Class of 2020   125     2.0720  1.01740
Class of 2021   149     2.6644    .96989
Total                   274     2.3942  1.03320


Fall CALC1 grades
                         SS             df      Mean Square     F       Sig.
Between Groups  23.857  1       23.857  24.252  .000 ***
Within Groups   267.573 272     .984           
Total                   291.431 273                     

*** The footnote on this data says "There is a statistically significant and moderately sized (d = .60) difference in Calculus I grades between 2020 and 2021. "

My question is where do you get "d = .60" (because I don't see it anywhere). If I can find that, at least I'll have a reference for the comment, then I'll study ANOVA myself and post questions as needed.

(The statistician who did last year's report is nowhere to be found ...)

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD


--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: ANOVA Statistical Significance, where does "d" value come from?

Mike
In reply to this post by Mike
My bad.  When I go to the address I gave I have automatic access to
the article (I guess I get it because I come from the nyu.edu domain but
I don't remember acess being so seamless).  The full reference for the
Cohen article is:

Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159.
doi:10.1037/0033-2909.112.1.155

A Google search may turn up an unlocked copy (it is a popular
article with over 32K citations according to Google Scholar).

-Mike Palij
New York University



On Wed, Jan 30, 2019 at 1:11 PM Jon Peck <[hidden email]> wrote:
The Cohen article you cite is under lock and key.

On Wed, Jan 30, 2019 at 10:27 AM Michael Palij <[hidden email]> wrote:
Note that, in general, there are two types of effect size measures:
(1) Percentage of variance accounted for (e.g., eta-square, R-square, etc;
historically, SPSS has peovided only this type of effect size measure).
and
(2) Difference(s) between means (e.g., d, g, f, etc.)

Because of the critical role effect size measures play in statistical power
analysis, one nice reference for effect size measures and power analysis
is Jack Cohen's 1992 article "A Power Primer"; see:

Note2:  For a one-way independent groups ANOVA, Cohen recommends
the effect size measure "f".  However, in your example you only have two
groups (which could have been analyzed by independent groups t-test),
though f can be calculated, d is a simpler and more familiar measure.
Cohen's Table 1 provides a listing of effect size measures, their formulas,
and guidelines for the interpretation of the magnitude of an ES.

HTH.

-Mike Palij
Nrw York University


On Wed, Jan 30, 2019 at 11:01 AM William Peck <[hidden email]> wrote:
5 months into learning SPSS / Statistics and need a hopefully quick primer on one statistic from ANOVA.

I am duplicating a report from last year, which worked great for this year's data. SPSS ran great and I got the output I expected, but need help in interpreting last year's report.

ANOVA output from last year is

Fall CALC1  grades
                        N       Mean            Std. Deviation
Class of 2020   125     2.0720  1.01740
Class of 2021   149     2.6644    .96989
Total                   274     2.3942  1.03320


Fall CALC1 grades
                         SS             df      Mean Square     F       Sig.
Between Groups  23.857  1       23.857  24.252  .000 ***
Within Groups   267.573 272     .984           
Total                   291.431 273                     

*** The footnote on this data says "There is a statistically significant and moderately sized (d = .60) difference in Calculus I grades between 2020 and 2021. "

My question is where do you get "d = .60" (because I don't see it anywhere). If I can find that, at least I'll have a reference for the comment, then I'll study ANOVA myself and post questions as needed.

(The statistician who did last year's report is nowhere to be found ...)

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD


--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: ANOVA Statistical Significance, where does "d" value come from?

bdates

There's an unlocked copy at http://www.bwgriffin.com/workshop/Sampling%20A%20Cohen%20tables.pdf 



Brian Dates

From: SPSSX(r) Discussion <[hidden email]> on behalf of Michael Palij <[hidden email]>
Sent: Wednesday, January 30, 2019 1:55:03 PM
To: [hidden email]
Subject: Re: ANOVA Statistical Significance, where does "d" value come from?
 
My bad.  When I go to the address I gave I have automatic access to
the article (I guess I get it because I come from the nyu.edu domain but
I don't remember acess being so seamless).  The full reference for the
Cohen article is:

Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159.
doi:10.1037/0033-2909.112.1.155

A Google search may turn up an unlocked copy (it is a popular
article with over 32K citations according to Google Scholar).

-Mike Palij
New York University



On Wed, Jan 30, 2019 at 1:11 PM Jon Peck <[hidden email]> wrote:
The Cohen article you cite is under lock and key.

On Wed, Jan 30, 2019 at 10:27 AM Michael Palij <[hidden email]> wrote:
Note that, in general, there are two types of effect size measures:
(1) Percentage of variance accounted for (e.g., eta-square, R-square, etc;
historically, SPSS has peovided only this type of effect size measure).
and
(2) Difference(s) between means (e.g., d, g, f, etc.)

Because of the critical role effect size measures play in statistical power
analysis, one nice reference for effect size measures and power analysis
is Jack Cohen's 1992 article "A Power Primer"; see:

Note2:  For a one-way independent groups ANOVA, Cohen recommends
the effect size measure "f".  However, in your example you only have two
groups (which could have been analyzed by independent groups t-test),
though f can be calculated, d is a simpler and more familiar measure.
Cohen's Table 1 provides a listing of effect size measures, their formulas,
and guidelines for the interpretation of the magnitude of an ES.

HTH.

-Mike Palij
Nrw York University


On Wed, Jan 30, 2019 at 11:01 AM William Peck <[hidden email]> wrote:
5 months into learning SPSS / Statistics and need a hopefully quick primer on one statistic from ANOVA.

I am duplicating a report from last year, which worked great for this year's data. SPSS ran great and I got the output I expected, but need help in interpreting last year's report.

ANOVA output from last year is

Fall CALC1  grades
                        N       Mean            Std. Deviation
Class of 2020   125     2.0720  1.01740
Class of 2021   149     2.6644    .96989
Total                   274     2.3942  1.03320


Fall CALC1 grades
                         SS             df      Mean Square     F       Sig.
Between Groups  23.857  1       23.857  24.252  .000 ***
Within Groups   267.573 272     .984           
Total                   291.431 273                     

*** The footnote on this data says "There is a statistically significant and moderately sized (d = .60) difference in Calculus I grades between 2020 and 2021. "

My question is where do you get "d = .60" (because I don't see it anywhere). If I can find that, at least I'll have a reference for the comment, then I'll study ANOVA myself and post questions as needed.

(The statistician who did last year's report is nowhere to be found ...)

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD


--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: ANOVA Statistical Significance, where does "d" value come from?

Bruce Weaver
Administrator
But before you use d (or some other standardized effect size measure) as the
basis of a sample size estimate, see this short note by Russell Lenth:

  https://homepage.divms.uiowa.edu/~rlenth/Power/2badHabits.pdf

If you can get access to it, take a look at Thom Baguley's nice article too:

  https://onlinelibrary.wiley.com/doi/full/10.1348/000712608X377117

HTH.



bdates wrote

> There's an unlocked copy at
> http://www.bwgriffin.com/workshop/Sampling%20A%20Cohen%20tables.pdf
>
> A Power Primer Jacob Cohen Psychological Bulletin [PsycARTICLES]; July
> 1992; 112, 1; PsycARTICLES pg. 155 - Bryan W. Griffin's
> Web&lt;http://www.bwgriffin.com/workshop/Sampling%20A%20Cohen%20tables.pdf&gt;
> www.bwgriffin.com
> Reproduced with permission of the copyright owner. Further reproduction
> prohibited without permission. Created Date: 06/23/05 11:02
>
>
>
>
> Brian Dates
> ________________________________
> From: SPSSX(r) Discussion &lt;

> SPSSX-L@.UGA

> &gt; on behalf of Michael Palij &lt;

> mp26@

> &gt;
> Sent: Wednesday, January 30, 2019 1:55:03 PM
> To:

> SPSSX-L@.UGA

> Subject: Re: ANOVA Statistical Significance, where does "d" value come
> from?
>
> My bad.  When I go to the address I gave I have automatic access to
> the article (I guess I get it because I come from the
> nyu.edu&lt;http://nyu.edu&gt; domain but
> I don't remember acess being so seamless).  The full reference for the
> Cohen article is:
>
> Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159.
> doi:10.1037/0033-2909.112.1.155
>
> A Google search may turn up an unlocked copy (it is a popular
> article with over 32K citations according to Google Scholar).
>
> -Mike Palij
> New York University
>
>
>
> On Wed, Jan 30, 2019 at 1:11 PM Jon Peck &lt;

> jkpeck@

> &lt;mailto:

> jkpeck@

> &gt;> wrote:
> The Cohen article you cite is under lock and key.
>
> On Wed, Jan 30, 2019 at 10:27 AM Michael Palij &lt;

> mp26@

> &lt;mailto:

> mp26@

> &gt;> wrote:
> Note that, in general, there are two types of effect size measures:
> (1) Percentage of variance accounted for (e.g., eta-square, R-square, etc;
> historically, SPSS has peovided only this type of effect size measure).
> and
> (2) Difference(s) between means (e.g., d, g, f, etc.)
>
> Because of the critical role effect size measures play in statistical
> power
> analysis, one nice reference for effect size measures and power analysis
> is Jack Cohen's 1992 article "A Power Primer"; see:
> https://psycnet.apa.org/fulltext/1992-37683-001.html&lt;https://urldefense.proofpoint.com/v2/url?u=https-3A__psycnet.apa.org_fulltext_1992-2D37683-2D001.html&amp;d=DwMFaQ&amp;c=slrrB7dE8n7gBJbeO0g-IQ&amp;r=A8kXUln5f-BYIUaapBvbXA&amp;m=uzwM-Iuv5YnaMvgQ6gAPwXd3GrThLHAwnuRBlKIVaMQ&amp;s=ohZK9sN5vgraudfffyXbERUe65gbw65WLVTbFPs3JJM&amp;e=&gt;
>
> Note2:  For a one-way independent groups ANOVA, Cohen recommends
> the effect size measure "f".  However, in your example you only have two
> groups (which could have been analyzed by independent groups t-test),
> though f can be calculated, d is a simpler and more familiar measure.
> Cohen's Table 1 provides a listing of effect size measures, their
> formulas,
> and guidelines for the interpretation of the magnitude of an ES.
>
> HTH.
>
> -Mike Palij
> Nrw York University
>
>
> On Wed, Jan 30, 2019 at 11:01 AM William Peck &lt;

> peck@

> &lt;mailto:

> peck@

> &gt;> wrote:
> 5 months into learning SPSS / Statistics and need a hopefully quick primer
> on one statistic from ANOVA.
>
> I am duplicating a report from last year, which worked great for this
> year's data. SPSS ran great and I got the output I expected, but need help
> in interpreting last year's report.
>
> ANOVA output from last year is
>
> Fall CALC1  grades
>                         N       Mean            Std. Deviation
> Class of 2020   125     2.0720  1.01740
> Class of 2021   149     2.6644    .96989
> Total                   274     2.3942  1.03320
>
>
> Fall CALC1 grades
>                          SS             df      Mean Square     F      
> Sig.
> Between Groups  23.857  1       23.857  24.252  .000 ***
> Within Groups   267.573 272     .984
> Total                   291.431 273
>
> *** The footnote on this data says "There is a statistically significant
> and moderately sized (d = .60) difference in Calculus I grades between
> 2020 and 2021. "
>
> My question is where do you get "d = .60" (because I don't see it
> anywhere). If I can find that, at least I'll have a reference for the
> comment, then I'll study ANOVA myself and post questions as needed.
>
> (The statistician who did last year's report is nowhere to be found ...)
>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

> &lt;mailto:

> LISTSERV@.UGA

> &gt; (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
> ===================== To manage your subscription to SPSSX-L, send a
> message to

> LISTSERV@.UGA

> &lt;mailto:

> LISTSERV@.UGA

> &gt; (not to SPSSX-L), with no body text except the command. To leave the
> list, send the command SIGNOFF SPSSX-L For a list of commands to manage
> subscriptions, send the command INFO REFCARD
>
>
> --
> Jon K Peck

> jkpeck@

> &lt;mailto:

> jkpeck@

> &gt;
>
> ===================== To manage your subscription to SPSSX-L, send a
> message to

> LISTSERV@.UGA

> &lt;mailto:

> LISTSERV@.UGA

> &gt; (not to SPSSX-L), with no body text except the command. To leave the
> list, send the command SIGNOFF SPSSX-L For a list of commands to manage
> subscriptions, send the command INFO REFCARD
>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

PLEASE NOTE THE FOLLOWING: 
1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above.
2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/).
Reply | Threaded
Open this post in threaded view
|

Re: ANOVA Statistical Significance, where does "d" value come from?

bdates

Again, if anyone is interested, there's an unlocked copy of the Bagueli article at https://pdfs.semanticscholar.org/86b6/bef80331f6afbbcb7371bd23ab3abc3ba0b2.pdf



Brian Dates

From: SPSSX(r) Discussion <[hidden email]> on behalf of Bruce Weaver <[hidden email]>
Sent: Wednesday, January 30, 2019 2:44:09 PM
To: [hidden email]
Subject: Re: ANOVA Statistical Significance, where does "d" value come from?
 
But before you use d (or some other standardized effect size measure) as the
basis of a sample size estimate, see this short note by Russell Lenth:

  https://homepage.divms.uiowa.edu/~rlenth/Power/2badHabits.pdf

If you can get access to it, take a look at Thom Baguley's nice article too:

  https://onlinelibrary.wiley.com/doi/full/10.1348/000712608X377117

HTH.



bdates wrote
> There's an unlocked copy at
> http://www.bwgriffin.com/workshop/Sampling%20A%20Cohen%20tables.pdf
>
> A Power Primer Jacob Cohen Psychological Bulletin [PsycARTICLES]; July
> 1992; 112, 1; PsycARTICLES pg. 155 - Bryan W. Griffin's
> Web&lt;http://www.bwgriffin.com/workshop/Sampling%20A%20Cohen%20tables.pdf&gt;
> www.bwgriffin.com
> Reproduced with permission of the copyright owner. Further reproduction
> prohibited without permission. Created Date: 06/23/05 11:02
>
>
>
>
> Brian Dates
> ________________________________
> From: SPSSX(r) Discussion &lt;

> SPSSX-L@.UGA

> &gt; on behalf of Michael Palij &lt;

> mp26@

> &gt;
> Sent: Wednesday, January 30, 2019 1:55:03 PM
> To:

> SPSSX-L@.UGA

> Subject: Re: ANOVA Statistical Significance, where does "d" value come
> from?
>
> My bad.  When I go to the address I gave I have automatic access to
> the article (I guess I get it because I come from the
> nyu.edu&lt;http://nyu.edu&gt; domain but
> I don't remember acess being so seamless).  The full reference for the
> Cohen article is:
>
> Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159.
> doi:10.1037/0033-2909.112.1.155
>
> A Google search may turn up an unlocked copy (it is a popular
> article with over 32K citations according to Google Scholar).
>
> -Mike Palij
> New York University
>
>
>
> On Wed, Jan 30, 2019 at 1:11 PM Jon Peck &lt;

> jkpeck@

> &lt;mailto:

> jkpeck@

> &gt;> wrote:
> The Cohen article you cite is under lock and key.
>
> On Wed, Jan 30, 2019 at 10:27 AM Michael Palij &lt;

> mp26@

> &lt;mailto:

> mp26@

> &gt;> wrote:
> Note that, in general, there are two types of effect size measures:
> (1) Percentage of variance accounted for (e.g., eta-square, R-square, etc;
> historically, SPSS has peovided only this type of effect size measure).
> and
> (2) Difference(s) between means (e.g., d, g, f, etc.)
>
> Because of the critical role effect size measures play in statistical
> power
> analysis, one nice reference for effect size measures and power analysis
> is Jack Cohen's 1992 article "A Power Primer"; see:
> https://psycnet.apa.org/fulltext/1992-37683-001.html&lt;https://urldefense.proofpoint.com/v2/url?u=https-3A__psycnet.apa.org_fulltext_1992-2D37683-2D001.html&amp;d=DwMFaQ&amp;c=slrrB7dE8n7gBJbeO0g-IQ&amp;r=A8kXUln5f-BYIUaapBvbXA&amp;m=uzwM-Iuv5YnaMvgQ6gAPwXd3GrThLHAwnuRBlKIVaMQ&amp;s=ohZK9sN5vgraudfffyXbERUe65gbw65WLVTbFPs3JJM&amp;e=&gt;
>
> Note2:  For a one-way independent groups ANOVA, Cohen recommends
> the effect size measure "f".  However, in your example you only have two
> groups (which could have been analyzed by independent groups t-test),
> though f can be calculated, d is a simpler and more familiar measure.
> Cohen's Table 1 provides a listing of effect size measures, their
> formulas,
> and guidelines for the interpretation of the magnitude of an ES.
>
> HTH.
>
> -Mike Palij
> Nrw York University
>
>
> On Wed, Jan 30, 2019 at 11:01 AM William Peck &lt;

> peck@

> &lt;mailto:

> peck@

> &gt;> wrote:
> 5 months into learning SPSS / Statistics and need a hopefully quick primer
> on one statistic from ANOVA.
>
> I am duplicating a report from last year, which worked great for this
> year's data. SPSS ran great and I got the output I expected, but need help
> in interpreting last year's report.
>
> ANOVA output from last year is
>
> Fall CALC1  grades
>                         N       Mean            Std. Deviation
> Class of 2020   125     2.0720  1.01740
> Class of 2021   149     2.6644    .96989
> Total                   274     2.3942  1.03320
>
>
> Fall CALC1 grades
>                          SS             df      Mean Square     F     
> Sig.
> Between Groups  23.857  1       23.857  24.252  .000 ***
> Within Groups   267.573 272     .984
> Total                   291.431 273
>
> *** The footnote on this data says "There is a statistically significant
> and moderately sized (d = .60) difference in Calculus I grades between
> 2020 and 2021. "
>
> My question is where do you get "d = .60" (because I don't see it
> anywhere). If I can find that, at least I'll have a reference for the
> comment, then I'll study ANOVA myself and post questions as needed.
>
> (The statistician who did last year's report is nowhere to be found ...)
>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

> &lt;mailto:

> LISTSERV@.UGA

> &gt; (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
> ===================== To manage your subscription to SPSSX-L, send a
> message to

> LISTSERV@.UGA

> &lt;mailto:

> LISTSERV@.UGA

> &gt; (not to SPSSX-L), with no body text except the command. To leave the
> list, send the command SIGNOFF SPSSX-L For a list of commands to manage
> subscriptions, send the command INFO REFCARD
>
>
> --
> Jon K Peck

> jkpeck@

> &lt;mailto:

> jkpeck@

> &gt;
>
> ===================== To manage your subscription to SPSSX-L, send a
> message to

> LISTSERV@.UGA

> &lt;mailto:

> LISTSERV@.UGA

> &gt; (not to SPSSX-L), with no body text except the command. To leave the
> list, send the command SIGNOFF SPSSX-L For a list of commands to manage
> subscriptions, send the command INFO REFCARD
>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: ANOVA Statistical Significance, where does "d" value come from?

Kornbrot, Diana
The Baguley article  is excellent, but it presents the argument as either or. Of course one needs BOTH
Of course one MUST have the RAW effects, aka DESCRIPTIVE statistics, together with confidence levels (which are inferential).
Arguments depend to be about how INFERENTIAL statistics are presented.
Statistical effect sizes measuring the magnitude of the effect of the predictor relative to everything else [random and confounding factors]. This as others have noted may be either of the form  of a magnitude divided by SD of that magnitude. E.g. Cohen’s d for difference between means or other parameter, Hedge’s g etc. Or they may be some kind of proportion of variance accounted for. This does not depend on sample size, which aids in interpretation. 
Probability of a null hypothesis measuring the probability that the effect obtained from the sample could have occurred by chance. This depends on sample size and sometimes there is strong evidence [low p-null] because smack,e size is large.
BOTH  ar important for interpretation. They are different ways of presenting the results of the same inferential test, e.g. an F-test. They are both valuable and ditching p-null  just because effects sizes are a good thing  is in my view insane. Small but reliable effect sizes may be very important in large populations. Large but unreliable effect sizes are only useful in suggesting replications - not in drawing inferences.
See
Lenhard, W., & Lenhard, A. (2016). Calculation of Effect Sizes. . Psychometrica. doi:http://dx.doi.org/10.13140/RG.2.1.3478.4245
for conversion between inferential measures.

Bayes Factor [BF] and associated credibility intervals. Are also valuable [to frequentists as well as Bayesians] because they provide evidence for the null swell as the alternative hypothesis. Exact values of BF require specialists software. However estimates can be obtained form F-values see 
Wetzels, R., Matzke, D., Lee, M. D., Rouder, J. N., Iverson, G. J., & Wagenmakers, E.-J. (2011). Statistical Evidence in Experimental Psychology. Perspectives on Psychological Science, 6(3), 291-298. doi:10.1177/1745691611406923
Best
Diana

On 30 Jan 2019, at 20:11, Dates, Brian <[hidden email]> wrote:

Again, if anyone is interested, there's an unlocked copy of the Bagueli article at https://pdfs.semanticscholar.org/86b6/bef80331f6afbbcb7371bd23ab3abc3ba0b2.pdf


Brian Dates

From: SPSSX(r) Discussion <[hidden email]> on behalf of Bruce Weaver <[hidden email]>
Sent: Wednesday, January 30, 2019 2:44:09 PM
To: [hidden email]
Subject: Re: ANOVA Statistical Significance, where does "d" value come from?
 
But before you use d (or some other standardized effect size measure) as the
basis of a sample size estimate, see this short note by Russell Lenth:

  https://homepage.divms.uiowa.edu/~rlenth/Power/2badHabits.pdf

If you can get access to it, take a look at Thom Baguley's nice article too:

  https://onlinelibrary.wiley.com/doi/full/10.1348/000712608X377117

HTH.



bdates wrote
> There's an unlocked copy at
> http://www.bwgriffin.com/workshop/Sampling%20A%20Cohen%20tables.pdf
> 
> A Power Primer Jacob Cohen Psychological Bulletin [PsycARTICLES]; July
> 1992; 112, 1; PsycARTICLES pg. 155 - Bryan W. Griffin's
> Web&lt;http://www.bwgriffin.com/workshop/Sampling%20A%20Cohen%20tables.pdf&gt;
> www.bwgriffin.com
> Reproduced with permission of the copyright owner. Further reproduction
> prohibited without permission. Created Date: 06/23/05 11:02
> 
> 
> 
> 
> Brian Dates
> ________________________________
> From: SPSSX(r) Discussion &lt;

> SPSSX-L@.UGA

> &gt; on behalf of Michael Palij &lt;

> mp26@

> &gt;
> Sent: Wednesday, January 30, 2019 1:55:03 PM
> To: 

> SPSSX-L@.UGA

> Subject: Re: ANOVA Statistical Significance, where does "d" value come
> from?
> 
> My bad.  When I go to the address I gave I have automatic access to
> the article (I guess I get it because I come from the
> nyu.edu&lt;http://nyu.edu&gt; domain but
> I don't remember acess being so seamless).  The full reference for the
> Cohen article is:
> 
> Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159.
> doi:10.1037/0033-2909.112.1.155
> 
> A Google search may turn up an unlocked copy (it is a popular
> article with over 32K citations according to Google Scholar).
> 
> -Mike Palij
> New York University
> 
> 
> 
> On Wed, Jan 30, 2019 at 1:11 PM Jon Peck &lt;

> jkpeck@

> &lt;mailto:

> jkpeck@

> &gt;> wrote:
> The Cohen article you cite is under lock and key.
> 
> On Wed, Jan 30, 2019 at 10:27 AM Michael Palij &lt;

> mp26@

> &lt;mailto:

> mp26@

> &gt;> wrote:
> Note that, in general, there are two types of effect size measures:
> (1) Percentage of variance accounted for (e.g., eta-square, R-square, etc;
> historically, SPSS has peovided only this type of effect size measure).
> and
> (2) Difference(s) between means (e.g., d, g, f, etc.)
> 
> Because of the critical role effect size measures play in statistical
> power
> analysis, one nice reference for effect size measures and power analysis
> is Jack Cohen's 1992 article "A Power Primer"; see:
> https://psycnet.apa.org/fulltext/1992-37683-001.html&lt;https://urldefense.proofpoint.com/v2/url?u=https-3A__psycnet.apa.org_fulltext_1992-2D37683-2D001.html&amp;d=DwMFaQ&amp;c=slrrB7dE8n7gBJbeO0g-IQ&amp;r=A8kXUln5f-BYIUaapBvbXA&amp;m=uzwM-Iuv5YnaMvgQ6gAPwXd3GrThLHAwnuRBlKIVaMQ&amp;s=ohZK9sN5vgraudfffyXbERUe65gbw65WLVTbFPs3JJM&amp;e=&gt;
> 
> Note2:  For a one-way independent groups ANOVA, Cohen recommends
> the effect size measure "f".  However, in your example you only have two
> groups (which could have been analyzed by independent groups t-test),
> though f can be calculated, d is a simpler and more familiar measure.
> Cohen's Table 1 provides a listing of effect size measures, their
> formulas,
> and guidelines for the interpretation of the magnitude of an ES.
> 
> HTH.
> 
> -Mike Palij
> Nrw York University
> 
> 
> On Wed, Jan 30, 2019 at 11:01 AM William Peck &lt;

> peck@

> &lt;mailto:

> peck@

> &gt;> wrote:
> 5 months into learning SPSS / Statistics and need a hopefully quick primer
> on one statistic from ANOVA.
> 
> I am duplicating a report from last year, which worked great for this
> year's data. SPSS ran great and I got the output I expected, but need help
> in interpreting last year's report.
> 
> ANOVA output from last year is
> 
> Fall CALC1  grades
>                         N       Mean            Std. Deviation
> Class of 2020   125     2.0720  1.01740
> Class of 2021   149     2.6644    .96989
> Total                   274     2.3942  1.03320
> 
> 
> Fall CALC1 grades
>                          SS             df      Mean Square     F      
> Sig.
> Between Groups  23.857  1       23.857  24.252  .000 ***
> Within Groups   267.573 272     .984
> Total                   291.431 273
> 
> *** The footnote on this data says "There is a statistically significant
> and moderately sized (d = .60) difference in Calculus I grades between
> 2020 and 2021. "
> 
> My question is where do you get "d = .60" (because I don't see it
> anywhere). If I can find that, at least I'll have a reference for the
> comment, then I'll study ANOVA myself and post questions as needed.
> 
> (The statistician who did last year's report is nowhere to be found ...)
> 
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

> &lt;mailto:

> LISTSERV@.UGA

> &gt; (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
> ===================== To manage your subscription to SPSSX-L, send a
> message to 

> LISTSERV@.UGA

> &lt;mailto:

> LISTSERV@.UGA

> &gt; (not to SPSSX-L), with no body text except the command. To leave the
> list, send the command SIGNOFF SPSSX-L For a list of commands to manage
> subscriptions, send the command INFO REFCARD
> 
> 
> --
> Jon K Peck

> jkpeck@

> &lt;mailto:

> jkpeck@

> &gt;
> 
> ===================== To manage your subscription to SPSSX-L, send a
> message to 

> LISTSERV@.UGA

> &lt;mailto:

> LISTSERV@.UGA

> &gt; (not to SPSSX-L), with no body text except the command. To leave the
> list, send the command SIGNOFF SPSSX-L For a list of commands to manage
> subscriptions, send the command INFO REFCARD
> 
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to[hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

_____________________________________
Professor Diana Kornbrot
Mobile
+44 (0) 7403 18 16 12
Work
University of Hertfordshire
College Lane, Hatfield, Hertfordshire AL10 9AB, UK
+44 (0) 170 728 4626
[hidden email]
http://dianakornbrot.wordpress.com/
http://go.herts.ac.uk/Diana_Kornbrot
skype:  kornbrotme
Home
19 Elmhurst Avenue
London N2 0LT, UK
+44 (0) 208 444 2081                                                   

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD