5 months into learning SPSS / Statistics and need a hopefully quick primer on one statistic from ANOVA.
I am duplicating a report from last year, which worked great for this year's data. SPSS ran great and I got the output I expected, but need help in interpreting last year's report. ANOVA output from last year is Fall CALC1 grades N Mean Std. Deviation Class of 2020 125 2.0720 1.01740 Class of 2021 149 2.6644 .96989 Total 274 2.3942 1.03320 Fall CALC1 grades SS df Mean Square F Sig. Between Groups 23.857 1 23.857 24.252 .000 *** Within Groups 267.573 272 .984 Total 291.431 273 *** The footnote on this data says "There is a statistically significant and moderately sized (d = .60) difference in Calculus I grades between 2020 and 2021. " My question is where do you get "d = .60" (because I don't see it anywhere). If I can find that, at least I'll have a reference for the comment, then I'll study ANOVA myself and post questions as needed. (The statistician who did last year's report is nowhere to be found ...) ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
It must be Cohen's d effect size. See Wikipedia.
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
In reply to this post by William Peck
I would bet that it is Cohen's d... The difference between the means is approximately .59. The pooled standard deviation is a little less than 1.00, so the ratio is approximately .6 or so..
Bill William B. Ware, Professor Emeritus Educational Psychology, Measurement, and Evaluation Learning Sciences and Psychological Studies University of North Carolina at Chapel Hill McMichael Term Professor of Education, 2011-2013 Adjunct Professor, School of Social Work Academy of Distinguished Teaching Scholars at UNC-Chapel Hill, Charter Member EMAIL: [hidden email] -----Original Message----- From: SPSSX(r) Discussion <[hidden email]> On Behalf Of William Peck Sent: Wednesday, January 30, 2019 11:01 AM To: [hidden email] Subject: ANOVA Statistical Significance, where does "d" value come from? 5 months into learning SPSS / Statistics and need a hopefully quick primer on one statistic from ANOVA. I am duplicating a report from last year, which worked great for this year's data. SPSS ran great and I got the output I expected, but need help in interpreting last year's report. ANOVA output from last year is Fall CALC1 grades N Mean Std. Deviation Class of 2020 125 2.0720 1.01740 Class of 2021 149 2.6644 .96989 Total 274 2.3942 1.03320 Fall CALC1 grades SS df Mean Square F Sig. Between Groups 23.857 1 23.857 24.252 .000 *** Within Groups 267.573 272 .984 Total 291.431 273 *** The footnote on this data says "There is a statistically significant and moderately sized (d = .60) difference in Calculus I grades between 2020 and 2021. " My question is where do you get "d = .60" (because I don't see it anywhere). If I can find that, at least I'll have a reference for the comment, then I'll study ANOVA myself and post questions as needed. (The statistician who did last year's report is nowhere to be found ...) ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
He probably calculated it by hand. SPSS produces the partial eta squared as the measure of effect size, if the option is checked when preparing the analysis.
Brian
From: SPSSX(r) Discussion <[hidden email]> on behalf of Ware, William B <[hidden email]>
Sent: Wednesday, January 30, 2019 11:12:02 AM To: [hidden email] Subject: Re: ANOVA Statistical Significance, where does "d" value come from? I would bet that it is Cohen's d... The difference between the means is approximately .59. The pooled standard deviation is a little less than 1.00, so the ratio is approximately .6 or so..
Bill William B. Ware, Professor Emeritus Educational Psychology, Measurement, and Evaluation Learning Sciences and Psychological Studies University of North Carolina at Chapel Hill McMichael Term Professor of Education, 2011-2013 Adjunct Professor, School of Social Work Academy of Distinguished Teaching Scholars at UNC-Chapel Hill, Charter Member EMAIL: [hidden email] -----Original Message----- From: SPSSX(r) Discussion <[hidden email]> On Behalf Of William Peck Sent: Wednesday, January 30, 2019 11:01 AM To: [hidden email] Subject: ANOVA Statistical Significance, where does "d" value come from? 5 months into learning SPSS / Statistics and need a hopefully quick primer on one statistic from ANOVA. I am duplicating a report from last year, which worked great for this year's data. SPSS ran great and I got the output I expected, but need help in interpreting last year's report. ANOVA output from last year is Fall CALC1 grades N Mean Std. Deviation Class of 2020 125 2.0720 1.01740 Class of 2021 149 2.6644 .96989 Total 274 2.3942 1.03320 Fall CALC1 grades SS df Mean Square F Sig. Between Groups 23.857 1 23.857 24.252 .000 *** Within Groups 267.573 272 .984 Total 291.431 273 *** The footnote on this data says "There is a statistically significant and moderately sized (d = .60) difference in Calculus I grades between 2020 and 2021. " My question is where do you get "d = .60" (because I don't see it anywhere). If I can find that, at least I'll have a reference for the comment, then I'll study ANOVA myself and post questions as needed. (The statistician who did last year's report is nowhere to be found ...) ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
In reply to this post by William Peck
Note that, in general, there are two types of effect size measures: (1) Percentage of variance accounted for (e.g., eta-square, R-square, etc; historically, SPSS has peovided only this type of effect size measure). and (2) Difference(s) between means (e.g., d, g, f, etc.) Because of the critical role effect size measures play in statistical power analysis, one nice reference for effect size measures and power analysis is Jack Cohen's 1992 article "A Power Primer"; see: Note2: For a one-way independent groups ANOVA, Cohen recommends the effect size measure "f". However, in your example you only have two groups (which could have been analyzed by independent groups t-test), though f can be calculated, d is a simpler and more familiar measure. Cohen's Table 1 provides a listing of effect size measures, their formulas, and guidelines for the interpretation of the magnitude of an ES. HTH. -Mike Palij Nrw York University On Wed, Jan 30, 2019 at 11:01 AM William Peck <[hidden email]> wrote: 5 months into learning SPSS / Statistics and need a hopefully quick primer on one statistic from ANOVA. |
And while the t test output doesn't include d, it is easy to add it to the output using the STATS TABLE CALC extension command as explained in my chapter (18) in McCormick and Salcedo, SPSS Statistics for Data Analysis and Visualization . I can send the code to anyone who wants it. On Wed, Jan 30, 2019 at 10:27 AM Michael Palij <[hidden email]> wrote:
|
In reply to this post by Mike
My bad. When I go to the address I gave I have automatic access to the article (I guess I get it because I come from the nyu.edu domain but I don't remember acess being so seamless). The full reference for the Cohen article is:
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159. doi:10.1037/0033-2909.112.1.155 A Google search may turn up an unlocked copy (it is a popular article with over 32K citations according to Google Scholar). -Mike Palij New York University On Wed, Jan 30, 2019 at 1:11 PM Jon Peck <[hidden email]> wrote:
|
There's an unlocked copy at http://www.bwgriffin.com/workshop/Sampling%20A%20Cohen%20tables.pdf
Brian Dates
From: SPSSX(r) Discussion <[hidden email]> on behalf of Michael Palij <[hidden email]>
Sent: Wednesday, January 30, 2019 1:55:03 PM To: [hidden email] Subject: Re: ANOVA Statistical Significance, where does "d" value come from? My bad. When I go to the address I gave I have automatic access to
the article (I guess I get it because I come from the
nyu.edu domain but
I don't remember acess being so seamless). The full reference for the
Cohen article is:
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159.
doi:10.1037/0033-2909.112.1.155
A Google search may turn up an unlocked copy (it is a popular
article with over 32K citations according to Google Scholar).
-Mike Palij
New York University
On Wed, Jan 30, 2019 at 1:11 PM Jon Peck <[hidden email]> wrote:
|
Administrator
|
But before you use d (or some other standardized effect size measure) as the
basis of a sample size estimate, see this short note by Russell Lenth: https://homepage.divms.uiowa.edu/~rlenth/Power/2badHabits.pdf If you can get access to it, take a look at Thom Baguley's nice article too: https://onlinelibrary.wiley.com/doi/full/10.1348/000712608X377117 HTH. bdates wrote > There's an unlocked copy at > http://www.bwgriffin.com/workshop/Sampling%20A%20Cohen%20tables.pdf > > A Power Primer Jacob Cohen Psychological Bulletin [PsycARTICLES]; July > 1992; 112, 1; PsycARTICLES pg. 155 - Bryan W. Griffin's > Web<http://www.bwgriffin.com/workshop/Sampling%20A%20Cohen%20tables.pdf> > www.bwgriffin.com > Reproduced with permission of the copyright owner. Further reproduction > prohibited without permission. Created Date: 06/23/05 11:02 > > > > > Brian Dates > ________________________________ > From: SPSSX(r) Discussion < > SPSSX-L@.UGA > > on behalf of Michael Palij < > mp26@ > > > Sent: Wednesday, January 30, 2019 1:55:03 PM > To: > SPSSX-L@.UGA > Subject: Re: ANOVA Statistical Significance, where does "d" value come > from? > > My bad. When I go to the address I gave I have automatic access to > the article (I guess I get it because I come from the > nyu.edu<http://nyu.edu> domain but > I don't remember acess being so seamless). The full reference for the > Cohen article is: > > Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159. > doi:10.1037/0033-2909.112.1.155 > > A Google search may turn up an unlocked copy (it is a popular > article with over 32K citations according to Google Scholar). > > -Mike Palij > New York University > > > > On Wed, Jan 30, 2019 at 1:11 PM Jon Peck < > jkpeck@ > <mailto: > jkpeck@ > >> wrote: > The Cohen article you cite is under lock and key. > > On Wed, Jan 30, 2019 at 10:27 AM Michael Palij < > mp26@ > <mailto: > mp26@ > >> wrote: > Note that, in general, there are two types of effect size measures: > (1) Percentage of variance accounted for (e.g., eta-square, R-square, etc; > historically, SPSS has peovided only this type of effect size measure). > and > (2) Difference(s) between means (e.g., d, g, f, etc.) > > Because of the critical role effect size measures play in statistical > power > analysis, one nice reference for effect size measures and power analysis > is Jack Cohen's 1992 article "A Power Primer"; see: > https://psycnet.apa.org/fulltext/1992-37683-001.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__psycnet.apa.org_fulltext_1992-2D37683-2D001.html&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=A8kXUln5f-BYIUaapBvbXA&m=uzwM-Iuv5YnaMvgQ6gAPwXd3GrThLHAwnuRBlKIVaMQ&s=ohZK9sN5vgraudfffyXbERUe65gbw65WLVTbFPs3JJM&e=> > > Note2: For a one-way independent groups ANOVA, Cohen recommends > the effect size measure "f". However, in your example you only have two > groups (which could have been analyzed by independent groups t-test), > though f can be calculated, d is a simpler and more familiar measure. > Cohen's Table 1 provides a listing of effect size measures, their > formulas, > and guidelines for the interpretation of the magnitude of an ES. > > HTH. > > -Mike Palij > Nrw York University > > > On Wed, Jan 30, 2019 at 11:01 AM William Peck < > peck@ > <mailto: > peck@ > >> wrote: > 5 months into learning SPSS / Statistics and need a hopefully quick primer > on one statistic from ANOVA. > > I am duplicating a report from last year, which worked great for this > year's data. SPSS ran great and I got the output I expected, but need help > in interpreting last year's report. > > ANOVA output from last year is > > Fall CALC1 grades > N Mean Std. Deviation > Class of 2020 125 2.0720 1.01740 > Class of 2021 149 2.6644 .96989 > Total 274 2.3942 1.03320 > > > Fall CALC1 grades > SS df Mean Square F > Sig. > Between Groups 23.857 1 23.857 24.252 .000 *** > Within Groups 267.573 272 .984 > Total 291.431 273 > > *** The footnote on this data says "There is a statistically significant > and moderately sized (d = .60) difference in Calculus I grades between > 2020 and 2021. " > > My question is where do you get "d = .60" (because I don't see it > anywhere). If I can find that, at least I'll have a reference for the > comment, then I'll study ANOVA myself and post questions as needed. > > (The statistician who did last year's report is nowhere to be found ...) > > ===================== > To manage your subscription to SPSSX-L, send a message to > LISTSERV@.UGA > <mailto: > LISTSERV@.UGA > > (not to SPSSX-L), with no body text except the > command. To leave the list, send the command > SIGNOFF SPSSX-L > For a list of commands to manage subscriptions, send the command > INFO REFCARD > ===================== To manage your subscription to SPSSX-L, send a > message to > LISTSERV@.UGA > <mailto: > LISTSERV@.UGA > > (not to SPSSX-L), with no body text except the command. To leave the > list, send the command SIGNOFF SPSSX-L For a list of commands to manage > subscriptions, send the command INFO REFCARD > > > -- > Jon K Peck > jkpeck@ > <mailto: > jkpeck@ > > > > ===================== To manage your subscription to SPSSX-L, send a > message to > LISTSERV@.UGA > <mailto: > LISTSERV@.UGA > > (not to SPSSX-L), with no body text except the command. To leave the > list, send the command SIGNOFF SPSSX-L For a list of commands to manage > subscriptions, send the command INFO REFCARD > > ===================== > To manage your subscription to SPSSX-L, send a message to > LISTSERV@.UGA > (not to SPSSX-L), with no body text except the > command. To leave the list, send the command > SIGNOFF SPSSX-L > For a list of commands to manage subscriptions, send the command > INFO REFCARD ----- -- Bruce Weaver [hidden email] http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." NOTE: My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. -- Sent from: http://spssx-discussion.1045642.n5.nabble.com/ ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
--
Bruce Weaver bweaver@lakeheadu.ca http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." PLEASE NOTE THE FOLLOWING: 1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. 2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/). |
Again, if anyone is interested, there's an unlocked copy of the Bagueli article at https://pdfs.semanticscholar.org/86b6/bef80331f6afbbcb7371bd23ab3abc3ba0b2.pdf
Brian Dates
From: SPSSX(r) Discussion <[hidden email]> on behalf of Bruce Weaver <[hidden email]>
Sent: Wednesday, January 30, 2019 2:44:09 PM To: [hidden email] Subject: Re: ANOVA Statistical Significance, where does "d" value come from? But before you use d (or some other standardized effect size measure) as the
basis of a sample size estimate, see this short note by Russell Lenth: https://homepage.divms.uiowa.edu/~rlenth/Power/2badHabits.pdf If you can get access to it, take a look at Thom Baguley's nice article too: https://onlinelibrary.wiley.com/doi/full/10.1348/000712608X377117 HTH. bdates wrote > There's an unlocked copy at > http://www.bwgriffin.com/workshop/Sampling%20A%20Cohen%20tables.pdf > > A Power Primer Jacob Cohen Psychological Bulletin [PsycARTICLES]; July > 1992; 112, 1; PsycARTICLES pg. 155 - Bryan W. Griffin's > Web<http://www.bwgriffin.com/workshop/Sampling%20A%20Cohen%20tables.pdf> > www.bwgriffin.com > Reproduced with permission of the copyright owner. Further reproduction > prohibited without permission. Created Date: 06/23/05 11:02 > > > > > Brian Dates > ________________________________ > From: SPSSX(r) Discussion < > SPSSX-L@.UGA > > on behalf of Michael Palij < > mp26@ > > > Sent: Wednesday, January 30, 2019 1:55:03 PM > To: > SPSSX-L@.UGA > Subject: Re: ANOVA Statistical Significance, where does "d" value come > from? > > My bad. When I go to the address I gave I have automatic access to > the article (I guess I get it because I come from the > nyu.edu<http://nyu.edu> domain but > I don't remember acess being so seamless). The full reference for the > Cohen article is: > > Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159. > doi:10.1037/0033-2909.112.1.155 > > A Google search may turn up an unlocked copy (it is a popular > article with over 32K citations according to Google Scholar). > > -Mike Palij > New York University > > > > On Wed, Jan 30, 2019 at 1:11 PM Jon Peck < > jkpeck@ > <mailto: > jkpeck@ > >> wrote: > The Cohen article you cite is under lock and key. > > On Wed, Jan 30, 2019 at 10:27 AM Michael Palij < > mp26@ > <mailto: > mp26@ > >> wrote: > Note that, in general, there are two types of effect size measures: > (1) Percentage of variance accounted for (e.g., eta-square, R-square, etc; > historically, SPSS has peovided only this type of effect size measure). > and > (2) Difference(s) between means (e.g., d, g, f, etc.) > > Because of the critical role effect size measures play in statistical > power > analysis, one nice reference for effect size measures and power analysis > is Jack Cohen's 1992 article "A Power Primer"; see: > https://psycnet.apa.org/fulltext/1992-37683-001.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__psycnet.apa.org_fulltext_1992-2D37683-2D001.html&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=A8kXUln5f-BYIUaapBvbXA&m=uzwM-Iuv5YnaMvgQ6gAPwXd3GrThLHAwnuRBlKIVaMQ&s=ohZK9sN5vgraudfffyXbERUe65gbw65WLVTbFPs3JJM&e=> > > Note2: For a one-way independent groups ANOVA, Cohen recommends > the effect size measure "f". However, in your example you only have two > groups (which could have been analyzed by independent groups t-test), > though f can be calculated, d is a simpler and more familiar measure. > Cohen's Table 1 provides a listing of effect size measures, their > formulas, > and guidelines for the interpretation of the magnitude of an ES. > > HTH. > > -Mike Palij > Nrw York University > > > On Wed, Jan 30, 2019 at 11:01 AM William Peck < > peck@ > <mailto: > peck@ > >> wrote: > 5 months into learning SPSS / Statistics and need a hopefully quick primer > on one statistic from ANOVA. > > I am duplicating a report from last year, which worked great for this > year's data. SPSS ran great and I got the output I expected, but need help > in interpreting last year's report. > > ANOVA output from last year is > > Fall CALC1 grades > N Mean Std. Deviation > Class of 2020 125 2.0720 1.01740 > Class of 2021 149 2.6644 .96989 > Total 274 2.3942 1.03320 > > > Fall CALC1 grades > SS df Mean Square F > Sig. > Between Groups 23.857 1 23.857 24.252 .000 *** > Within Groups 267.573 272 .984 > Total 291.431 273 > > *** The footnote on this data says "There is a statistically significant > and moderately sized (d = .60) difference in Calculus I grades between > 2020 and 2021. " > > My question is where do you get "d = .60" (because I don't see it > anywhere). If I can find that, at least I'll have a reference for the > comment, then I'll study ANOVA myself and post questions as needed. > > (The statistician who did last year's report is nowhere to be found ...) > > ===================== > To manage your subscription to SPSSX-L, send a message to > LISTSERV@.UGA > <mailto: > LISTSERV@.UGA > > (not to SPSSX-L), with no body text except the > command. To leave the list, send the command > SIGNOFF SPSSX-L > For a list of commands to manage subscriptions, send the command > INFO REFCARD > ===================== To manage your subscription to SPSSX-L, send a > message to > LISTSERV@.UGA > <mailto: > LISTSERV@.UGA > > (not to SPSSX-L), with no body text except the command. To leave the > list, send the command SIGNOFF SPSSX-L For a list of commands to manage > subscriptions, send the command INFO REFCARD > > > -- > Jon K Peck > jkpeck@ > <mailto: > jkpeck@ > > > > ===================== To manage your subscription to SPSSX-L, send a > message to > LISTSERV@.UGA > <mailto: > LISTSERV@.UGA > > (not to SPSSX-L), with no body text except the command. To leave the > list, send the command SIGNOFF SPSSX-L For a list of commands to manage > subscriptions, send the command INFO REFCARD > > ===================== > To manage your subscription to SPSSX-L, send a message to > LISTSERV@.UGA > (not to SPSSX-L), with no body text except the > command. To leave the list, send the command > SIGNOFF SPSSX-L > For a list of commands to manage subscriptions, send the command > INFO REFCARD ----- -- Bruce Weaver [hidden email] http://sites.google.com/a/lakeheadu.ca/bweaver/ "When all else fails, RTFM." NOTE: My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above. -- Sent from: http://spssx-discussion.1045642.n5.nabble.com/ ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
The Baguley article is excellent, but it presents the argument as either or. Of course one needs BOTH
Of course one MUST have the RAW effects, aka DESCRIPTIVE statistics, together with confidence levels (which are inferential).
Arguments depend to be about how INFERENTIAL statistics are presented.
Statistical effect sizes measuring the
magnitude of the effect of the predictor relative to everything else [random and confounding factors]. This as others have noted may be either of the form of a magnitude divided by SD of that magnitude. E.g. Cohen’s d for difference between means or other
parameter, Hedge’s g etc. Or they may be some kind of proportion of variance accounted for. This does not depend on sample size, which aids in interpretation.
Probability of a null hypothesis measuring the probability that the effect obtained from the sample could have occurred by chance. This depends on sample size and sometimes there is strong evidence [low p-null] because smack,e
size is large.
BOTH ar important for interpretation. They are different ways of presenting the results of the same inferential test, e.g. an F-test. They are both valuable and ditching p-null just because effects sizes are a
good thing is in my view insane. Small but reliable effect sizes may be very important in large populations. Large but unreliable effect sizes are only useful in suggesting replications - not in drawing inferences.
See
Lenhard, W., & Lenhard, A. (2016). Calculation of Effect Sizes. .
Psychometrica. doi:http://dx.doi.org/10.13140/RG.2.1.3478.4245
for conversion between inferential measures.
Bayes Factor [BF] and associated credibility intervals.
Are also valuable [to frequentists as well as Bayesians] because they provide evidence for the null swell as the alternative hypothesis. Exact values of BF require specialists software. However
estimates can be obtained form F-values see
Wetzels, R., Matzke, D., Lee, M. D., Rouder, J. N., Iverson, G. J., & Wagenmakers, E.-J. (2011). Statistical Evidence in Experimental Psychology.
Perspectives on Psychological Science, 6(3), 291-298. doi:10.1177/1745691611406923
Best
Diana
_____________________________________
Professor Diana Kornbrot
Mobile
+44 (0) 7403 18 16 12 Work University of Hertfordshire College Lane, Hatfield, Hertfordshire AL10 9AB, UK +44 (0) 170 728 4626 [hidden email] http://dianakornbrot.wordpress.com/ http://go.herts.ac.uk/Diana_Kornbrot skype: kornbrotme Home 19 Elmhurst Avenue London N2 0LT, UK +44 (0) 208 444 2081 |
Free forum by Nabble | Edit this page |