Significance test for gamma-correlations

[Jazgul Ismailova-2]

Hello,

I have two gamma-correlation coefficients, based on correlation between brand associations and purchase intent. Those coefficients are very close to each other 0,89 vs 0,86.  Is there a way to  prove that there is a statistical difference between them?

Thanks for your input!

Kind regards,
Jazgul

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Jazgul Ismailova-2]

Hello Rich,

Thanks for your reply! Let me explain my question in more detail. 

I have 2 correlation coeficients (no matter if it is gamma or pearson): 0,89 and 0,86. I need to show that 0,89 statistically higher than 0,86. 

I already know that both drive purchase intent, but would like to show that attribute one is statistically more important than attribute 2. 

Are there any methods I can apply?

Thanks!

//Jazgul

28 mar 2013 kl. 18:25 skrev "Rich Ulrich" <[hidden email]>:

The way to test this difference between two tables of ordinal
values is start by forgetting that you have used gamma.  Test
for whether the association is different.  That can be fairly easy,
using Logistic regression. For the hard way, you might figure out
what an approximate error should be from using the approximate
test on gamma.  (See Wikip, for details that are confusing -- note
that for a 2x2 table [A,B; C,D]  the discordant/accordant fraction
comes to AD/BC, an odds ratio.)

If purchase-intent is a dichotomy, you can treat it very rationally
as the outcome in logistic regression; and the test is whether
"brand" matters for the prediction. 

If purchase-intent is scaled and Brand is a dichotomy, you can get
the effective test by taking Brand as the outcome.  Then, look at the
test of whether Brand is predicted by Intent.

--
Rich Ulrich

> Date: Thu, 28 Mar 2013 10:07:25 +0000
> From: [hidden email]
> Subject: Significance test for gamma-correlations
> To: [hidden email]
>
> Hello,
>
> I have two gamma-correlation coefficients, based on correlation between brand associations and purchase intent. Those coefficients are very close to each other 0,89 vs 0,86. Is there a way to prove that there is a statistical difference between them?
>
> Thanks for your input!
> ...

[Mark Miller]

If you have Gamma coefficients,
then you probably also should have the ASE (asymptotic standard error).
Then a rough test of whether they are equal (or not) would be to compare
Gamma #A  plus/or/minus ASE #A with
Gamma#B   plus/or/minus ASE #B
If these intervals do NOT overlap then they are different
otherwise they are not statistically different


... Mark Miller

On Thu, Mar 28, 2013 at 1:40 PM, Jazgul Ismailova
<[hidden email]> wrote:

> Hello Rich,
>
> Thanks for your reply! Let me explain my question in more detail.
>
> I have 2 correlation coeficients (no matter if it is gamma or pearson): 0,89
> and 0,86. I need to show that 0,89 statistically higher than 0,86.
> I already know that both drive purchase intent, but would like to show that
> attribute one is statistically more important than attribute 2.
>
> Are there any methods I can apply?
>
> Thanks!
>
> //Jazgul
>
>
> 28 mar 2013 kl. 18:25 skrev "Rich Ulrich" <[hidden email]>:
>
> The way to test this difference between two tables of ordinal
> values is start by forgetting that you have used gamma.  Test
> for whether the association is different.  That can be fairly easy,
> using Logistic regression. For the hard way, you might figure out
> what an approximate error should be from using the approximate
> test on gamma.  (See Wikip, for details that are confusing -- note
> that for a 2x2 table [A,B; C,D]  the discordant/accordant fraction
> comes to AD/BC, an odds ratio.)
>
> If purchase-intent is a dichotomy, you can treat it very rationally
> as the outcome in logistic regression; and the test is whether
> "brand" matters for the prediction.
>
> If purchase-intent is scaled and Brand is a dichotomy, you can get
> the effective test by taking Brand as the outcome.  Then, look at the
> test of whether Brand is predicted by Intent.
>
> --
> Rich Ulrich
>
>> Date: Thu, 28 Mar 2013 10:07:25 +0000
>> From: [hidden email]
>> Subject: Significance test for gamma-correlations
>> To: [hidden email]
>
>>
>> Hello,
>>
>> I have two gamma-correlation coefficients, based on correlation between
>> brand associations and purchase intent. Those coefficients are very close to
>> each other 0,89 vs 0,86. Is there a way to prove that there is a statistical
>> difference between them?
>>
>> Thanks for your input!
>> ...

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Jo Fennessey]

I will be away from the office on Good Friday (3/29) returning Monday 4/1. 

 

[Rich Ulrich]

Oops, I slipped there -- Dichotomies simplify many analyses, but not so
much that "mean level" becomes the same as "slope."

I'll back up a step, and ask a question that I should have asked before.  Are
these "independent" ratings?  That is, are new raters making judgements in
the second case?  Are they independent?

There are two types of tests for differences in Pearson r's -- for independent
r's and for dependent r's.  If it is the same rater ("dependent"), there is potentially
more power for a test, but it is also a little more complicated.  If you need to do
that, I recommend Steiger's program, MULTICORR.  I think it is a free download
that you can Google for.  Tests for independent r's are in any textbook - using
the Fisher z-transformation works pretty well.

Further - I will mention that it is usually (not always) a bad practice to compare
correlations if-and-when you can compare regression coefficients instead. 
When you compare correlations, you build in a strong assumption that the
variances are equal.  It would usually be considered "bad inference" to
conclude that the colinear relationships differ when the regression slopes
are identical; and the difference exists in r  because one sample has a smaller
range of scores (that is, less variance).  

For independent ratings -- You can still use regression (as I suggested before)
if you want to test for differences in slope.  The correct procedure uses Rating
as outcome, Brand as a dummy variable to account for level differences (a test
that you ignore in this case), and the Group-by-Brand interaction to test whether
the association is the same.  You can do that in LR in a fashion similar to what
is so often done in OLS regression.  (See "Chow test" for one version of that.)
That's what I meant to be suggesting, in my first message.

Finally - If you have two dichotomies, the tests described above are going to
fail to account for the small counts that (probably) are involved in the differences,
so the approximations may be poor.  I would want to start with the exact N's from
the tables, to see if an exact test could apply -- McNemar's test for changes, or
some similar variation of the simple sign test.

--
Rich Ulrich

---

Date: Thu, 28 Mar 2013 20:40:55 +0000
From: [hidden email]
Subject: Re: Significance test for gamma-correlations
To: [hidden email]

Hello Rich,

Thanks for your reply! Let me explain my question in more detail. 

I have 2 correlation coeficients (no matter if it is gamma or pearson): 0,89 and 0,86. I need to show that 0,89 statistically higher than 0,86. 

I already know that both drive purchase intent, but would like to show that attribute one is statistically more important than attribute 2. 

Are there any methods I can apply?

Thanks!

//Jazgul

28 mar 2013 kl. 18:25 skrev "Rich Ulrich" <[hidden email]>:

The way to test this difference between two tables of ordinal
values is start by forgetting that you have used gamma.  Test
for whether the association is different.  That can be fairly easy,
using Logistic regression. For the hard way, you might figure out
what an approximate error should be from using the approximate
test on gamma.  (See Wikip, for details that are confusing -- note
that for a 2x2 table [A,B; C,D]  the discordant/accordant fraction
comes to AD/BC, an odds ratio.)

If purchase-intent is a dichotomy, you can treat it very rationally
as the outcome in logistic regression; and the test is whether
"brand" matters for the prediction. 

If purchase-intent is scaled and Brand is a dichotomy, you can get
the effective test by taking Brand as the outcome.  Then, look at the
test of whether Brand is predicted by Intent.

--
Rich Ulrich

> Date: Thu, 28 Mar 2013 10:07:25 +0000
> From: [hidden email]
> Subject: Significance test for gamma-correlations
> To: [hidden email]
>
> Hello,
>
> I have two gamma-correlation coefficients, based on correlation between brand associations and purchase intent. Those coefficients are very close to each other 0,89 vs 0,86. Is there a way to prove that there is a statistical difference between them?
>
> Thanks for your input!
> ...

[Jazgul Ismailova-2]

Dear Mark Miller,

This is so brilliant! Thank you very much and Happy Easter!

/Jazgul

28 mar 2013 kl. 22:51 skrev "Mark Miller" <[hidden email]>:

> If you have Gamma coefficients,
> then you probably also should have the ASE (asymptotic standard error).
> Then a rough test of whether they are equal (or not) would be to compare
> Gamma #A  plus/or/minus ASE #A with
> Gamma#B   plus/or/minus ASE #B
> If these intervals do NOT overlap then they are different
> otherwise they are not statistically different
>
>
> ... Mark Miller
>
> On Thu, Mar 28, 2013 at 1:40 PM, Jazgul Ismailova
> <[hidden email]> wrote:
>> Hello Rich,
>>
>> Thanks for your reply! Let me explain my question in more detail.
>>
>> I have 2 correlation coeficients (no matter if it is gamma or pearson): 0,89
>> and 0,86. I need to show that 0,89 statistically higher than 0,86.
>> I already know that both drive purchase intent, but would like to show that
>> attribute one is statistically more important than attribute 2.
>>
>> Are there any methods I can apply?
>>
>> Thanks!
>>
>> //Jazgul
>>
>>
>> 28 mar 2013 kl. 18:25 skrev "Rich Ulrich" <[hidden email]>:
>>
>> The way to test this difference between two tables of ordinal
>> values is start by forgetting that you have used gamma.  Test
>> for whether the association is different.  That can be fairly easy,
>> using Logistic regression. For the hard way, you might figure out
>> what an approximate error should be from using the approximate
>> test on gamma.  (See Wikip, for details that are confusing -- note
>> that for a 2x2 table [A,B; C,D]  the discordant/accordant fraction
>> comes to AD/BC, an odds ratio.)
>>
>> If purchase-intent is a dichotomy, you can treat it very rationally
>> as the outcome in logistic regression; and the test is whether
>> "brand" matters for the prediction.
>>
>> If purchase-intent is scaled and Brand is a dichotomy, you can get
>> the effective test by taking Brand as the outcome.  Then, look at the
>> test of whether Brand is predicted by Intent.
>>
>> --
>> Rich Ulrich
>>
>>> Date: Thu, 28 Mar 2013 10:07:25 +0000
>>> From: [hidden email]
>>> Subject: Significance test for gamma-correlations
>>> To: [hidden email]
>>
>>>
>>> Hello,
>>>
>>> I have two gamma-correlation coefficients, based on correlation between
>>> brand associations and purchase intent. Those coefficients are very close to
>>> each other 0,89 vs 0,86. Is there a way to prove that there is a statistical
>>> difference between them?
>>>
>>> Thanks for your input!
>>> ...
>

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Jazgul Ismailova-2]

Dear Rich Ulrich,

All variables are dichotomies (0,1).

Thanks for valuable input, I will take those into consideration if need to dig further, otherwise looking at asymptotic errors as suggested earlier seems to be the easiest way.

Happy Easter!

/Jazgul

28 mar 2013 kl. 23:33 skrev "Rich Ulrich" <[hidden email]>:

Oops, I slipped there -- Dichotomies simplify many analyses, but not so
much that "mean level" becomes the same as "slope."

I'll back up a step, and ask a question that I should have asked before.  Are
these "independent" ratings?  That is, are new raters making judgements in
the second case?  Are they independent?

There are two types of tests for differences in Pearson r's -- for independent
r's and for dependent r's.  If it is the same rater ("dependent"), there is potentially
more power for a test, but it is also a little more complicated.  If you need to do
that, I recommend Steiger's program, MULTICORR.  I think it is a free download
that you can Google for.  Tests for independent r's are in any textbook - using
the Fisher z-transformation works pretty well.

Further - I will mention that it is usually (not always) a bad practice to compare
correlations if-and-when you can compare regression coefficients instead. 
When you compare correlations, you build in a strong assumption that the
variances are equal.  It would usually be considered "bad inference" to
conclude that the colinear relationships differ when the regression slopes
are identical; and the difference exists in r  because one sample has a smaller
range of scores (that is, less variance).  

For independent ratings -- You can still use regression (as I suggested before)
if you want to test for differences in slope.  The correct procedure uses Rating
as outcome, Brand as a dummy variable to account for level differences (a test
that you ignore in this case), and the Group-by-Brand interaction to test whether
the association is the same.  You can do that in LR in a fashion similar to what
is so often done in OLS regression.  (See "Chow test" for one version of that.)
That's what I meant to be suggesting, in my first message.

Finally - If you have two dichotomies, the tests described above are going to
fail to account for the small counts that (probably) are involved in the differences,
so the approximations may be poor.  I would want to start with the exact N's from
the tables, to see if an exact test could apply -- McNemar's test for changes, or
some similar variation of the simple sign test.

--
Rich Ulrich

---

Date: Thu, 28 Mar 2013 20:40:55 +0000
From: [hidden email]
Subject: Re: Significance test for gamma-correlations
To: [hidden email]

Hello Rich,

Thanks for your reply! Let me explain my question in more detail. 

I have 2 correlation coeficients (no matter if it is gamma or pearson): 0,89 and 0,86. I need to show that 0,89 statistically higher than 0,86. 

I already know that both drive purchase intent, but would like to show that attribute one is statistically more important than attribute 2. 

Are there any methods I can apply?

Thanks!

//Jazgul

28 mar 2013 kl. 18:25 skrev "Rich Ulrich" <[hidden email]>:

The way to test this difference between two tables of ordinal
values is start by forgetting that you have used gamma.  Test
for whether the association is different.  That can be fairly easy,
using Logistic regression. For the hard way, you might figure out
what an approximate error should be from using the approximate
test on gamma.  (See Wikip, for details that are confusing -- note
that for a 2x2 table [A,B; C,D]  the discordant/accordant fraction
comes to AD/BC, an odds ratio.)

If purchase-intent is a dichotomy, you can treat it very rationally
as the outcome in logistic regression; and the test is whether
"brand" matters for the prediction. 

If purchase-intent is scaled and Brand is a dichotomy, you can get
the effective test by taking Brand as the outcome.  Then, look at the
test of whether Brand is predicted by Intent.

--
Rich Ulrich

> Date: Thu, 28 Mar 2013 10:07:25 +0000
> From: [hidden email]
> Subject: Significance test for gamma-correlations
> To: [hidden email]
>
> Hello,
>
> I have two gamma-correlation coefficients, based on correlation between brand associations and purchase intent. Those coefficients are very close to each other 0,89 vs 0,86. Is there a way to prove that there is a statistical difference between them?
>
> Thanks for your input!
> ...

[Bruce Weaver]

Mark, did you mean to say plus or minus two standard errors (i.e., an approximate 95% CI)?  Re the use of overlap of 95% CIs to judge whether the difference between two point estimates is statistically significant, see this nice note in CMAJ, which points out that this method is indeed rough (as you said), and quite conservative.

    http://www.cmaj.ca/content/166/1/65.long

The algorithms for CROSSTABS should show the formula for the ASE for (Goodman & Kruskal's) gamma.  The SE of the difference between two independent gammas would then be computed in the usual way, I think.  I.e., if SE1 and SE2 are the SEs for the two independent gammas:

compute #V1 = SE1**2.
compute #V2 = SE2**2.
compute SEdiff = SQRT(#V1 + #V2).
compute t = (Gamma1 - Gamma2) / SEdiff.

I don't have my books with me right now, and don't remember off the top of my head what the df would be for that t.  

But going back to this from the OP:

> I have 2 correlation coeficients (no matter if it is gamma or pearson): 0,89
> and 0,86. I need to show that 0,89 statistically higher than 0,86.

Does a difference that small really matter?  You'll need a HUGE sample size for it to be statistically significant.


HTH.

Mark Miller wrote

If you have Gamma coefficients,
then you probably also should have the ASE (asymptotic standard error).
Then a rough test of whether they are equal (or not) would be to compare
Gamma #A  plus/or/minus ASE #A with
Gamma#B   plus/or/minus ASE #B
If these intervals do NOT overlap then they are different
otherwise they are not statistically different


... Mark Miller

On Thu, Mar 28, 2013 at 1:40 PM, Jazgul Ismailova
<[hidden email]> wrote:
> Hello Rich,
>
> Thanks for your reply! Let me explain my question in more detail.
>
> I have 2 correlation coeficients (no matter if it is gamma or pearson): 0,89
> and 0,86. I need to show that 0,89 statistically higher than 0,86.
> I already know that both drive purchase intent, but would like to show that
> attribute one is statistically more important than attribute 2.
>
> Are there any methods I can apply?
>
> Thanks!
>
> //Jazgul
>
>
> 28 mar 2013 kl. 18:25 skrev "Rich Ulrich" <[hidden email]>:
>
> The way to test this difference between two tables of ordinal
> values is start by forgetting that you have used gamma.  Test
> for whether the association is different.  That can be fairly easy,
> using Logistic regression. For the hard way, you might figure out
> what an approximate error should be from using the approximate
> test on gamma.  (See Wikip, for details that are confusing -- note
> that for a 2x2 table [A,B; C,D]  the discordant/accordant fraction
> comes to AD/BC, an odds ratio.)
>
> If purchase-intent is a dichotomy, you can treat it very rationally
> as the outcome in logistic regression; and the test is whether
> "brand" matters for the prediction.
>
> If purchase-intent is scaled and Brand is a dichotomy, you can get
> the effective test by taking Brand as the outcome.  Then, look at the
> test of whether Brand is predicted by Intent.
>
> --
> Rich Ulrich
>
>> Date: Thu, 28 Mar 2013 10:07:25 +0000
>> From: [hidden email]
>> Subject: Significance test for gamma-correlations
>> To: [hidden email]
>
>>
>> Hello,
>>
>> I have two gamma-correlation coefficients, based on correlation between
>> brand associations and purchase intent. Those coefficients are very close to
>> each other 0,89 vs 0,86. Is there a way to prove that there is a statistical
>> difference between them?
>>
>> Thanks for your input!
>> ...

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Tierney, Mary Lou]

I am out of the office and offline today, Friday 29 March.

Please contact:

 

·         [hidden email] for questions or assistance with FY14 proposal process questions including Idea Market, Strategy Market or Innovation@MITRE (sponsor portal)

·         [hidden email] for questions or assistance with MIP funded project processes or the website

·         [hidden email] for questions about or assistance with MIP ProjectPages (page functionality, content on the pages, i.e. PI name, project title, etc.)

·         [hidden email] for questions about charge numbers

·         [hidden email] for assistance with other Innovation Zone sites, such as CI&T InZone

 

Regards,

Mary Lou