stats question - statistical significance

[Tanya Dockendorf-2]

Dear listers,

I have a question about an appropriate test to use for testing statistical significance. I am calculating a score by subtracting % of respondents giving a rating of 1-6 from % of respondents giving a rating of 9-10 on a 10-point scale.  This score is calculated for each month of the year. What is the correct test to use to determine if the changes in this score are statistically significant from month to month? Is it ok to use the z-test or should it be something else?

Thanks in advance!

Tanya

====================To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Katkowski, David]

Tanya,

Looks like you are interested in differences between proportions. You can use chi square if you want to check significance across all months, as a whole. Data setup would probably be different from what you would expect. If you want to check from month to month, you can use the z test for differences between proportions.

-David

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Tanya Dockendorf
Sent: Monday, December 10, 2007 11:50 AM
To: [hidden email]
Subject: stats question - statistical significance

Dear listers,

I have a question about an appropriate test to use for testing statistical significance. I am calculating a score by subtracting % of respondents giving a rating of 1-6 from % of respondents giving a rating of 9-10 on a 10-point scale.  This score is calculated for each month of the year. What is the correct test to use to determine if the changes in this score are statistically significant from month to month? Is it ok to use the z-test or should it be something else?

Thanks in advance!

Tanya

=======
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.5.503 / Virus Database: 269.16.17/1179 - Release Date: 12/9/2007 11:06 AM


No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.5.503 / Virus Database: 269.16.17/1179 - Release Date: 12/9/2007 11:06 AM

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Tanya Dockendorf-2]

Yes, they are... I guess I am just wondering if it is still ok to treat this calculated score as a "proportion". It is being calculated from the 2 proportions (proportion of high ratings minus proportion of low ratings), but can I still use statistical tests that are normally used for differences in proportions?

Tanya


-----Original Message-----
From: Swank, Paul R [mailto:[hidden email]]
Sent: Monday, December 10, 2007 1:36 PM
To: Tanya Dockendorf
Subject: RE: stats question - statistical significance

Are the respondents independent from month to month?

Paul R. Swank, Ph.D.
Professor and Director of Research
Children's Learning Institute
University of Texas Health Science Center - Houston


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Tanya Dockendorf
Sent: Monday, December 10, 2007 10:50 AM
To: [hidden email]
Subject: stats question - statistical significance

Dear listers,

I have a question about an appropriate test to use for testing
statistical significance. I am calculating a score by subtracting % of
respondents giving a rating of 1-6 from % of respondents giving a rating
of 9-10 on a 10-point scale.  This score is calculated for each month of
the year. What is the correct test to use to determine if the changes in
this score are statistically significant from month to month? Is it ok
to use the z-test or should it be something else?

Thanks in advance!

Tanya

==========To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[David Taber]

Tanya:

  don't know if there is an answer to your question but when you calculate the net promoter (NP) score, the z-test is not applicable.  The reason for this is that when when you track this measure you may be double-counting a change in the proprotion of respondents.  This is because when the propotion who answered 9 or 10 goes up by 1 pct point you will be doubling that increase if the propotion who answered 1 to 6 decreased by 1 pct point.

  The resulting NP score is actually not a propotion -- it's just a calculated score. You would expect much greater variation with a NP score than a top 2-box percentage.

  on a seperate note, an easy way to track net promoter scores in spss is to recode 9-10 to +100, 7-8 to 0, and 1-6 (or 0-6) to  -100 -- then just compare the mean of the new variable over over time.

  you might want to check with the company that advocates this technique
  http://www.satmetrix.com/netpromoter/index.htm
  to see what they say about statitical differences over time


"Katkowski, David" <[hidden email]> wrote:
  Tanya,

Looks like you are interested in differences between proportions. You can use chi square if you want to check significance across all months, as a whole. Data setup would probably be different from what you would expect. If you want to check from month to month, you can use the z test for differences between proportions.

-David

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Tanya Dockendorf
Sent: Monday, December 10, 2007 11:50 AM
To: [hidden email]
Subject: stats question - statistical significance

Dear listers,

I have a question about an appropriate test to use for testing statistical significance. I am calculating a score by subtracting % of respondents giving a rating of 1-6 from % of respondents giving a rating of 9-10 on a 10-point scale. This score is calculated for each month of the year. What is the correct test to use to determine if the changes in this score are statistically significant from month to month? Is it ok to use the z-test or should it be something else?

Thanks in advance!

Tanya

=======
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.5.503 / Virus Database: 269.16.17/1179 - Release Date: 12/9/2007 11:06 AM


No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.5.503 / Virus Database: 269.16.17/1179 - Release Date: 12/9/2007 11:06 AM

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Tanya Dockendorf-2]

Thanks so much. You confirmed my doubts about using z-test. The link and the rescaling suggestion are also very helpful!

Tanya

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of David Taber
Sent: Monday, December 10, 2007 2:05 PM
To: [hidden email]
Subject: Re: stats question - statistical significance

Tanya:

  don't know if there is an answer to your question but when you calculate the net promoter (NP) score, the z-test is not applicable.  The reason for this is that when when you track this measure you may be double-counting a change in the proprotion of respondents.  This is because when the propotion who answered 9 or 10 goes up by 1 pct point you will be doubling that increase if the propotion who answered 1 to 6 decreased by 1 pct point.

  The resulting NP score is actually not a propotion -- it's just a calculated score. You would expect much greater variation with a NP score than a top 2-box percentage.

  on a seperate note, an easy way to track net promoter scores in spss is to recode 9-10 to +100, 7-8 to 0, and 1-6 (or 0-6) to  -100 -- then just compare the mean of the new variable over over time.

  you might want to check with the company that advocates this technique
  http://www.satmetrix.com/netpromoter/index.htm
  to see what they say about statitical differences over time


"Katkowski, David" <[hidden email]> wrote:
  Tanya,

Looks like you are interested in differences between proportions. You can use chi square if you want to check significance across all months, as a whole. Data setup would probably be different from what you would expect. If you want to check from month to month, you can use the z test for differences between proportions.

-David

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Tanya Dockendorf
Sent: Monday, December 10, 2007 11:50 AM
To: [hidden email]
Subject: stats question - statistical significance

Dear listers,

I have a question about an appropriate test to use for testing statistical significance. I am calculating a score by subtracting % of respondents giving a rating of 1-6 from % of respondents giving a rating of 9-10 on a 10-point scale. This score is calculated for each month of the year. What is the correct test to use to determine if the changes in this score are statistically significant from month to month? Is it ok to use the z-test or should it be something else?

Thanks in advance!

Tanya

=======
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.5.503 / Virus Database: 269.16.17/1179 - Release Date: 12/9/2007 11:06 AM


No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.5.503 / Virus Database: 269.16.17/1179 - Release Date: 12/9/2007 11:06 AM

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD