comparison question

[Luo, Peter]

Thank Lisa for adding the subject line to my original message.

I totally agree with all the comments. However, we didn't have access to the
raw data, so have to work with the aggregated numbers.

Let me re-phrase the situation:

A survey was conducted among 150 randomly selected customers; each customer
was invited to express his/her opinion on 15 brands. For every brand, the
option was: 1-'more likely', 2-'the same', 3-'less likely'. Naturally, not
every customer answered all 15 questions.

The data come back and was tabulated; now our researcher has the report in
front of her. What she saw on the report is: for every brand, the percent of
customers selected 1, percent selected 2, percent 3, and the number of
answers.

Then she
1. calculated a 'net gain' on each brand (net gain= % more likely - % less
likely), ignoring % the same
2. calculated a weighted average net gain across all 15 brands

Her question: when comparing the brand specific net gain against the average
net gain, can I tell which brand's net gain is significantly from the
average?

Here is what I was thinking to solve her request

1. since brand specific 'net gain' is the difference between two
proportions, I can calculate the standard deviation of that difference

2. assume the average net gain (cross 15 brands) is fixed

3. conduct a z or t test on the difference between the brand specific net
gain and the average net gain, using mean comparison rather than proportion
comparison since 'net gain' could be negative. The test will need the
standard deviation derived from the 1st step.

Need comments and suggestions.

-----Original Message-----
From: Lisa Stickney [mailto:[hidden email]]
Sent: Thursday, July 06, 2006 12:02 PM
To: Luo, Peter
Cc: [hidden email]
Subject: Re: comparision question

Hi Peter,

     I can understand why you're puzzled.  Before I try to answer your
question, I have a few of my own.

> For each brand she deducted a 'net' gain, which is the balance of the % of
> consumers who answered "more likely' and % who answered 'less likely.'
> It's
> a 3 choice question: 1-less likely, 2-the same, 3-more likely.
>

I'm not entirely sure what your friend did here:

Did she compute a net gain (% more likely - % less likely) or did she deduct

the "net gain" from something else?  If so, what?

Assuming the former (% more likely - % less likely), the question becomes,
why? I'm not sure what the point of this is.  If you do this, you omit all
repsondents who selected #2 (the same) from your data.  What is she trying
to find out about the brands?  Why not just look at the average response for

the question?

>
>
> Then she took the average of the net gains cross all 15 brands.
>

Again, why?  Are all 15 brands releated in some way?  Is there some
theoretical reason for averaging the difference of the extremes in
repsondent scores for 15 different brands?  More information would help
here.

>
>
> Her question is: to what extent one can determine if a particular brand
> was
> significantly more (or less) favored by these consumers, say if brand A
> got
> a net gain of +10%, but the average net gain is +15%, can I be confident
> to
> say the brand A is less favored?
>
>  the number of consumers who answered each brand varied from 35 to 74.

I don't think so (your concern is valid).  Since Brand A is in the average,
this muddies already murky waters.  Again, I'm not sure what the point is or

what would be gained by performing a comparison in this manner.

It's my understanding that in general & if possible, it's best to leave data

in it's original form.  Not only are do you deal with the participants'
actual responses, but it's much easier to interpret.

If your friend wants to see if there are significant differences between
consumer's brand perceptions, then run an ANOVA on the original data.  Plus,

if she did that she'd retain all participants' repsonses and not not just
those who selected an extreme.

However, that said, the size of this sample is a major concern.  I seriously

doubt there's enough power to pick up anything but the largest of
differences (e.g. I strongly agree w/ 'I prefer to eat in a 5 star
restaurant than a cafeteria').  Is there any way your friend can collect
more data?  Also, if she's going to recollect the data, she might want to
consider expanding her response set to 5 or 6 possible answers (e.g. 1-much
less likely, 2-less likely 3-the same, 4-more likely, 5-much more likely).

Good luck to your friend, and I hope this helps.

    Best,
        Lisa

P.S.  When e-mailing this list, it's generally good to include a brief
description in the subject line -- I took the liberty of filling one in for
you.


Lisa T. Stickney
Ph.D. Candidate
The Fox School of Business
      and Management
Temple University
[hidden email]