Re: four-point Likert scale items

[Art Kendall-2]

Thank you for the articles.

To some extent each article uses a straw man argument with regard to the OP question.� They both discuss situations where the construct is represented by a single variable.� A Likert item is not a Likert scale. Further, not every set of ordered values on the item response scale has been shown to be close to interval level in the way people respond the way Likert items have.

They also seem to assume that appropriateness of an approach is a strict dichotomy. Correct or not correct.� Warts are different from fatal diseases.

The syntax I posted shows that even with a single variable to represent a construct with distortions of the measurement process, the results are often not very different.� The OP seemed to be asking about a summative scale.

That being said, at the design stage I would try very hard to have as high a quality of measurement as is practical. That would include pretesting the response scale, using as many levels in the response scale as members of the respondent population can deal with, care in choosing anchors for the response values, and including numeric as well as verbal anchors in the instrument.

� In looking at data that has already been gathered it is often necessary to prescribe large doses of salt and go with what you have.

These days , especially when, the measurement process is not of very good quality, now that there are procedures such as CATREG and CATPCA I recommend actually checking to see if the discrepancy from the perfectly interval level makes a substantive difference in the conclusions.� If it does not make a meaningful difference I would present the conventional results and mention that using purely ordinal level measurement assumptions did not make a substantive difference.� If there were, a substantive difference in the conclusions, I would present the results from the ordinal level solution and mention how the results would be different in a conventional approach.

The more complex and less widely familiar approach may be necessary, but often the conventional does not not yield meaningfully different conclusions.

The second article also has a serious flaw.� It talks about accepting� the null hypothesis. The null hypothesis is accepted before the study is done(the default) (a priori) (status quo ante).� If there is insufficient evidence to accept (or "go with") the alternative hypothesis, the null hypothesis is kept (retained, stayed with).

Art Kendall
Social Research Consultants
Bruce Weaver wrote:

Hi Art.� Here are the two articles I mentioned in my post.
Cheers,
Bruce

On Mon, Jun 22, 2009 at 4:46 PM, Art Kendall <[hidden email]> wrote:

Bruce if you have access to the article would you look to see if they are talking about a summative Likert scale made of of several items that some would consider ordinal or if they are talking about a single item with a Likert response scale as the complete representation to the underlying construct? � Are they distinguishing ordinal level of the representation vs the intrinsic ordinal level of the underlying construct.


Art Kendall

Bruce Weaver wrote:

On Jun 22, 2:52 pm, Rich Ulrich <[hidden email]> wrote:

On Mon, 22 Jun 2009 05:30:31 -0700 (PDT), khacker





<[hidden email]> wrote:

On Jun 20, 10:21 pm, Ray Koopman <[hidden email]> wrote:

On Jun 20, 5:50 pm, khacker <[hidden email]> wrote:

[snip]

Treat them as interval. The argument is that although they are not
exactly interval -- nothing is -- they are unlikely to be so far
from interval that treating them as interval will lead you to wrong
substantive conclusions.

Thanks for the advice. This is usually what communication researchers
do and also what psychologists do in their analysis. But I have a
statistician colleague who says that Likert scale data are clearly
ordinal data and must be treated as such for a valid statistical
analysis.
KH

I'm always curious about who is spreading whatever it is
that I consider "obsolete statistical advice". � A dozen years
ago, posters who wanted to do stepwise regressions occasionally
came up with textbook citations for their position -- *invariably* � misparaphrased or misunderstood. � Or very old. � I debunked several.

This question about Likert has come up before, and the support
has always been secondhand. � Can your colleague provide a textbook
reference in support? � � I can imagine that this is � still "good
advice" � in the context of some particular paradigm of data
collection and analysis. � I would be particularly interested if
there are arguments to that � effect.

You say that psychologists usually treat Likert as interval,
but from my own experience, I thought that it was psychologists
who had acted as the vector. � I've known professionals who
learned their statistics in psychology classes, being taught
by psychologists-turned-professor-of-statistics in the 1970s � who
had learned their own trade in the 1950s. � Siegal published his
nice little book on nonparametric tests in 1956, which led (I
think) to a nice little fad, but a fad that did not much affect
people who were more thoroughly grounded as statisticians.

--
Rich Ulrich

An article by Susan Jamieson in Medical Education (MEDICAL EDUCATION
2004; 38: 1212–1218) gives some references that take the hard-line
view on this. � Apparently one of them (Kuzon et al, reference below)
describes analysis of ordinal data with parametric tests as the first
deadly sin of statistical analysis.

Kuzon WM Jr, Urbanchek MG, McCabe S. The seven deadly sins of
statistical analysis. Ann Plastic Surg 1996; 37:265–72.

--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/
"When all else fails, RTFM."

--
Bruce Weaver
Research Associate, Centre for Research on Safe Driving, Lakehead University
Assistant Professor of Biostatistics, Human Sciences Division
Northern Ontario School of Medicine, West Campus
955 Oliver Road, MS-2006, Thunder Bay, ON � P7B 5E1

Tel: � 807-346-7704 � � � E-mail: � [hidden email]
Fax: � 807-766-7362 � � � Web: � http://sites.google.com/a/lakeheadu.ca/bweaver/

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Oliver, Richard]

Back in the stone age when I took stats courses, we didn’t ‘accept’ the null hypothesis; we “failed to reject” it.  

 

---

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Art Kendall
Sent: Tuesday, June 23, 2009 7:50 AM
To: [hidden email]
Subject: Re: four-point Likert scale items

 

Thank you for the articles.

To some extent each article uses a straw man argument with regard to the OP question. They both discuss situations where the construct is represented by a single variable. A Likert item is not a Likert scale. Further, not every set of ordered values on the item response scale has been shown to be close to interval level in the way people respond the way Likert items have.

They also seem to assume that appropriateness of an approach is a strict dichotomy. Correct or not correct. Warts are different from fatal diseases.

The syntax I posted shows that even with a single variable to represent a construct with distortions of the measurement process, the results are often not very different. The OP seemed to be asking about a summative scale.

That being said, at the design stage I would try very hard to have as high a quality of measurement as is practical. That would include pretesting the response scale, using as many levels in the response scale as members of the respondent population can deal with, care in choosing anchors for the response values, and including numeric as well as verbal anchors in the instrument.

In looking at data that has already been gathered it is often necessary to prescribe large doses of salt and go with what you have.

These days , especially when, the measurement process is not of very good quality, now that there are procedures such as CATREG and CATPCA I recommend actually checking to see if the discrepancy from the perfectly interval level makes a substantive difference in the conclusions. If it does not make a meaningful difference I would present the conventional results and mention that using purely ordinal level measurement assumptions did not make a substantive difference. If there were, a substantive difference in the conclusions, I would present the results from the ordinal level solution and mention how the results would be different in a conventional approach.

The more complex and less widely familiar approach may be necessary, but often the conventional does not not yield meaningfully different conclusions.

The second article also has a serious flaw. It talks about accepting the null hypothesis. The null hypothesis is accepted before the study is done(the default) (a priori) (status quo ante). If there is insufficient evidence to accept (or "go with") the alternative hypothesis, the null hypothesis is kept (retained, stayed with).

Art Kendall
Social Research Consultants
Bruce Weaver wrote:

Hi Art. Here are the two articles I mentioned in my post.
Cheers,
Bruce

On Mon, Jun 22, 2009 at 4:46 PM, Art Kendall <[hidden email]> wrote:

Bruce if you have access to the article would you look to see if they are talking about a summative Likert scale made of of several items that some would consider ordinal or if they are talking about a single item with a Likert response scale as the complete representation to the underlying construct? Are they distinguishing ordinal level of the representation vs the intrinsic ordinal level of the underlying construct.


Art Kendall

Bruce Weaver wrote:

On Jun 22, 2:52 pm, Rich Ulrich <[hidden email]> wrote:

On Mon, 22 Jun 2009 05:30:31 -0700 (PDT), khacker





<[hidden email]> wrote:

On Jun 20, 10:21 pm, Ray Koopman <[hidden email]> wrote:

On Jun 20, 5:50 pm, khacker <[hidden email]> wrote:

[snip]

Treat them as interval. The argument is that although they are not
exactly interval -- nothing is -- they are unlikely to be so far
from interval that treating them as interval will lead you to wrong
substantive conclusions.

Thanks for the advice. This is usually what communication researchers
do and also what psychologists do in their analysis. But I have a
statistician colleague who says that Likert scale data are clearly
ordinal data and must be treated as such for a valid statistical
analysis.
KH

I'm always curious about who is spreading whatever it is
that I consider "obsolete statistical advice". A dozen years
ago, posters who wanted to do stepwise regressions occasionally
came up with textbook citations for their position -- *invariably* misparaphrased or misunderstood. Or very old. I debunked several.

This question about Likert has come up before, and the support
has always been secondhand. Can your colleague provide a textbook
reference in support? I can imagine that this is still "good
advice" in the context of some particular paradigm of data
collection and analysis. I would be particularly interested if
there are arguments to that effect.

You say that psychologists usually treat Likert as interval,
but from my own experience, I thought that it was psychologists
who had acted as the vector. I've known professionals who
learned their statistics in psychology classes, being taught
by psychologists-turned-professor-of-statistics in the 1970s who
had learned their own trade in the 1950s. Siegal published his
nice little book on nonparametric tests in 1956, which led (I
think) to a nice little fad, but a fad that did not much affect
people who were more thoroughly grounded as statisticians.

--
Rich Ulrich

An article by Susan Jamieson in Medical Education (MEDICAL EDUCATION
2004; 38: 1212–1218) gives some references that take the hard-line
view on this. Apparently one of them (Kuzon et al, reference below)
describes analysis of ordinal data with parametric tests as the first
deadly sin of statistical analysis.

Kuzon WM Jr, Urbanchek MG, McCabe S. The seven deadly sins of
statistical analysis. Ann Plastic Surg 1996; 37:265–72.

--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/
"When all else fails, RTFM."

 

--
Bruce Weaver
Research Associate, Centre for Research on Safe Driving, Lakehead University
Assistant Professor of Biostatistics, Human Sciences Division
Northern Ontario School of Medicine, West Campus
955 Oliver Road, MS-2006, Thunder Bay, ON P7B 5E1

Tel: [phone] E-mail: [hidden email]
Fax: [phone] Web: http://sites.google.com/a/lakeheadu.ca/bweaver/

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD