Quick question - advice on statistic to use for criterion validity with continuous measure, categorical outcome

[Hunna Watson]

I am rusty and need a quick answer and would appreciate help from listers -
 
I am investigating the criterion validity of a continuous psychometric measure (symptom severity for a mental health problem). What statistic is best to use to examine the relationship to a categorical outcome (presence or absence of a physiological feature of the disorder)? I understand pearson correlation for continuous variables, spearman rho for ordinal, but what for categorical..
 
Hunna Watson
 

====================To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Mark A Davenport MADAVENP]

All,

I have a colleague that is wanting to compare the proportion of
satisfied/not satisfied responses on faculty evals.  It looks like one
school (a comparitively small one) wants to know if their evals are
different than those of the university.  At first blush, this seemed to me
to be a simple z-test comparing a ratio of sat/not sat responses for the
school to the university as a whole.  However, the question came up about
removing this department's evals from the population before the pop
proportion is calculated.  My response was: If you want to compare the
sample to it's parent population, leave it in as the sample is a part of
the population.  Suddenly I grew concerned.  My concern is when we get to
the College of Arts and Sciences (a comparatively LARGE and diverse part
of the population), we end up comparing A & S to a pop that is more like
themselves than for any other smaller school. My sense is I need to weigh
the data or correct the procedure such that comparisons are more
equitable.    I have advised this person not to compare schools but,
inevitably, the schools are likely to try compare themselves anyway.

Suggestions?

Mark

***************************************************************************************************************************************************************
Mark A. Davenport Ph.D.
Senior Research Analyst
Office of Institutional Research
The University of North Carolina at Greensboro
336.256.0395
[hidden email]

'An approximate answer to the right question is worth a good deal more
than an exact answer to an approximate question.' --a paraphrase of J. W.
Tukey (1962)

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Swank, Paul R]

Why not do a logistic regression with odds ratios. Or if a correlation
is a must then the Pearson is the same as a point biserial. Otherwise it
is polychoric correlations, which in this case is the biserial. It is
based on the idea that the dichotomy is based on an underlying normal.

Paul R. Swank, Ph.D.
Professor and Director of Research
Children's Learning Institute
University of Texas Health Science Center - Houston


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Hunna Watson
Sent: Monday, April 14, 2008 11:04 AM
To: [hidden email]
Subject: Quick question - advice on statistic to use for criterion
validity with continuous measure, categorical outcome

I am rusty and need a quick answer and would appreciate help from
listers -

I am investigating the criterion validity of a continuous psychometric
measure (symptom severity for a mental health problem). What statistic
is best to use to examine the relationship to a categorical outcome
(presence or absence of a physiological feature of the disorder)? I
understand pearson correlation for continuous variables, spearman rho
for ordinal, but what for categorical..

Hunna Watson


==========To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Swank, Paul R]

There are a number of ways one might go about this. There is the
generalized linear model with a binomial distribution and a logit link
function to get at whether there is a difference overall. Then one could
also use trinary coding to get the effect size for each department
relative to the grand mean or one might find the median proportion for
the whole school and covert each departmental proportion to a binary by
comparing to the median proportion, However, in every case you will have
one very large problem. With large differences in the number of faculty
per department, you will have large differences in the power for
individual comparisons with either of these tests.

Paul R. Swank, Ph.D.
Professor and Director of Research
Children's Learning Institute
University of Texas Health Science Center - Houston


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Mark A Davenport MADAVENP
Sent: Monday, April 14, 2008 12:21 PM
To: [hidden email]
Subject: Proportion test question

All,

I have a colleague that is wanting to compare the proportion of
satisfied/not satisfied responses on faculty evals.  It looks like one
school (a comparitively small one) wants to know if their evals are
different than those of the university.  At first blush, this seemed to
me
to be a simple z-test comparing a ratio of sat/not sat responses for the
school to the university as a whole.  However, the question came up
about
removing this department's evals from the population before the pop
proportion is calculated.  My response was: If you want to compare the
sample to it's parent population, leave it in as the sample is a part of
the population.  Suddenly I grew concerned.  My concern is when we get
to
the College of Arts and Sciences (a comparatively LARGE and diverse part
of the population), we end up comparing A & S to a pop that is more like
themselves than for any other smaller school. My sense is I need to
weigh
the data or correct the procedure such that comparisons are more
equitable.    I have advised this person not to compare schools but,
inevitably, the schools are likely to try compare themselves anyway.

Suggestions?

Mark

************************************************************************
************************************************************************
***************
Mark A. Davenport Ph.D.
Senior Research Analyst
Office of Institutional Research
The University of North Carolina at Greensboro
336.256.0395
[hidden email]

'An approximate answer to the right question is worth a good deal more
than an exact answer to an approximate question.' --a paraphrase of J.
W.
Tukey (1962)

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Peck, Jon]

At a very simple level, the PROPOR extension command available for SPSS 16 from SPSS Developer Central will give you a table of confidence intervals for each proportion and a ci for each difference, assuming that the data have been partitioned.

Regards,
Jon Peck

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Swank, Paul R
Sent: Monday, April 14, 2008 12:21 PM
To: [hidden email]
Subject: Re: [SPSSX-L] Proportion test question

There are a number of ways one might go about this. There is the
generalized linear model with a binomial distribution and a logit link
function to get at whether there is a difference overall. Then one could
also use trinary coding to get the effect size for each department
relative to the grand mean or one might find the median proportion for
the whole school and covert each departmental proportion to a binary by
comparing to the median proportion, However, in every case you will have
one very large problem. With large differences in the number of faculty
per department, you will have large differences in the power for
individual comparisons with either of these tests.

Paul R. Swank, Ph.D.
Professor and Director of Research
Children's Learning Institute
University of Texas Health Science Center - Houston


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Mark A Davenport MADAVENP
Sent: Monday, April 14, 2008 12:21 PM
To: [hidden email]
Subject: Proportion test question

All,

I have a colleague that is wanting to compare the proportion of
satisfied/not satisfied responses on faculty evals.  It looks like one
school (a comparitively small one) wants to know if their evals are
different than those of the university.  At first blush, this seemed to
me
to be a simple z-test comparing a ratio of sat/not sat responses for the
school to the university as a whole.  However, the question came up
about
removing this department's evals from the population before the pop
proportion is calculated.  My response was: If you want to compare the
sample to it's parent population, leave it in as the sample is a part of
the population.  Suddenly I grew concerned.  My concern is when we get
to
the College of Arts and Sciences (a comparatively LARGE and diverse part
of the population), we end up comparing A & S to a pop that is more like
themselves than for any other smaller school. My sense is I need to
weigh
the data or correct the procedure such that comparisons are more
equitable.    I have advised this person not to compare schools but,
inevitably, the schools are likely to try compare themselves anyway.

Suggestions?

Mark

************************************************************************
************************************************************************
***************
Mark A. Davenport Ph.D.
Senior Research Analyst
Office of Institutional Research
The University of North Carolina at Greensboro
336.256.0395
[hidden email]

'An approximate answer to the right question is worth a good deal more
than an exact answer to an approximate question.' --a paraphrase of J.
W.
Tukey (1962)

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Richard Ristow]

At 01:20 PM 4/14/2008, Mark A Davenport MADAVENP wrote:

>I have a colleague wanting to compare the proportion of
>satisfied/not satisfied responses on faculty evals.  One school (a
>comparatively small one) wants to know if their evals are different
>than those of the university.
>
>The question came up about removing this department's evals from the
>population before the pop proportion is calculated.  My response
>was: If you want to compare the sample to it's parent population,
>leave it in as the sample is a part of the population.

Briefly, this issue, in its general form, has been discussed on the list.

The answer seems to be, statistical tests, as we have them, compare
different groups; they don't compare a group to part of itself. So
you do want to remove the responses of the school you're testing
individually, whether it's a big school or a small one.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[cbautista]

Hi list,

I have a binary longitudinal data (5 times) and have applied the Cochran's
Q test to detect any significant difference over time, but now, I'd like
to know/evaluate the extent of change. Any idea on this?

Thanks in advance,

/Christian

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Hector Maletta]

Christian,
The "extent of change" is something you can model in various ways, and
analyze with several approaches and statistical procedures. Do you have a
single binary variable observed 5 times, or several? Or you also have other
variables influencing the behavior of the binary outcome variables? Are you
interested in the total net cumulative change from the first to the fifth
observation, or would also try to explain the ups and downs of the series
along the way? Do you have in mind to measure just the behavior of your
binary or binaries, or the impact of other variables on your binary outcome,
observed 5 times (as in repeated measures), or you are rather interested in
analyzing or modeling a process of change over time? Are the other
(explanatory, predictor, causal) variables measured only once, or measured
again at each of the 5 occasions, along with the effects? All these
questions may impinge on the kind of analysis you want to perform.

Hector

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Christian Bautista
Sent: 17 April 2008 17:00
To: [hidden email]
Subject: Cochran's Q test

Hi list,

I have a binary longitudinal data (5 times) and have applied the Cochran's
Q test to detect any significant difference over time, but now, I'd like
to know/evaluate the extent of change. Any idea on this?

Thanks in advance,

/Christian

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[cbautista]

Thanks Hector for your reply.  First,  I have a single variable observed 5
times, and I am interested in the total net cumulative change from the
first to the fifth. In my study, during each visit, they received
counseling, and I'd like to know whether some variables changed over time,
for example: "wash hands before eating".

thanks,

/Christian




"Hector Maletta" <[hidden email]>
04/17/2008 04:21 PM

To
<[hidden email]>, <[hidden email]>
cc

Subject
RE: Cochran's Q test






Christian,
The "extent of change" is something you can model in various ways, and
analyze with several approaches and statistical procedures. Do you have a
single binary variable observed 5 times, or several? Or you also have
other
variables influencing the behavior of the binary outcome variables? Are
you
interested in the total net cumulative change from the first to the fifth
observation, or would also try to explain the ups and downs of the series
along the way? Do you have in mind to measure just the behavior of your
binary or binaries, or the impact of other variables on your binary
outcome,
observed 5 times (as in repeated measures), or you are rather interested
in
analyzing or modeling a process of change over time? Are the other
(explanatory, predictor, causal) variables measured only once, or measured
again at each of the 5 occasions, along with the effects? All these
questions may impinge on the kind of analysis you want to perform.

Hector

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Christian Bautista
Sent: 17 April 2008 17:00
To: [hidden email]
Subject: Cochran's Q test

Hi list,

I have a binary longitudinal data (5 times) and have applied the Cochran's
Q test to detect any significant difference over time, but now, I'd like
to know/evaluate the extent of change. Any idea on this?

Thanks in advance,

/Christian

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD