test-retest reliability and Spearman-Brown split half coefficient

[Paul Mcgeoghan]

Hi,

I have been reading a lot of websites relating to reliability and agreement this week and have a
customer who has carried out the following:

She has 9 people who answered nominal questions:
Yes, No, Don't Know
on 2 seperate occasions.
She is interested in knowing whether the respondents reliably answer the questions the same over
the 2 time periods.

She also has a number of likert questions also where the same 9 respondents answered the questions
at time 1 and time 2 and wants to see if they have reliably answered each question the same over the
2 time periods.

From what I have read, this seems to be referred to as test-retest reliability and a correlation
between each pair of questions seems to be the way to approach it.

So for the likert scale questions, I could just do a Spearman's correlation coefficient?
For the nominal questions, what can I use?

I have also seen reference to Spearman-Brown split half coefficient (Analyse Scale Reliability and
Split-Half).
Can this be used to compare each pair of questions at the 2 time intervals, and does the data have
to be likert-scale in this instance?

Hope I have explained the problem clearly.

Thanks,
Paul


==================
Paul McGeoghan,
Application support specialist (Statistics and Databases),
University Infrastructure Group (UIG),
Information Services,
Cardiff University.
Tel. 02920 (875035).

[statisticsdoc]

Paul,

I imagine that you have suggested that more data be collected (a sample of
nine is not enough to accurately estimate test-retest reliability).

Having said that, for the Likert items, you want to compute the correlation
between two time points, not the split-half reliability (which applies to
ratings that are summed into scale scores at the same point in time).  Is
your customer interested in knowing about the test-retest reliability of
single items, or the test-retest reliability of a score that has been
computed from these items.  For single Likert items, a Spearman rank-order
correlation would be appropriate, given the ordinal nature of the data.  For
computing the test-retest reliability of the scale total, you could use
Pearson's correlation if you are willing to assume that the sums behave like
interval level data (this is a topic that is frequently discussed on this
list, as you know), otherwise stick with Spearman's coefficient.

HTH,

Stephen Brand

For personalized and professional consultation in statistics and research
design, visit
www.statisticsdoc.com


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
Paul Mcgeoghan
Sent: Thursday, January 25, 2007 10:03 AM
To: [hidden email]
Subject: test-retest reliability and Spearman-Brown split half
coefficient


Hi,

I have been reading a lot of websites relating to reliability and agreement
this week and have a
customer who has carried out the following:

She has 9 people who answered nominal questions:
Yes, No, Don't Know
on 2 seperate occasions.
She is interested in knowing whether the respondents reliably answer the
questions the same over
the 2 time periods.

She also has a number of likert questions also where the same 9 respondents
answered the questions
at time 1 and time 2 and wants to see if they have reliably answered each
question the same over the
2 time periods.

From what I have read, this seems to be referred to as test-retest
reliability and a correlation
between each pair of questions seems to be the way to approach it.

So for the likert scale questions, I could just do a Spearman's correlation
coefficient?
For the nominal questions, what can I use?

I have also seen reference to Spearman-Brown split half coefficient (Analyse
Scale Reliability and
Split-Half).
Can this be used to compare each pair of questions at the 2 time intervals,
and does the data have
to be likert-scale in this instance?

Hope I have explained the problem clearly.

Thanks,
Paul


==================
Paul McGeoghan,
Application support specialist (Statistics and Databases),
University Infrastructure Group (UIG),
Information Services,
Cardiff University.
Tel. 02920 (875035).

[Paul Mcgeoghan]

Stephen,

Thanks, I was also thinking along the lines of Spearman rank-order as a possibility.
She is only interested in comparing the single Likert items.

However, she also has the problem with Yes, No and Don't Know questions which to me would be
considered nominal rather than likert scale so what does she do for these type of questions?

Paul


==================
Paul McGeoghan,
Application support specialist (Statistics and Databases),
University Infrastructure Group (UIG),
Information Services,
Cardiff University.
Tel. 02920 (875035).

>>> "Statisticsdoc" <[hidden email]> 25/01/2007 15:45:25 >>>
Paul,

I imagine that you have suggested that more data be collected (a sample of
nine is not enough to accurately estimate test-retest reliability).

Having said that, for the Likert items, you want to compute the correlation
between two time points, not the split-half reliability (which applies to
ratings that are summed into scale scores at the same point in time).  Is
your customer interested in knowing about the test-retest reliability of
single items, or the test-retest reliability of a score that has been
computed from these items.  For single Likert items, a Spearman rank-order
correlation would be appropriate, given the ordinal nature of the data.  For
computing the test-retest reliability of the scale total, you could use
Pearson's correlation if you are willing to assume that the sums behave like
interval level data (this is a topic that is frequently discussed on this
list, as you know), otherwise stick with Spearman's coefficient.

HTH,

Stephen Brand

For personalized and professional consultation in statistics and research
design, visit
www.statisticsdoc.com


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
Paul Mcgeoghan
Sent: Thursday, January 25, 2007 10:03 AM
To: [hidden email]
Subject: test-retest reliability and Spearman-Brown split half
coefficient


Hi,

I have been reading a lot of websites relating to reliability and agreement
this week and have a
customer who has carried out the following:

She has 9 people who answered nominal questions:
Yes, No, Don't Know
on 2 seperate occasions.
She is interested in knowing whether the respondents reliably answer the
questions the same over
the 2 time periods.

She also has a number of likert questions also where the same 9 respondents
answered the questions
at time 1 and time 2 and wants to see if they have reliably answered each
question the same over the
2 time periods.

From what I have read, this seems to be referred to as test-retest
reliability and a correlation
between each pair of questions seems to be the way to approach it.

So for the likert scale questions, I could just do a Spearman's correlation
coefficient?
For the nominal questions, what can I use?

I have also seen reference to Spearman-Brown split half coefficient (Analyse
Scale Reliability and
Split-Half).
Can this be used to compare each pair of questions at the 2 time intervals,
and does the data have
to be likert-scale in this instance?

Hope I have explained the problem clearly.

Thanks,
Paul


==================
Paul McGeoghan,
Application support specialist (Statistics and Databases),
University Infrastructure Group (UIG),
Information Services,
Cardiff University.
Tel. 02920 (875035).

[Swank, Paul R]

I would not use the Spearman for the Likert items since the number of
ties is likely to be large. For both you could use the contingency
coefficient or Cramer's V from crosstabs. These are like Phi
coefficients but for greater than 2 by 2 tables. For the Likert items,
you could use a Mantel-Haenszel test for significance if needed since
this will take into account the ordering. For the yes, no, don't know,
the chi square test would be the appropriate test statistic.

Paul R. Swank, Ph.D. Professor
Director of Reseach
Children's Learning Institute
University of Texas Health Science Center-Houston

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Paul Mcgeoghan
Sent: Thursday, January 25, 2007 9:51 AM
To: [hidden email]
Subject: Re: test-retest reliability and Spearman-Brown split half
coefficient

Stephen,

Thanks, I was also thinking along the lines of Spearman rank-order as a
possibility.
She is only interested in comparing the single Likert items.

However, she also has the problem with Yes, No and Don't Know questions
which to me would be considered nominal rather than likert scale so what
does she do for these type of questions?

Paul


==================
Paul McGeoghan,
Application support specialist (Statistics and Databases), University
Infrastructure Group (UIG), Information Services, Cardiff University.
Tel. 02920 (875035).

>>> "Statisticsdoc" <[hidden email]> 25/01/2007 15:45:25 >>>
Paul,

I imagine that you have suggested that more data be collected (a sample
of nine is not enough to accurately estimate test-retest reliability).

Having said that, for the Likert items, you want to compute the
correlation between two time points, not the split-half reliability
(which applies to ratings that are summed into scale scores at the same
point in time).  Is your customer interested in knowing about the
test-retest reliability of single items, or the test-retest reliability
of a score that has been computed from these items.  For single Likert
items, a Spearman rank-order correlation would be appropriate, given the
ordinal nature of the data.  For computing the test-retest reliability
of the scale total, you could use Pearson's correlation if you are
willing to assume that the sums behave like interval level data (this is
a topic that is frequently discussed on this list, as you know),
otherwise stick with Spearman's coefficient.

HTH,

Stephen Brand

For personalized and professional consultation in statistics and
research design, visit www.statisticsdoc.com


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
Paul Mcgeoghan
Sent: Thursday, January 25, 2007 10:03 AM
To: [hidden email]
Subject: test-retest reliability and Spearman-Brown split half
coefficient


Hi,

I have been reading a lot of websites relating to reliability and
agreement this week and have a customer who has carried out the
following:

She has 9 people who answered nominal questions:
Yes, No, Don't Know
on 2 seperate occasions.
She is interested in knowing whether the respondents reliably answer the
questions the same over the 2 time periods.

She also has a number of likert questions also where the same 9
respondents answered the questions at time 1 and time 2 and wants to see
if they have reliably answered each question the same over the
2 time periods.

From what I have read, this seems to be referred to as test-retest
reliability and a correlation between each pair of questions seems to be
the way to approach it.

So for the likert scale questions, I could just do a Spearman's
correlation coefficient?
For the nominal questions, what can I use?

I have also seen reference to Spearman-Brown split half coefficient
(Analyse Scale Reliability and Split-Half).
Can this be used to compare each pair of questions at the 2 time
intervals, and does the data have to be likert-scale in this instance?

Hope I have explained the problem clearly.

Thanks,
Paul


==================
Paul McGeoghan,
Application support specialist (Statistics and Databases), University
Infrastructure Group (UIG), Information Services, Cardiff University.
Tel. 02920 (875035).

[statisticsdoc]

Paul,

I think that you are right to consider Yes No Don't Know as nominal.  Don't
Know is not an intermediate category between Yes and No, the way that
Neutral is an intermediate between Agree and Disagree.

I would suggest that you start with computing the raw percent agreement
between time points for each item.

You want to do more than test whether the ratings are significantly related
(in a larger sample - 9 will not give you much power), you want to assess
the strength of the association between ratings at different time points.
In Crosstabs, if you tabulate Time 1 versus Time 2, and include PHI as a
statistic, you will get Cramer's V (an index of the association between
nominal items).  Cramer's V varies between 0 and 1.  There are other indices
of association for these data, but I think that V is the most readily
interpretable.  V can be viewed as the association between variables as a
proportion of their maximum possible association.

HTH,

Stephen Brand

For personalized and professional consultation in statistics and research
design, visit
www.statisticsdoc.com


-----Original Message-----
From: Paul Mcgeoghan [mailto:[hidden email]]
Sent: Thursday, January 25, 2007 10:51 AM
To: Statisticsdoc; [hidden email]
Subject: RE: test-retest reliability and Spearman-Brown split
halfcoefficient


Stephen,

Thanks, I was also thinking along the lines of Spearman rank-order as a
possibility.
She is only interested in comparing the single Likert items.

However, she also has the problem with Yes, No and Don't Know questions
which to me would be
considered nominal rather than likert scale so what does she do for these
type of questions?

Paul


==================
Paul McGeoghan,
Application support specialist (Statistics and Databases),
University Infrastructure Group (UIG),
Information Services,
Cardiff University.
Tel. 02920 (875035).

>>> "Statisticsdoc" <[hidden email]> 25/01/2007 15:45:25 >>>
Paul,

I imagine that you have suggested that more data be collected (a sample of
nine is not enough to accurately estimate test-retest reliability).

Having said that, for the Likert items, you want to compute the correlation
between two time points, not the split-half reliability (which applies to
ratings that are summed into scale scores at the same point in time).  Is
your customer interested in knowing about the test-retest reliability of
single items, or the test-retest reliability of a score that has been
computed from these items.  For single Likert items, a Spearman rank-order
correlation would be appropriate, given the ordinal nature of the data.  For
computing the test-retest reliability of the scale total, you could use
Pearson's correlation if you are willing to assume that the sums behave like
interval level data (this is a topic that is frequently discussed on this
list, as you know), otherwise stick with Spearman's coefficient.

HTH,

Stephen Brand

For personalized and professional consultation in statistics and research
design, visit
www.statisticsdoc.com


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
Paul Mcgeoghan
Sent: Thursday, January 25, 2007 10:03 AM
To: [hidden email]
Subject: test-retest reliability and Spearman-Brown split half
coefficient


Hi,

I have been reading a lot of websites relating to reliability and agreement
this week and have a
customer who has carried out the following:

She has 9 people who answered nominal questions:
Yes, No, Don't Know
on 2 seperate occasions.
She is interested in knowing whether the respondents reliably answer the
questions the same over
the 2 time periods.

She also has a number of likert questions also where the same 9 respondents
answered the questions
at time 1 and time 2 and wants to see if they have reliably answered each
question the same over the
2 time periods.

From what I have read, this seems to be referred to as test-retest
reliability and a correlation
between each pair of questions seems to be the way to approach it.

So for the likert scale questions, I could just do a Spearman's correlation
coefficient?
For the nominal questions, what can I use?

I have also seen reference to Spearman-Brown split half coefficient (Analyse
Scale Reliability and
Split-Half).
Can this be used to compare each pair of questions at the 2 time intervals,
and does the data have
to be likert-scale in this instance?

Hope I have explained the problem clearly.

Thanks,
Paul


==================
Paul McGeoghan,
Application support specialist (Statistics and Databases),
University Infrastructure Group (UIG),
Information Services,
Cardiff University.
Tel. 02920 (875035).

[statisticsdoc]

Paul,

Good ideas for significance testing to see whether ratings are related over
time (hopefully the sample can be increased).  However, the issue of
test-retest reliability requires a relationship that is not only significant
but also reasonably strong.  For the Likert items, it would be desirable, as
you have said elsewhere, to use a statistic that considers ordinality, hence
the advantage of the Spearman's rank-order correlation over Cramer's V for
these items.  You are quite right to raise the issue of ties, which would be
a huge problem if one were using the quick computational formula for
Spearman's rank-order correlation that assumes no ties.  However, when the
proportion of ties in the data is large, Spearman's rank-order coefficient
can be computed accurately by using the regular Pearson formula on the
ranked data).   NONPAR CORR in SPSS will handle this.

Cramer's V would still be a useful way to assess how stable answers to the
nominal items are over time.

Best,

Stephen Brand

For personalized and professional consultation in statistics and research
design, visit
www.statisticsdoc.com


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
Swank, Paul R
Sent: Thursday, January 25, 2007 11:24 AM
To: [hidden email]
Subject: Re: test-retest reliability and Spearman-Brown split half
coefficient


I would not use the Spearman for the Likert items since the number of
ties is likely to be large. For both you could use the contingency
coefficient or Cramer's V from crosstabs. These are like Phi
coefficients but for greater than 2 by 2 tables. For the Likert items,
you could use a Mantel-Haenszel test for significance if needed since
this will take into account the ordering. For the yes, no, don't know,
the chi square test would be the appropriate test statistic.

Paul R. Swank, Ph.D. Professor
Director of Reseach
Children's Learning Institute
University of Texas Health Science Center-Houston

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Paul Mcgeoghan
Sent: Thursday, January 25, 2007 9:51 AM
To: [hidden email]
Subject: Re: test-retest reliability and Spearman-Brown split half
coefficient

Stephen,

Thanks, I was also thinking along the lines of Spearman rank-order as a
possibility.
She is only interested in comparing the single Likert items.

However, she also has the problem with Yes, No and Don't Know questions
which to me would be considered nominal rather than likert scale so what
does she do for these type of questions?

Paul


==================
Paul McGeoghan,
Application support specialist (Statistics and Databases), University
Infrastructure Group (UIG), Information Services, Cardiff University.
Tel. 02920 (875035).

>>> "Statisticsdoc" <[hidden email]> 25/01/2007 15:45:25 >>>
Paul,

I imagine that you have suggested that more data be collected (a sample
of nine is not enough to accurately estimate test-retest reliability).

Having said that, for the Likert items, you want to compute the
correlation between two time points, not the split-half reliability
(which applies to ratings that are summed into scale scores at the same
point in time).  Is your customer interested in knowing about the
test-retest reliability of single items, or the test-retest reliability
of a score that has been computed from these items.  For single Likert
items, a Spearman rank-order correlation would be appropriate, given the
ordinal nature of the data.  For computing the test-retest reliability
of the scale total, you could use Pearson's correlation if you are
willing to assume that the sums behave like interval level data (this is
a topic that is frequently discussed on this list, as you know),
otherwise stick with Spearman's coefficient.

HTH,

Stephen Brand

For personalized and professional consultation in statistics and
research design, visit www.statisticsdoc.com


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
Paul Mcgeoghan
Sent: Thursday, January 25, 2007 10:03 AM
To: [hidden email]
Subject: test-retest reliability and Spearman-Brown split half
coefficient


Hi,

I have been reading a lot of websites relating to reliability and
agreement this week and have a customer who has carried out the
following:

She has 9 people who answered nominal questions:
Yes, No, Don't Know
on 2 seperate occasions.
She is interested in knowing whether the respondents reliably answer the
questions the same over the 2 time periods.

She also has a number of likert questions also where the same 9
respondents answered the questions at time 1 and time 2 and wants to see
if they have reliably answered each question the same over the
2 time periods.

From what I have read, this seems to be referred to as test-retest
reliability and a correlation between each pair of questions seems to be
the way to approach it.

So for the likert scale questions, I could just do a Spearman's
correlation coefficient?
For the nominal questions, what can I use?

I have also seen reference to Spearman-Brown split half coefficient
(Analyse Scale Reliability and Split-Half).
Can this be used to compare each pair of questions at the 2 time
intervals, and does the data have to be likert-scale in this instance?

Hope I have explained the problem clearly.

Thanks,
Paul


==================
Paul McGeoghan,
Application support specialist (Statistics and Databases), University
Infrastructure Group (UIG), Information Services, Cardiff University.
Tel. 02920 (875035).

[Zdaniuk, Bozena]

Hello, I know it used to be impossible but maybe something has
changed...
Is there a way to set up an automatic periodical back up of the syntax
files as they are being open and worked on? My spss sometimes crashes
and I don't remember saving and sometimes lose a whole day of work. Just
hoping...
Bozena

Bozena Zdaniuk, Ph.D.

University of Pittsburgh

UCSUR, 6th Fl.

121 University Place

Pittsburgh, PA 15260

Ph.: 412-624-5736

Fax: [phone]

email: [hidden email]

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Statisticsdoc
Sent: Thursday, January 25, 2007 3:46 PM
To: [hidden email]
Subject: Re: test-retest reliability and Spearman-Brown split half
coefficient

Paul,

Good ideas for significance testing to see whether ratings are related
over
time (hopefully the sample can be increased).  However, the issue of
test-retest reliability requires a relationship that is not only
significant
but also reasonably strong.  For the Likert items, it would be
desirable, as
you have said elsewhere, to use a statistic that considers ordinality,
hence
the advantage of the Spearman's rank-order correlation over Cramer's V
for
these items.  You are quite right to raise the issue of ties, which
would be
a huge problem if one were using the quick computational formula for
Spearman's rank-order correlation that assumes no ties.  However, when
the
proportion of ties in the data is large, Spearman's rank-order
coefficient
can be computed accurately by using the regular Pearson formula on the
ranked data).   NONPAR CORR in SPSS will handle this.

Cramer's V would still be a useful way to assess how stable answers to
the
nominal items are over time.

Best,

Stephen Brand

For personalized and professional consultation in statistics and
research
design, visit
www.statisticsdoc.com


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
Swank, Paul R
Sent: Thursday, January 25, 2007 11:24 AM
To: [hidden email]
Subject: Re: test-retest reliability and Spearman-Brown split half
coefficient


I would not use the Spearman for the Likert items since the number of
ties is likely to be large. For both you could use the contingency
coefficient or Cramer's V from crosstabs. These are like Phi
coefficients but for greater than 2 by 2 tables. For the Likert items,
you could use a Mantel-Haenszel test for significance if needed since
this will take into account the ordering. For the yes, no, don't know,
the chi square test would be the appropriate test statistic.

Paul R. Swank, Ph.D. Professor
Director of Reseach
Children's Learning Institute
University of Texas Health Science Center-Houston

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Paul Mcgeoghan
Sent: Thursday, January 25, 2007 9:51 AM
To: [hidden email]
Subject: Re: test-retest reliability and Spearman-Brown split half
coefficient

Stephen,

Thanks, I was also thinking along the lines of Spearman rank-order as a
possibility.
She is only interested in comparing the single Likert items.

However, she also has the problem with Yes, No and Don't Know questions
which to me would be considered nominal rather than likert scale so what
does she do for these type of questions?

Paul


==================
Paul McGeoghan,
Application support specialist (Statistics and Databases), University
Infrastructure Group (UIG), Information Services, Cardiff University.
Tel. 02920 (875035).

>>> "Statisticsdoc" <[hidden email]> 25/01/2007 15:45:25 >>>
Paul,

I imagine that you have suggested that more data be collected (a sample
of nine is not enough to accurately estimate test-retest reliability).

Having said that, for the Likert items, you want to compute the
correlation between two time points, not the split-half reliability
(which applies to ratings that are summed into scale scores at the same
point in time).  Is your customer interested in knowing about the
test-retest reliability of single items, or the test-retest reliability
of a score that has been computed from these items.  For single Likert
items, a Spearman rank-order correlation would be appropriate, given the
ordinal nature of the data.  For computing the test-retest reliability
of the scale total, you could use Pearson's correlation if you are
willing to assume that the sums behave like interval level data (this is
a topic that is frequently discussed on this list, as you know),
otherwise stick with Spearman's coefficient.

HTH,

Stephen Brand

For personalized and professional consultation in statistics and
research design, visit www.statisticsdoc.com


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
Paul Mcgeoghan
Sent: Thursday, January 25, 2007 10:03 AM
To: [hidden email]
Subject: test-retest reliability and Spearman-Brown split half
coefficient


Hi,

I have been reading a lot of websites relating to reliability and
agreement this week and have a customer who has carried out the
following:

She has 9 people who answered nominal questions:
Yes, No, Don't Know
on 2 seperate occasions.
She is interested in knowing whether the respondents reliably answer the
questions the same over the 2 time periods.

She also has a number of likert questions also where the same 9
respondents answered the questions at time 1 and time 2 and wants to see
if they have reliably answered each question the same over the
2 time periods.

From what I have read, this seems to be referred to as test-retest
reliability and a correlation between each pair of questions seems to be
the way to approach it.

So for the likert scale questions, I could just do a Spearman's
correlation coefficient?
For the nominal questions, what can I use?

I have also seen reference to Spearman-Brown split half coefficient
(Analyse Scale Reliability and Split-Half).
Can this be used to compare each pair of questions at the 2 time
intervals, and does the data have to be likert-scale in this instance?

Hope I have explained the problem clearly.

Thanks,
Paul


==================
Paul McGeoghan,
Application support specialist (Statistics and Databases), University
Infrastructure Group (UIG), Information Services, Cardiff University.
Tel. 02920 (875035).

[Raynald Levesque]

Hi

I am not aware of a way but the following works for me: I have moved the
"save file" icon next to the "run selection" icon, then each time I want to
click the "run selection: icon, I see the "save file" syntax icon and click
it before running the syntax.

Regards

Raynald Levesque [hidden email]
Website: www.spsstools.net

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Zdaniuk, Bozena
Sent: January 25, 2007 4:29 PM
To: [hidden email]
Subject: back - up for syntax files?

Hello, I know it used to be impossible but maybe something has
changed...
Is there a way to set up an automatic periodical back up of the syntax
files as they are being open and worked on? My spss sometimes crashes
and I don't remember saving and sometimes lose a whole day of work. Just
hoping...
Bozena

Bozena Zdaniuk, Ph.D.

University of Pittsburgh

UCSUR, 6th Fl.

121 University Place

Pittsburgh, PA 15260

Ph.: 412-624-5736

Fax: [phone]

email: [hidden email]

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Statisticsdoc
Sent: Thursday, January 25, 2007 3:46 PM
To: [hidden email]
Subject: Re: test-retest reliability and Spearman-Brown split half
coefficient

Paul,

Good ideas for significance testing to see whether ratings are related
over
time (hopefully the sample can be increased).  However, the issue of
test-retest reliability requires a relationship that is not only
significant
but also reasonably strong.  For the Likert items, it would be
desirable, as
you have said elsewhere, to use a statistic that considers ordinality,
hence
the advantage of the Spearman's rank-order correlation over Cramer's V
for
these items.  You are quite right to raise the issue of ties, which
would be
a huge problem if one were using the quick computational formula for
Spearman's rank-order correlation that assumes no ties.  However, when
the
proportion of ties in the data is large, Spearman's rank-order
coefficient
can be computed accurately by using the regular Pearson formula on the
ranked data).   NONPAR CORR in SPSS will handle this.

Cramer's V would still be a useful way to assess how stable answers to
the
nominal items are over time.

Best,

Stephen Brand

For personalized and professional consultation in statistics and
research
design, visit
www.statisticsdoc.com


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
Swank, Paul R
Sent: Thursday, January 25, 2007 11:24 AM
To: [hidden email]
Subject: Re: test-retest reliability and Spearman-Brown split half
coefficient


I would not use the Spearman for the Likert items since the number of
ties is likely to be large. For both you could use the contingency
coefficient or Cramer's V from crosstabs. These are like Phi
coefficients but for greater than 2 by 2 tables. For the Likert items,
you could use a Mantel-Haenszel test for significance if needed since
this will take into account the ordering. For the yes, no, don't know,
the chi square test would be the appropriate test statistic.

Paul R. Swank, Ph.D. Professor
Director of Reseach
Children's Learning Institute
University of Texas Health Science Center-Houston

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Paul Mcgeoghan
Sent: Thursday, January 25, 2007 9:51 AM
To: [hidden email]
Subject: Re: test-retest reliability and Spearman-Brown split half
coefficient

Stephen,

Thanks, I was also thinking along the lines of Spearman rank-order as a
possibility.
She is only interested in comparing the single Likert items.

However, she also has the problem with Yes, No and Don't Know questions
which to me would be considered nominal rather than likert scale so what
does she do for these type of questions?

Paul


==================
Paul McGeoghan,
Application support specialist (Statistics and Databases), University
Infrastructure Group (UIG), Information Services, Cardiff University.
Tel. 02920 (875035).

>>> "Statisticsdoc" <[hidden email]> 25/01/2007 15:45:25 >>>
Paul,

I imagine that you have suggested that more data be collected (a sample
of nine is not enough to accurately estimate test-retest reliability).

Having said that, for the Likert items, you want to compute the
correlation between two time points, not the split-half reliability
(which applies to ratings that are summed into scale scores at the same
point in time).  Is your customer interested in knowing about the
test-retest reliability of single items, or the test-retest reliability
of a score that has been computed from these items.  For single Likert
items, a Spearman rank-order correlation would be appropriate, given the
ordinal nature of the data.  For computing the test-retest reliability
of the scale total, you could use Pearson's correlation if you are
willing to assume that the sums behave like interval level data (this is
a topic that is frequently discussed on this list, as you know),
otherwise stick with Spearman's coefficient.

HTH,

Stephen Brand

For personalized and professional consultation in statistics and
research design, visit www.statisticsdoc.com


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
Paul Mcgeoghan
Sent: Thursday, January 25, 2007 10:03 AM
To: [hidden email]
Subject: test-retest reliability and Spearman-Brown split half
coefficient


Hi,

I have been reading a lot of websites relating to reliability and
agreement this week and have a customer who has carried out the
following:

She has 9 people who answered nominal questions:
Yes, No, Don't Know
on 2 seperate occasions.
She is interested in knowing whether the respondents reliably answer the
questions the same over the 2 time periods.

She also has a number of likert questions also where the same 9
respondents answered the questions at time 1 and time 2 and wants to see
if they have reliably answered each question the same over the
2 time periods.

From what I have read, this seems to be referred to as test-retest
reliability and a correlation between each pair of questions seems to be
the way to approach it.

So for the likert scale questions, I could just do a Spearman's
correlation coefficient?
For the nominal questions, what can I use?

I have also seen reference to Spearman-Brown split half coefficient
(Analyse Scale Reliability and Split-Half).
Can this be used to compare each pair of questions at the 2 time
intervals, and does the data have to be likert-scale in this instance?

Hope I have explained the problem clearly.

Thanks,
Paul


==================
Paul McGeoghan,
Application support specialist (Statistics and Databases), University
Infrastructure Group (UIG), Information Services, Cardiff University.
Tel. 02920 (875035).

[Peck, Jon]

You could go this one better and create a new run button or replace the standard one so that it automatically does a SaveAs to some standard temporary file every time you do the Run.

-Jon Peck

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Raynald Levesque
Sent: Monday, January 29, 2007 7:16 PM
To: [hidden email]
Subject: Re: [SPSSX-L] back - up for syntax files?

Hi

I am not aware of a way but the following works for me: I have moved the
"save file" icon next to the "run selection" icon, then each time I want to
click the "run selection: icon, I see the "save file" syntax icon and click
it before running the syntax.

Regards

Raynald Levesque [hidden email]
Website: www.spsstools.net

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Zdaniuk, Bozena
Sent: January 25, 2007 4:29 PM
To: [hidden email]
Subject: back - up for syntax files?

Hello, I know it used to be impossible but maybe something has
changed...
Is there a way to set up an automatic periodical back up of the syntax
files as they are being open and worked on? My spss sometimes crashes
and I don't remember saving and sometimes lose a whole day of work. Just
hoping...
Bozena

Bozena Zdaniuk, Ph.D.

University of Pittsburgh

UCSUR, 6th Fl.

121 University Place

Pittsburgh, PA 15260

Ph.: 412-624-5736

Fax: [phone]

email: [hidden email]

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Statisticsdoc
Sent: Thursday, January 25, 2007 3:46 PM
To: [hidden email]
Subject: Re: test-retest reliability and Spearman-Brown split half
coefficient

Paul,

Good ideas for significance testing to see whether ratings are related
over
time (hopefully the sample can be increased).  However, the issue of
test-retest reliability requires a relationship that is not only
significant
but also reasonably strong.  For the Likert items, it would be
desirable, as
you have said elsewhere, to use a statistic that considers ordinality,
hence
the advantage of the Spearman's rank-order correlation over Cramer's V
for
these items.  You are quite right to raise the issue of ties, which
would be
a huge problem if one were using the quick computational formula for
Spearman's rank-order correlation that assumes no ties.  However, when
the
proportion of ties in the data is large, Spearman's rank-order
coefficient
can be computed accurately by using the regular Pearson formula on the
ranked data).   NONPAR CORR in SPSS will handle this.

Cramer's V would still be a useful way to assess how stable answers to
the
nominal items are over time.

Best,

Stephen Brand

For personalized and professional consultation in statistics and
research
design, visit
www.statisticsdoc.com


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
Swank, Paul R
Sent: Thursday, January 25, 2007 11:24 AM
To: [hidden email]
Subject: Re: test-retest reliability and Spearman-Brown split half
coefficient


I would not use the Spearman for the Likert items since the number of
ties is likely to be large. For both you could use the contingency
coefficient or Cramer's V from crosstabs. These are like Phi
coefficients but for greater than 2 by 2 tables. For the Likert items,
you could use a Mantel-Haenszel test for significance if needed since
this will take into account the ordering. For the yes, no, don't know,
the chi square test would be the appropriate test statistic.

Paul R. Swank, Ph.D. Professor
Director of Reseach
Children's Learning Institute
University of Texas Health Science Center-Houston

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Paul Mcgeoghan
Sent: Thursday, January 25, 2007 9:51 AM
To: [hidden email]
Subject: Re: test-retest reliability and Spearman-Brown split half
coefficient

Stephen,

Thanks, I was also thinking along the lines of Spearman rank-order as a
possibility.
She is only interested in comparing the single Likert items.

However, she also has the problem with Yes, No and Don't Know questions
which to me would be considered nominal rather than likert scale so what
does she do for these type of questions?

Paul


==================
Paul McGeoghan,
Application support specialist (Statistics and Databases), University
Infrastructure Group (UIG), Information Services, Cardiff University.
Tel. 02920 (875035).

>>> "Statisticsdoc" <[hidden email]> 25/01/2007 15:45:25 >>>
Paul,

I imagine that you have suggested that more data be collected (a sample
of nine is not enough to accurately estimate test-retest reliability).

Having said that, for the Likert items, you want to compute the
correlation between two time points, not the split-half reliability
(which applies to ratings that are summed into scale scores at the same
point in time).  Is your customer interested in knowing about the
test-retest reliability of single items, or the test-retest reliability
of a score that has been computed from these items.  For single Likert
items, a Spearman rank-order correlation would be appropriate, given the
ordinal nature of the data.  For computing the test-retest reliability
of the scale total, you could use Pearson's correlation if you are
willing to assume that the sums behave like interval level data (this is
a topic that is frequently discussed on this list, as you know),
otherwise stick with Spearman's coefficient.

HTH,

Stephen Brand

For personalized and professional consultation in statistics and
research design, visit www.statisticsdoc.com


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
Paul Mcgeoghan
Sent: Thursday, January 25, 2007 10:03 AM
To: [hidden email]
Subject: test-retest reliability and Spearman-Brown split half
coefficient


Hi,

I have been reading a lot of websites relating to reliability and
agreement this week and have a customer who has carried out the
following:

She has 9 people who answered nominal questions:
Yes, No, Don't Know
on 2 seperate occasions.
She is interested in knowing whether the respondents reliably answer the
questions the same over the 2 time periods.

She also has a number of likert questions also where the same 9
respondents answered the questions at time 1 and time 2 and wants to see
if they have reliably answered each question the same over the
2 time periods.

From what I have read, this seems to be referred to as test-retest
reliability and a correlation between each pair of questions seems to be
the way to approach it.

So for the likert scale questions, I could just do a Spearman's
correlation coefficient?
For the nominal questions, what can I use?

I have also seen reference to Spearman-Brown split half coefficient
(Analyse Scale Reliability and Split-Half).
Can this be used to compare each pair of questions at the 2 time
intervals, and does the data have to be likert-scale in this instance?

Hope I have explained the problem clearly.

Thanks,
Paul


==================
Paul McGeoghan,
Application support specialist (Statistics and Databases), University
Infrastructure Group (UIG), Information Services, Cardiff University.
Tel. 02920 (875035).