Treatment Group

[DKUKEC]

I apologize in advance for posting this unrelated question; however, I felt that most of you are not only SPSS experts; you are also extremely knowledgeable in research/evaluation methods.  

In outcome evaluations, how are “treatment groups” defined?  

For example, in a clinical trial if I sign up for treatment protocol A and I decide it’s not for me at what point – if ever would I be considered a participant in the trial and/or as a participant in an outcome study?  Second, what if I signed up for treatment protocol A and I passed away due to complications related to treatment.

In another example, a voluntary court program provides supervision and treatment for first time non-violent felony offenders (before conviction or “pre-adjudication”).  After “screening”, individuals are offered an opportunity to participate in a 12 month program which includes varying degrees of court supervision, drug testing, and counseling.  If the individual agrees to the offer, they are required to sign a “contract” and have the “contract” approved by the court following a hearing.  

Since this is a voluntary program, participants may decline to participate in the program at any time during the 12 month period.  Furthermore, the court may expel participants from the program for non-compliance and/or new arrests.  All unsuccessful exits from the program (except deaths) require the offender to be assigned to the regular court process.  Those participants that exit the program after successfully completing all of the requirements (e.g., graduation) are offered a “nolle prosse” or the opportunity for the participant to have their records removed from the official record.  For the purposes of an outcome evaluation (and for that matter performance indicators); how should the “treatment group" be defined in this example?

I have recommended that our local programs use the “intended to treat” approach for outcome evaluations and performance indicators.  This approach would include all that have signed up for programming.  Some have objected to this approach arguing that it is not an accurate portrayal of their program, since 50% of their participants did not fully benefit from treatment - since they did not successfully complete the program.

All input, comments, and questions welcomed.

Damir  

[Rich Ulrich]

"Randomize, then analyze!"  My own experience is with clinical trials,
but the same principles will apply to court programs, etc.

"Intent to treat" is what is the primary topic being investigated.
 - "Should we *try*  to do this?"  What are the outcomes for a
*randomized* sample?  The primary criterion for the study has to
be based on the randomization, or you are back to the status of not-
having a randomized trial.

Just last week, I read a newspaper report of a test which did report both
ways.  There was, IIRC, 50% success for Intent (well above the control
group), explained especially by the 90% success for those who stayed
with the treatment.  I thought that was a good perspective to work from
for further questions.

The "inability to comply with treatment" is, perhaps, an outcome itself,
as well as being one of the mediators of other outcomes.  For a full report,
it should be described.  Dying due to complications of a treatment is
certainly a (negative) outcome.  I was statistician on a study of oral versus
injectible medication for schizophrenic outpatients, where medication was
guaranteed for the latter group, who where therefore expected to do better. 

The criterion was "rehospitalization, or imminent rehospitalization."  The oral
group, as expected, usually stopped their meds before they relapsed, and they
had more "schizophrenic-type" relapses.  Unexpectedly, the *total* for relapses
in the injection group was just as high -- they relapsed with atypical symptoms,
which looked more like depression.  This creates a picture different from the one
of truculent patients.  Instead, it looks like they stop their own meds when they
find that the meds are creating as many problems as they cure.

For long followups, we usually did have a category of "adminstrative
terminations", subjects whose exit was not considered negative.  (Young patient,
whose family moves out of town....)  But you have to be very careful in assigning
that, and they should be accounted for in the report.  Dying in a commercial plane
crash seems pretty obviously unrelated to any treatment.  Dying from suicide or
from high-risk behaviors is *not* completely unimaginable as a treatment effect...
which was the eventual conclusion of a large scale trial which initially did not
want to count those as medical outcomes in contrast to heart disease.

Hope this helps.

--
Rich Ulrich

> Date: Tue, 18 Jun 2013 06:10:56 -0700

> From: [hidden email]
> Subject: Treatment Group
> To: [hidden email]
>
> I apologize in advance for posting this unrelated question; however, I felt
> that most of you are not only SPSS experts; you are also extremely
> knowledgeable in research/evaluation methods.
>
> In outcome evaluations, how are “treatment groups” defined?
>
> For example, in a clinical trial if I sign up for treatment protocol A and I
> decide it’s not for me at what point – if ever would I be considered a
> participant in the trial and/or as a participant in an outcome study?
> Second, what if I signed up for treatment protocol A and I passed away due
> to complications related to treatment.
>
> In another example, a voluntary court program provides supervision and
> treatment for first time non-violent felony offenders (before conviction or
> “pre-adjudication”). After “screening”, individuals are offered an
> opportunity to participate in a 12 month program which includes varying
> degrees of court supervision, drug testing, and counseling. If the
> individual agrees to the offer, they are required to sign a “contract” and
> have the “contract” approved by the court following a hearing.
>
> Since this is a voluntary program, participants may decline to participate
> in the program at any time during the 12 month period. Furthermore, the
> court may expel participants from the program for non-compliance and/or new
> arrests. All unsuccessful exits from the program (except deaths) require
> the offender to be assigned to the regular court process. Those
> participants that exit the program after successfully completing all of the
> requirements (e.g., graduation) are offered a “nolle prosse” or the
> opportunity for the participant to have their records removed from the
> official record. For the purposes of an outcome evaluation (and for that
> matter performance indicators); how should the “treatment group" be defined
> in this example?
>
> I have recommended that our local programs use the “intended to treat”
> approach for outcome evaluations and performance indicators. This approach
> would include all that have signed up for programming. Some have objected
> to this approach arguing that it is not an accurate portrayal of their
> program, since 50% of their participants did not fully benefit from
> treatment - since they did not successfully complete the program.
>
> All input, comments, and questions welcomed.
>
> Damir
>
>
> ...

[Bruce Weaver]

The other dichotomy that comes to mind here is efficacy versus effectiveness.  A pure efficacy study asks how well can the treatment work under ideal circumstances (e.g., very homogeneous patient population, no problems with non-compliance, etc), whereas a pure effectiveness study asks how well does it work in typical circumstances (where patients are heterogeneous, and non-compliance and other issues rear their ugly heads).  

Last year, I heard a talk by David Streiner (of PDQ Statistics & PDQ Epidemiology fame) on why Phase III Trials don't always tell the truth.  He set up the dichotomy of pure efficacy trial versus pure effectiveness trial, but then argued that in practice, most trials fall somewhere on a continuum between those points.  So when describing a trial, authors ought to provide good clear descriptions of what was done in a way that helps readers locate the trial on that continuum.

HTH.

Rich Ulrich-2 wrote

"Randomize, then analyze!"  My own experience is with clinical trials,
but the same principles will apply to court programs, etc.

"Intent to treat" is what is the primary topic being investigated.
 - "Should we *try*  to do this?"  What are the outcomes for a
*randomized* sample?  The primary criterion for the study has to
be based on the randomization, or you are back to the status of not-
having a randomized trial.

Just last week, I read a newspaper report of a test which did report both
ways.  There was, IIRC, 50% success for Intent (well above the control
group), explained especially by the 90% success for those who stayed
with the treatment.  I thought that was a good perspective to work from
for further questions.

The "inability to comply with treatment" is, perhaps, an outcome itself,
as well as being one of the mediators of other outcomes.  For a full report,
it should be described.  Dying due to complications of a treatment is
certainly a (negative) outcome.  I was statistician on a study of oral versus
injectible medication for schizophrenic outpatients, where medication was
guaranteed for the latter group, who where therefore expected to do better.  

The criterion was "rehospitalization, or imminent rehospitalization."  The oral
group, as expected, usually stopped their meds before they relapsed, and they
had more "schizophrenic-type" relapses.  Unexpectedly, the *total* for relapses
in the injection group was just as high -- they relapsed with atypical symptoms,
which looked more like depression.  This creates a picture different from the one
of truculent patients.  Instead, it looks like they stop their own meds when they
find that the meds are creating as many problems as they cure.

For long followups, we usually did have a category of "adminstrative
terminations", subjects whose exit was not considered negative.  (Young patient,
whose family moves out of town....)  But you have to be very careful in assigning
that, and they should be accounted for in the report.  Dying in a commercial plane
crash seems pretty obviously unrelated to any treatment.  Dying from suicide or
from high-risk behaviors is *not* completely unimaginable as a treatment effect...
which was the eventual conclusion of a large scale trial which initially did not
want to count those as medical outcomes in contrast to heart disease.

Hope this helps.

--
Rich Ulrich

> Date: Tue, 18 Jun 2013 06:10:56 -0700
> From: [hidden email]
> Subject: Treatment Group
> To: [hidden email]
>
> I apologize in advance for posting this unrelated question; however, I felt
> that most of you are not only SPSS experts; you are also extremely
> knowledgeable in research/evaluation methods.
>
> In outcome evaluations, how are “treatment groups” defined?
>
> For example, in a clinical trial if I sign up for treatment protocol A and I
> decide it’s not for me at what point – if ever would I be considered a
> participant in the trial and/or as a participant in an outcome study?
> Second, what if I signed up for treatment protocol A and I passed away due
> to complications related to treatment.
>
> In another example, a voluntary court program provides supervision and
> treatment for first time non-violent felony offenders (before conviction or
> “pre-adjudication”).  After “screening”, individuals are offered an
> opportunity to participate in a 12 month program which includes varying
> degrees of court supervision, drug testing, and counseling.  If the
> individual agrees to the offer, they are required to sign a “contract” and
> have the “contract” approved by the court following a hearing.
>
> Since this is a voluntary program, participants may decline to participate
> in the program at any time during the 12 month period.  Furthermore, the
> court may expel participants from the program for non-compliance and/or new
> arrests.  All unsuccessful exits from the program (except deaths) require
> the offender to be assigned to the regular court process.  Those
> participants that exit the program after successfully completing all of the
> requirements (e.g., graduation) are offered a “nolle prosse” or the
> opportunity for the participant to have their records removed from the
> official record.  For the purposes of an outcome evaluation (and for that
> matter performance indicators); how should the “treatment group" be defined
> in this example?
>
> I have recommended that our local programs use the “intended to treat”
> approach for outcome evaluations and performance indicators.  This approach
> would include all that have signed up for programming.  Some have objected
> to this approach arguing that it is not an accurate portrayal of their
> program, since 50% of their participants did not fully benefit from
> treatment - since they did not successfully complete the program.
>
> All input, comments, and questions welcomed.
>
> Damir
>
>
> ...

[DKUKEC]

Thank you Bruce,

 

This is very helpful… much appreciated.

 

Damir

 

From: Bruce Weaver [via SPSSX Discussion] [mailto:ml-node+[hidden email]]
Sent: Tuesday, June 18, 2013 2:15 PM
To: Damir Kukec
Subject: Re: Treatment Group

 

The other dichotomy that comes to mind here is efficacy versus effectiveness.  A pure efficacy study asks how well can the treatment work under ideal circumstances (e.g., very homogeneous patient population, no problems with non-compliance, etc), whereas a pure effectiveness study asks how well does it work in typical circumstances (where patients are heterogeneous, and non-compliance and other issues rear their ugly heads).  

Last year, I heard a talk by David Streiner (of PDQ Statistics & PDQ Epidemiology fame) on why Phase III Trials don't always tell the truth.  He set up the dichotomy of pure efficacy trial versus pure effectiveness trial, but then argued that in practice, most trials fall somewhere on a continuum between those points.  So when describing a trial, authors ought to provide good clear descriptions of what was done in a way that helps readers locate the trial on that continuum.

HTH.

Rich Ulrich-2 wrote

"Randomize, then analyze!"  My own experience is with clinical trials,
but the same principles will apply to court programs, etc.

"Intent to treat" is what is the primary topic being investigated.
 - "Should we *try*  to do this?"  What are the outcomes for a
*randomized* sample?  The primary criterion for the study has to
be based on the randomization, or you are back to the status of not-
having a randomized trial.

Just last week, I read a newspaper report of a test which did report both
ways.  There was, IIRC, 50% success for Intent (well above the control
group), explained especially by the 90% success for those who stayed
with the treatment.  I thought that was a good perspective to work from
for further questions.

The "inability to comply with treatment" is, perhaps, an outcome itself,
as well as being one of the mediators of other outcomes.  For a full report,
it should be described.  Dying due to complications of a treatment is
certainly a (negative) outcome.  I was statistician on a study of oral versus
injectible medication for schizophrenic outpatients, where medication was
guaranteed for the latter group, who where therefore expected to do better.  

The criterion was "rehospitalization, or imminent rehospitalization."  The oral
group, as expected, usually stopped their meds before they relapsed, and they
had more "schizophrenic-type" relapses.  Unexpectedly, the *total* for relapses
in the injection group was just as high -- they relapsed with atypical symptoms,
which looked more like depression.  This creates a picture different from the one
of truculent patients.  Instead, it looks like they stop their own meds when they
find that the meds are creating as many problems as they cure.

For long followups, we usually did have a category of "adminstrative
terminations", subjects whose exit was not considered negative.  (Young patient,
whose family moves out of town....)  But you have to be very careful in assigning
that, and they should be accounted for in the report.  Dying in a commercial plane
crash seems pretty obviously unrelated to any treatment.  Dying from suicide or
from high-risk behaviors is *not* completely unimaginable as a treatment effect...
which was the eventual conclusion of a large scale trial which initially did not
want to count those as medical outcomes in contrast to heart disease.

Hope this helps.

--
Rich Ulrich

> Date: Tue, 18 Jun 2013 06:10:56 -0700
> From: [hidden email]
> Subject: Treatment Group
> To: [hidden email]
>
> I apologize in advance for posting this unrelated question; however, I felt
> that most of you are not only SPSS experts; you are also extremely
> knowledgeable in research/evaluation methods.
>
> In outcome evaluations, how are “treatment groups” defined?
>
> For example, in a clinical trial if I sign up for treatment protocol A and I
> decide it’s not for me at what point – if ever would I be considered a
> participant in the trial and/or as a participant in an outcome study?
> Second, what if I signed up for treatment protocol A and I passed away due
> to complications related to treatment.
>
> In another example, a voluntary court program provides supervision and
> treatment for first time non-violent felony offenders (before conviction or
> “pre-adjudication”).  After “screening”, individuals are offered an
> opportunity to participate in a 12 month program which includes varying
> degrees of court supervision, drug testing, and counseling.  If the
> individual agrees to the offer, they are required to sign a “contract” and
> have the “contract” approved by the court following a hearing.
>
> Since this is a voluntary program, participants may decline to participate
> in the program at any time during the 12 month period.  Furthermore, the
> court may expel participants from the program for non-compliance and/or new
> arrests.  All unsuccessful exits from the program (except deaths) require
> the offender to be assigned to the regular court process.  Those
> participants that exit the program after successfully completing all of the
> requirements (e.g., graduation) are offered a “nolle prosse” or the
> opportunity for the participant to have their records removed from the
> official record.  For the purposes of an outcome evaluation (and for that
> matter performance indicators); how should the “treatment group" be defined
> in this example?
>
> I have recommended that our local programs use the “intended to treat”
> approach for outcome evaluations and performance indicators.  This approach
> would include all that have signed up for programming.  Some have objected
> to this approach arguing that it is not an accurate portrayal of their
> program, since 50% of their participants did not fully benefit from
> treatment - since they did not successfully complete the program.
>
> All input, comments, and questions welcomed.
>
> Damir
>
>
> ...

--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

 

---

If you reply to this email, your message will be added to the discussion below:

http://spssx-discussion.1045642.n5.nabble.com/Treatment-Group-tp5720786p5720801.html

To unsubscribe from Treatment Group, click here.
NAML

---

Under Florida law, e-mail addresses are public records. If you do not want your e-mail address released in response to a public records request, do not send electronic mail to this entity. Instead, contact this office by phone or in writing.