Re-appeal

[Allan Lundy, PhD]

Dear Listers,
I sent out the below on Jan 20 with the Subject line: "Shotgun wedding of repeated measures and between-groups designs," and, unusually for this usually very chatty group, I seem to have gotten no comments whatsoever.  I'm back to working on this issue, and on the off chance it went astray, I'm repeating the query.  If people remember  getting it before, if one or two could reply (off-list would be fine), at least I'd know it went out.

Thanks!
Allan

Dear Listers,
I've been out of the loop for quite a while, but here's an interesting issue I don't think I've ever seen addressed. 

I have a client who intended to use a wait-list control design -- participants were to wait 6 months or so, then get a 6-month treatment.  For various reasons, the participants ended up being 8 wait-list only, 12 treatment-only, and 4 both.  That is, if we pretend that all participants were on the same timeline, it would look like:
Pre-wait testing ---- Post-wait/pre-treatment testing ----- Post-treatment testing.  But only 4 participants have all 3 testings.
Because of the very small N, I want to squeeze the maximum info out of the data.  What we have is a sort of combined repeated-measures and between-groups design, but note it is IN ADDITION to the usual mixed design, where the treatment and control groups get repeated testings.  If possible, I would like to combine the advantages of the wait-list design (for only 4 cases) with the independent-groups design.  I suspect this is impossible or would require mathematics well beyond me -- that is, no standard analytic program could deal with it.  But just in case anybody knows of someone who, for example, wrote a statistics dissertation on this issue, I'd like to hear about it.

Secondly, I intend to argue that in effect, the 4 participants who were in both the wait-list and treatment conditions can and should be treated as 8 cases.  That is, usually there would be concerns about counting the same person as two subjects, but I contend that my solution is actually conservative.  You want to avoid counting a person twice in the same treatment group because that artificially reduces within-group variance, hence inflates relative between-group differences.  In fact, husband-and-wife pairs should not be assigned to the same condition for that reason, unless you use a nested-group design (basically using the couple as the unit of analysis rather than the individual). In this case, the "matched" cases are assigned to "different" conditions -- this would be similar to making sure that identical twins went into different conditions.

Does anyone see any glaring holes in my logic?  This will be peachy for the client, but when the work is submitted for publication, I expect some raised eyebrows.  Does anyone know of a precedent or better yet, an explicit argument for this in a previous publication?  Or if not, would any of the (relatively) big names on this list be willing to be used as a reference?

Thanks!
Allan

Allan Lundy, PhD
Research Consulting
[hidden email]

Business & Cell (any time): 215-820-8100
Home (8am-10pm, 7 days/week): 215-885-5313
Address:  108 Cliff Terrace, Wyncote, PA 19095
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Steve Simon, P.Mean Consulting]

Allan Lundy, PhD wrote:

>
> I have a client who intended to use a wait-list control design --
> participants were to wait 6 months or so, then get a 6-month treatment.
> For various reasons, the participants ended up being 8 wait-list only,
> 12 treatment-only, and 4 both.  That is, if we pretend that all
> participants were on the same timeline, it would look like:
> Pre-wait testing ---- Post-wait/pre-treatment testing -----
> Post-treatment testing.  But only 4 participants have all 3 testings.
> Because of the very small N, I want to squeeze the maximum info out of
> the data.  What we have is a sort of combined repeated-measures and
> between-groups design, but note it is IN ADDITION to the usual mixed
> design, where the treatment and control groups get repeated testings.
> If possible, I would like to combine the advantages of the wait-list
> design (for only 4 cases) with the independent-groups design.  I suspect
> this is impossible or would require mathematics well beyond me -- that
> is, no standard analytic program could deal with it.  But just in case
> anybody knows of someone who, for example, wrote a statistics
> dissertation on this issue, I'd like to hear about it.

Mixed linear regression models are pretty forgiving of imbalance and
missing values. Have you considered these?

> Secondly, I intend to argue that in effect, the 4 participants who were
> in both the wait-list and treatment conditions can and should be treated
> as 8 cases.  That is, usually there would be concerns about counting the
> same person as two subjects, but I contend that my solution is actually
> conservative.  You want to avoid counting a person twice in the same
> treatment group because that artificially reduces within-group variance,
> hence inflates relative between-group differences.  In fact,
> husband-and-wife pairs should not be assigned to the same condition for
> that reason, unless you use a nested-group design (basically using the
> couple as the unit of analysis rather than the individual). In this
> case, the "matched" cases are assigned to "different" conditions -- this
> would be similar to making sure that identical twins went into different
> conditions.
>
> Does anyone see any glaring holes in my logic?  This will be peachy for
> the client, but when the work is submitted for publication, I expect
> some raised eyebrows.  Does anyone know of a precedent or better yet, an
> explicit argument for this in a previous publication?  Or if not, would
> any of the (relatively) big names on this list be willing to be used as
> a reference?

Why in the world would you want to artifically double the number of
subjects in a group? Is it to get balance back in your data? There's no
way that this will fly. It may be a perfectly reasonable thing (I doubt
it, but I'm trying to be generous here). The problem is that there is no
precedent for it. Almost every referee will question this, if only to
avoid getting in trouble themselves. Many referees will see this
suggestion as either naive or an attempt to produce fraudulent results,
and the time you would spend trying to convince them otherwise will be
time wasted. Do yourself a favor and avoid this battle as you will
certainly lose it and it might drag down the paper with it.

Steve Simon, Standard Disclaimer
Sign up for The Monthly Mean, the newsletter that
dares to call itself "average" at www.pmean.com/news

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

Well said, Steve.

Steve Simon, P.Mean Consulting wrote

Allan Lundy, PhD wrote:
>
> I have a client who intended to use a wait-list control design --
> participants were to wait 6 months or so, then get a 6-month treatment.
> For various reasons, the participants ended up being 8 wait-list only,
> 12 treatment-only, and 4 both.  That is, if we pretend that all
> participants were on the same timeline, it would look like:
> Pre-wait testing ---- Post-wait/pre-treatment testing -----
> Post-treatment testing.  But only 4 participants have all 3 testings.
> Because of the very small N, I want to squeeze the maximum info out of
> the data.  What we have is a sort of combined repeated-measures and
> between-groups design, but note it is IN ADDITION to the usual mixed
> design, where the treatment and control groups get repeated testings.
> If possible, I would like to combine the advantages of the wait-list
> design (for only 4 cases) with the independent-groups design.  I suspect
> this is impossible or would require mathematics well beyond me -- that
> is, no standard analytic program could deal with it.  But just in case
> anybody knows of someone who, for example, wrote a statistics
> dissertation on this issue, I'd like to hear about it.

Mixed linear regression models are pretty forgiving of imbalance and
missing values. Have you considered these?

> Secondly, I intend to argue that in effect, the 4 participants who were
> in both the wait-list and treatment conditions can and should be treated
> as 8 cases.  That is, usually there would be concerns about counting the
> same person as two subjects, but I contend that my solution is actually
> conservative.  You want to avoid counting a person twice in the same
> treatment group because that artificially reduces within-group variance,
> hence inflates relative between-group differences.  In fact,
> husband-and-wife pairs should not be assigned to the same condition for
> that reason, unless you use a nested-group design (basically using the
> couple as the unit of analysis rather than the individual). In this
> case, the "matched" cases are assigned to "different" conditions -- this
> would be similar to making sure that identical twins went into different
> conditions.
>
> Does anyone see any glaring holes in my logic?  This will be peachy for
> the client, but when the work is submitted for publication, I expect
> some raised eyebrows.  Does anyone know of a precedent or better yet, an
> explicit argument for this in a previous publication?  Or if not, would
> any of the (relatively) big names on this list be willing to be used as
> a reference?

Why in the world would you want to artifically double the number of
subjects in a group? Is it to get balance back in your data? There's no
way that this will fly. It may be a perfectly reasonable thing (I doubt
it, but I'm trying to be generous here). The problem is that there is no
precedent for it. Almost every referee will question this, if only to
avoid getting in trouble themselves. Many referees will see this
suggestion as either naive or an attempt to produce fraudulent results,
and the time you would spend trying to convince them otherwise will be
time wasted. Do yourself a favor and avoid this battle as you will
certainly lose it and it might drag down the paper with it.

Steve Simon, Standard Disclaimer
Sign up for The Monthly Mean, the newsletter that
dares to call itself "average" at www.pmean.com/news

[statisticsdoc]

Allan,

 

I would underscore what has been said about NOT treating the 4 cases as though they were 4 wait list and 4 treatment cases.  This violates assumptions about independence very seriously.  Nonetheless, one thing you might learn from these data concerns thus question - why is it that only 4 cases made it all the way through the wait list and the treatment.  Does this suggest some barriers to collecting all of the data that was originally sought?

 

I would suggest using just the wait list data from those four cases (as they were presumably assigned to the waiting list control group), and comparing 12 control and 12 treament cases.  Look for time by group interaction effects on the dependent variables (i.e., do the pre-post trends differ between control and treatment groups).  Above all, see if your client can collect more data.  Run some power analyses.  See how strong the pre-post correlations are.  You might be surprised to see how powerful a repeated measures design becomes with the addition of further cases. 

 

Better to focus on a defensible design than to overstretch.

 

HTH,

 

Steve Brand

 

www.StatisticsDoc.com

 -----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of Allan Lundy, PhD
Sent: Saturday, March 20, 2010 11:46 AM
To: [hidden email]
Subject: Re-appeal

Dear Listers,
I sent out the below on Jan 20 with the Subject line: "Shotgun wedding of repeated measures and between-groups designs," and, unusually for this usually very chatty group, I seem to have gotten no comments whatsoever.  I'm back to working on this issue, and on the off chance it went astray, I'm repeating the query.  If people remember  getting it before, if one or two could reply (off-list would be fine), at least I'd know it went out.

Thanks!
Allan

Dear Listers,
I've been out of the loop for quite a while, but here's an interesting issue I don't think I've ever seen addressed. 

I have a client who intended to use a wait-list control design -- participants were to wait 6 months or so, then get a 6-month treatment.  For various reasons, the participants ended up being 8 wait-list only, 12 treatment-only, and 4 both.  That is, if we pretend that all participants were on the same timeline, it would look like:
Pre-wait testing ---- Post-wait/pre-treatment testing ----- Post-treatment testing.  But only 4 participants have all 3 testings.
Because of the very small N, I want to squeeze the maximum info out of the data.  What we have is a sort of combined repeated-measures and between-groups design, but note it is IN ADDITION to the usual mixed design, where the treatment and control groups get repeated testings.  If possible, I would like to combine the advantages of the wait-list design (for only 4 cases) with the independent-groups design.  I suspect this is impossible or would require mathematics well beyond me -- that is, no standard analytic program could deal with it.  But just in case anybody knows of someone who, for example, wrote a statistics dissertation on this issue, I'd like to hear about it.

Secondly, I intend to argue that in effect, the 4 participants who were in both the wait-list and treatment conditions can and should be treated as 8 cases.  That is, usually there would be concerns about counting the same person as two subjects, but I contend that my solution is actually conservative.  You want to avoid counting a person twice in the same treatment group because that artificially reduces within-group variance, hence inflates relative between-group differences.  In fact, husband-and-wife pairs should not be assigned to the same condition for that reason, unless you use a nested-group design (basically using the couple as the unit of analysis rather than the individual). In this case, the "matched" cases are assigned to "different" conditions -- this would be similar to making sure that identical twins went into different conditions.

Does anyone see any glaring holes in my logic?  This will be peachy for the client, but when the work is submitted for publication, I expect some raised eyebrows.  Does anyone know of a precedent or better yet, an explicit argument for this in a previous publication?  Or if not, would any of the (relatively) big names on this list be willing to be used as a reference?

Thanks!
Allan

Allan Lundy, PhD
Research Consulting
[hidden email]

Business & Cell (any time): 215-820-8100
Home (8am-10pm, 7 days/week): 215-885-5313
Address:  108 Cliff Terrace, Wyncote, PA 19095
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Sandra Sigmon]

Has anyone had any problem in printing output from spss version 17?

Thanks,


Sandra T. Sigmon, Ph.D.
Professor, Department of Psychology
Senior Scientist, Maine Institute of Human Genetics & Health
376 Little Hall
University of Maine, Orono, ME  04469
phone: [phone]
fax: [phone]

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Kylie]

Hi Sandra,

I know that SPSS doesn't cooperate with Brother USB printers. In my case
(SPSS 17.0.3), if I try to print to my such printer (DCP-130C) SPSS freezes
altogether. SPSS Support couldn't offer any work around and it's logged in
the support knowledgebase. Network printers have never given me a problem
though.

Cheers,
Kylie.


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Sandra Sigmon
Sent: Tuesday, 23 March 2010 10:15 AM
To: [hidden email]
Subject: problems printing from spss 17

Has anyone had any problem in printing output from spss version 17?

Thanks,


Sandra T. Sigmon, Ph.D.
Professor, Department of Psychology
Senior Scientist, Maine Institute of Human Genetics & Health
376 Little Hall
University of Maine, Orono, ME  04469
phone: [phone]
fax: [phone]

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD