Advice on analyzing data

[Allen Frommelt]

I am looking for some advice on the best approach for analyzing our data in SPSS.  We are currently running version 19.  We have a group of 2,405 people who received an intervention and 5,702 people who did not receive the intervention (the groups were not randomly selected so we know we have bias in that area).  For each individual, we have up to 27 months of data from the period prior to the start of the intervention and 12 months of data post-intervention.   For most of the outcomes of interest, the data is highly skewed and not normally distributed.

 

Thanks,

 

Allen

This email and all attachments are confidential and intended solely
for the use of the individual or entity to which they are addressed.
If you have received this email in error please notify the sender
by replying to this message. If you are not the intended recipient,
please delete this message and all attachments immediately.  Do not
copy, disclose, use or act upon the information contained. Please
note that any views or opinions presented in this email are solely
those of the author and do not necessarily represent those of the
company. Finally, the recipient should check this email and any
attachments for the presence of viruses. While every attempt is made
to verify that the contents are safe, the company accepts no liability
for any damage caused by any virus transmitted by this email.

[Maguin, Eugene]

Allen,

 

I understand you are coming from a commercial entity and, therefore, may not be able to say very much, if anything, about the study design and measurement as it relates to your question. However, would it be possible to name and describe your key DV and describe the measurement frequency?

 

IF you haven’t already, one thing that you should begin investigating are propensity score methods. Depending on things, you may be able to create a statistically equivalent comparison group.

 

Gene Maguin

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Allen Frommelt
Sent: Thursday, May 10, 2012 11:28 AM
To: [hidden email]
Subject: Advice on analyzing data

 

I am looking for some advice on the best approach for analyzing our data in SPSS.  We are currently running version 19.  We have a group of 2,405 people who received an intervention and 5,702 people who did not receive the intervention (the groups were not randomly selected so we know we have bias in that area).  For each individual, we have up to 27 months of data from the period prior to the start of the intervention and 12 months of data post-intervention.   For most of the outcomes of interest, the data is highly skewed and not normally distributed.

 

Thanks,

 

Allen

This email and all attachments are confidential and intended solely
for the use of the individual or entity to which they are addressed.
If you have received this email in error please notify the sender
by replying to this message. If you are not the intended recipient,
please delete this message and all attachments immediately.  Do not
copy, disclose, use or act upon the information contained. Please
note that any views or opinions presented in this email are solely
those of the author and do not necessarily represent those of the
company. Finally, the recipient should check this email and any
attachments for the presence of viruses. While every attempt is made
to verify that the contents are safe, the company accepts no liability
for any damage caused by any virus transmitted by this email.

[Poes, Matthew Joseph]

I’ll add to Gene’s comment by saying that if you use propensity score methods, you can retain the entire comparison population, and weight the people/units to create a balanced design, allowing you to analyze the data using methods effected by unbalanced designs.  However you must meet and test the assumptions for propensity score matching, if you don’t, or it’s in question, don’t use it, it has been shown to create increased bias in this situation.

 

However, if this is a design which can make use of MLM/HLM analysis, they will not be biased by the unbalanced design, and I actually have a paper coming out discussing the advantages to power of having a greater number of control units, without weighted matching.  However, as it’s a working paper, I can’t/won’t say much more about it.

 

If this is a repeated measures design, meaning you have data collected on the same people repeatedly, and can match the data at each time point to an individual such that time points are nested within people, I would suggest this route.  It will give the greatest strength in your analysis.  Assuming the repeated measures exist as your outcome, as well as predictors, you can analyze the time variant and time invariant predictors.

 

If, when it comes right down to this, you have a design in which you want to see if treatment scored better than control over time, and the covariates in the model are simply things like demographics, then they can be treated as fixed effects at the person level.  The outcome/s would simply be the DV’s in the model, and you would use an indicator variable for the treatment/control portion.  Covariates then serve to equalize the groups at baseline.

 

Matthew J Poes

Research Data Specialist

Center for Prevention Research and Development

University of Illinois

510 Devonshire Dr.

Champaign, IL 61820

Phone: [phone]

email: [hidden email]

 

 

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Allen Frommelt
Sent: Thursday, May 10, 2012 10:28 AM
To: [hidden email]
Subject: Advice on analyzing data

 

I am looking for some advice on the best approach for analyzing our data in SPSS.  We are currently running version 19.  We have a group of 2,405 people who received an intervention and 5,702 people who did not receive the intervention (the groups were not randomly selected so we know we have bias in that area).  For each individual, we have up to 27 months of data from the period prior to the start of the intervention and 12 months of data post-intervention.   For most of the outcomes of interest, the data is highly skewed and not normally distributed.

 

Thanks,

 

Allen

This email and all attachments are confidential and intended solely
for the use of the individual or entity to which they are addressed.
If you have received this email in error please notify the sender
by replying to this message. If you are not the intended recipient,
please delete this message and all attachments immediately.  Do not
copy, disclose, use or act upon the information contained. Please
note that any views or opinions presented in this email are solely
those of the author and do not necessarily represent those of the
company. Finally, the recipient should check this email and any
attachments for the presence of viruses. While every attempt is made
to verify that the contents are safe, the company accepts no liability
for any damage caused by any virus transmitted by this email.

[David Marso]

I would add:  Why did you bother to collect the data in the first place and what do you expect it to tell you?  Perhaps check your budget and see if it has allowances for statistical consulting.  This group is GREAT for addressing specific targeted questions but a shotgun 'what the f* do I do with my data' is probably best directed to a dedicated knowledgeable resource.  Data analysis is highly nuanced and to provide *ANY* specific advise without detailed knowledge of the design/variables/hypothesized relationships/a priori model/intentions is at best irresponsible IMNSHO.  If this somehow violates some nebulous NDA then you are likely SOL!
---

Maguin, Eugene wrote

Allen,

I understand you are coming from a commercial entity and, therefore, may not be able to say very much, if anything, about the study design and measurement as it relates to your question. However, would it be possible to name and describe your key DV and describe the measurement frequency?

IF you haven't already, one thing that you should begin investigating are propensity score methods. Depending on things, you may be able to create a statistically equivalent comparison group.

Gene Maguin

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Allen Frommelt
Sent: Thursday, May 10, 2012 11:28 AM
To: [hidden email]
Subject: Advice on analyzing data

I am looking for some advice on the best approach for analyzing our data in SPSS.  We are currently running version 19.  We have a group of 2,405 people who received an intervention and 5,702 people who did not receive the intervention (the groups were not randomly selected so we know we have bias in that area).  For each individual, we have up to 27 months of data from the period prior to the start of the intervention and 12 months of data post-intervention.   For most of the outcomes of interest, the data is highly skewed and not normally distributed.

Thanks,

Allen

This email and all attachments are confidential and intended solely
for the use of the individual or entity to which they are addressed.
If you have received this email in error please notify the sender
by replying to this message. If you are not the intended recipient,
please delete this message and all attachments immediately.  Do not
copy, disclose, use or act upon the information contained. Please
note that any views or opinions presented in this email are solely
those of the author and do not necessarily represent those of the
company. Finally, the recipient should check this email and any
attachments for the presence of viruses. While every attempt is made
to verify that the contents are safe, the company accepts no liability
for any damage caused by any virus transmitted by this email.

[Rich Ulrich]

Sigh.  If you did not have experts in statistics and design
involved from the start, you probably overlooked vital questions
that could have been asked for control, or you missed good
chances for getting better data.  So you might have a hard
time in recruiting good, experienced, competent experts at
this late stage.

But you need to try.  Someone will need to lay hands on the
data to see what the chances are.  Drawing inferences is always
difficult for observational data.  Your non-random control puts
your data pretty much into that class.  And then there are
measurement issues that might have been avoided if the study
were "designed" in the first place.

For instance, you state, about the outcomes, that "the data
is highly skewed...."   I suspect that this is a naive comment
about some set(s) of numbers.  But any set of numbers
might be (perhaps) easily transformed to a different scale. 
Is "skewness" inherent in the set of outcomes, as latent variables,
or is that an artifact of measurement?  - If you really do have
relatively-few extreme outcomes (successes?) versus everything
else, that might reduce the power of analyses and reduce your
possibilities for doing internal replications.

--
Rich Ulrich

---

Date: Thu, 10 May 2012 15:27:53 +0000
From: [hidden email]
Subject: Advice on analyzing data
To: [hidden email]

I am looking for some advice on the best approach for analyzing our data in SPSS.  We are currently running version 19.  We have a group of 2,405 people who received an intervention and 5,702 people who did not receive the intervention (the groups were not randomly selected so we know we have bias in that area).  For each individual, we have up to 27 months of data from the period prior to the start of the intervention and 12 months of data post-intervention.   For most of the outcomes of interest, the data is highly skewed and not normally distributed.

 

Thanks,

 ...

[Kornbrot, Diana]

Another sigh ....
Your company should NOT be embarking on such a study without expert statistical advice BEFORE data anlysis
I presume this is post hoc, in that you have the data to know about who did or did not experience the intervention
BUT, do you have consent from close to 10k people?
Does study have ethical approval?

Think anyone should be very wary of claims by nurturehealth as to the effect of the intervention
Best
Diana



On 10/05/2012 16:27, "Allen Frommelt" <rfrommelt@...> wrote:

I am looking for some advice on the best approach for analyzing our data in SPSS.  We are currently running version 19.  We have a group of 2,405 people who received an intervention and 5,702 people who did not receive the intervention (the groups were not randomly selected so we know we have bias in that area).  For each individual, we have up to 27 months of data from the period prior to the start of the intervention and 12 months of data post-intervention.   For most of the outcomes of interest, the data is highly skewed and not normally distributed.
 
Thanks,
 
Allen

This email and all attachments are confidential and intended solely
for the use of the individual or entity to which they are addressed.
If you have received this email in error please notify the sender
by replying to this message. If you are not the intended recipient,
please delete this message and all attachments immediately.  Do not
copy, disclose, use or act upon the information contained. Please
note that any views or opinions presented in this email are solely
those of the author and do not necessarily represent those of the
company. Finally, the recipient should check this email and any
attachments for the presence of viruses. While every attempt is made
to verify that the contents are safe, the company accepts no liability
for any damage caused by any virus transmitted by this email.

---

Emeritus Professor Diana Kornbrot
email:  d.e.kornbrot@...    
web:    http://dianakornbrot.wordpress.com/
Work
School of Psychology
 University of Hertfordshire
 College Lane, Hatfield, Hertfordshire AL10 9AB, UK
 voice:   [phone]
   fax:     [phone]
Home
 19 Elmhurst Avenue
 London N2 0LT, UK
    voice:   [phone]
    mobile: [phone]

[David Marso]

Maybe these folks can help ;-)
--
http://nurturhealth.com/value-measurement.php