Analyzing Data from the Entire "Population"

[Brian Dates]

I've been asked to define the design for a study of a large food distribution organization. There are approximately equal numbers of new distribution method sites (grocery style) with traditional distribution method sites, about 33 in each group. Sites self-selected into one method or the other. The total number of sites comprises the population served by the organiztion. The question I've been posed is why should they do a statistical analysis given that the entire population will be included. My thoughts are that if they 1) consider the population they serve is not the entire population of persons who need this type of service in the area (there are other distribution agencies), 2) consider applying the findings to future decisions, and/or 3) consider that the sites within methods will differ in results, they really need to do a statistical analysis.

Thoughts?

Thanks, all. Hope you're staying healthy.

Brian

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Rich Ulrich]

You are generally right - If they want to draw inferences

"statistically," they want statistics.

In particular, though - How serious is that handicap, that

the methods were "self-selected" by the sites?  If that is

serious, then reporting an apparent statistical superiority

of one method could be grossly misleading. 

On the other hand, 33 for each method (if I follow that right)

does not give a lot of power for multiple-group comparisons.

What you could say, at the least, is that the visible difference

between two methods (taking them two at a time) is NOT beyond

what one would expect by chance.  - for equal N comparisons,

the p-value can serve as a surrogate for "effect size", if you

can't show effect size otherwise.

As with other observational studies, you can only try to

"account for" other sources of variance, by covarying or

including other factors in the design.  That might be an important

thing some statistics could help.  That is, if testing does suggest

some differences, do those differences get reduced or wiped

out by controlling for other factors?

For observational studies, any findings (of differences, or less often

of similarities)  can be argued as being due to the self-selection.

--

Rich Ulrich

---

From: SPSSX(r) Discussion <[hidden email]> on behalf of Brian Dates <[hidden email]>
Sent: Friday, May 29, 2020 12:18 PM
To: [hidden email] <[hidden email]>
Subject: Analyzing Data from the Entire "Population"

 

I've been asked to define the design for a study of a large food distribution organization. There are approximately equal numbers of new distribution method sites (grocery style) with traditional distribution method sites, about 33 in each group. Sites self-selected into one method or the other. The total number of sites comprises the population served by the organiztion. The question I've been posed is why should they do a statistical analysis given that the entire population will be included. My thoughts are that if they 1) consider the population they serve is not the entire population of persons who need this type of service in the area (there are other distribution agencies), 2) consider applying the findings to future decisions, and/or 3) consider that the sites within methods will differ in results, they really need to do a statistical analysis.

Thoughts?

Thanks, all. Hope you're staying healthy.

Brian

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD