Adjusting significance level

[Bianco, Joseph]

Hi All:

 

I have a rather simplistic question for the listserv: under what circumstances (or following what protocol) does one expand the cutoff for statistical significance (i.e., adopting  a p of .1 rather than .05)?

I understand that there isn’t one set way to do this, and that there are multiple factors to consider depending on the study design, etc. However, if there are any good resources or rules of thumb that anyone could recommend, I’d greatly appreciate it. (If it helps, I’m working with self-reported data derived from a survey pre and post intervention and have a sample size of 98 complete pairs. The sample is self-selected and there are no comparison groups. Almost all variables are categorical or ordinal.).

 

Thanks in advance,

 

Joe

 

[mpirritano]

To my mind the only time this is acceptable is when you are doing exploratory data analysis for a study that you want to conduct. This casts the net a little wider for potential research questions. The actual analysis in the study would use p < .05.

 

Matthew Pirritano, Ph.D.

Research Analyst IV

Medical Services Initiative (MSI)

Orange County Health Care Agency

[phone]

---

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Bianco, Joseph
Sent: Monday, August 03, 2009 1:18 PM
To: [hidden email]
Subject: Adjusting significance level

 

Hi All:

 

I have a rather simplistic question for the listserv: under what circumstances (or following what protocol) does one expand the cutoff for statistical significance (i.e., adopting  a p of .1 rather than .05)?

I understand that there isn’t one set way to do this, and that there are multiple factors to consider depending on the study design, etc. However, if there are any good resources or rules of thumb that anyone could recommend, I’d greatly appreciate it. (If it helps, I’m working with self-reported data derived from a survey pre and post intervention and have a sample size of 98 complete pairs. The sample is self-selected and there are no comparison groups. Almost all variables are categorical or ordinal.).

 

Thanks in advance,

 

Joe