Re: Negative Adjusted R Square is a "good" thing?

Posted by Bruce Weaver on Feb 09, 2014; 8:47pm
URL: http://spssx-discussion.165.s1.nabble.com/Negative-Adjusted-R-Square-is-a-good-thing-tp5724399p5724402.html

Andy W replied:  "You only have 26 states and you estimated 37 parameters in the model? I'm surprised SPSS spit out anything! (Did it silently drop some predictors?)"

And cynicalflyer replied to that:  "Should read 43 states."

I assume, then, that the unit of analysis is state.  Is that right?  Here's why I think it must be:

For completely random data, the expected value of the multiple correlation coefficient R is  p / (N-1), where p = the number of predictors and N = the sample size.  You supplied p = 37 and R = .853, so I rearranged the formula to work out that your sample size must be around 43 (i.e., 37 / .853 = 43.38).  

If state is the unit of analysis, your model is grossly over-fitted.  See Mike Babyak's nice article for more info on that topic.  

  http://people.duke.edu/~mababyak/papers/babyakregression.pdf

HTH.

cynicalflyer wrote

The theory: in K-12 education putting more administrative authority in the state board of education is "better" that leaving it to the local boards.
<p>
DVs: I have two ways to measure "better" from 26 states: % kids graduating high school within 4 years and scores by school district on a standardized test. I'll be examining them separately.
<p>
IVs: I have 37 different measures for the types of administrative authority: 3 (state has complete control), 2 (shared/split) and 1 (locality has complete control).
<p>
So I fire up SPSS, plunk in the % kids graduating high school within 4 years by state in 26 states as my DV, plunk in the 37 IVs, use "Enter" as my method (I've been told stepwise is evil, evil, evil) and...
<p>
                                                               

<p align="center"><strong>Model    Summary</strong> </p>
<p>Model</p>	<p align="center">R</p>	<p align="center">R Square</p>	<p align="center">Adjusted    R Square</p>	<p align="center">Std.    Error of the Estimate</p>
<p>1</p>	<p align="right">.853a</p>	<p align="right">.727</p>	<p align="right">-.041</p>	<p align="right">.148600607125323</p>

<p>
This might be "good" if it means the predictors are useless. It is "bad" if I am getting this because my model stinks. How can I determine which?