Re: Fixed Effects Model (LSDV) and Overfitting??

Posted by Rich Ulrich on Jan 03, 2013; 12:01am
URL: http://spssx-discussion.165.s1.nabble.com/Fixed-Effects-Model-LSDV-and-Overfitting-tp5717179p5717202.html

For the effects of changing a ban:  Tiny, tiny Ns, with 4 lifted and one imposed.
This is too few to expect tests to show much, especially since there are so many
other potential differences between states -- like, How big?  How industrialized?

You have a question of what happens across years.  Is there a big, general change?

You have questions of "scaling" -- Are you looking at logs, or at some sort of
"normalized" value that makes states comparable?

I would say that your major *test* comparison can only be the no-ban/ all-ban
contrast, using 20 versus 22 states.  You can plot those means across the 17
years and decide whether there is a time-trend that needs to be accommodated.
(That is most easily done if "linear" takes out most of the change.)  The data you
have available only lets you try to tell a somewhat-solid story about these states
(if there is one story).

If this were mine, what seems to come next is to use a log-scale (vertical) to
show the costs across time for the 5 states with Changes, marking the point
of "intervention".  On that, superimpose the two lines for the averages of the
20 and 22 states.  Is there a further story here?

--
Rich Ulrich

> Date: Tue, 1 Jan 2013 09:54:46 -0800

> From: [hidden email]
> Subject: Re: Fixed Effects Model (LSDV) and Overfitting??
> To: [hidden email]
>
> Rich:
>
> During the time period 1977-1994, a corporate contribution ban was
> implemented in one state and lifted in four states. A total of 20 states had
> a ban in place throughout the time period and the remaining 22 states never
> had a ban in place. There is indeed a lot of fluctuation between the states
> in terms of the dependent variable (pollution abatement costs). The reason
> for these constant variations is what I aim to control for by employing
> state fixed effects. Just as you say, I suspect that it is the heterogeneity
> of these fixed effects is the reason for much of the high R2. Noteworthy
> that the state fixed effects have a combined F-value of 29,3 and is very
> significant.
>
> As I wrote in the other reply, I absolutely believe that there’s a trend in
> the time series but that is what I hope to control for when I in a further
> analysis employ a lagged dependent variable. This gives me a total number of
> observations of 686 instead of 799, since the first year of the time series
> is dropped for each state. I do also control for time fixed effects but they
> seem to be very limited and not statistically significant.
>
> Thanks so much for your help
>
> A further question of mine regards the importance of interpreting the
> significance of my results. Since almost all US states are included in the
> study, can it be said to resemble a study of the total population and if so,
> doesn’t the importance of the p-value diminish since I’m not using a small
> sample from a larger universe of US states?
>
> ...