non stationary data

[drfg2008]

A methodological question:

I would like to compare the number of registrations of two periods of a year (first half and second half) and check if the number of applications of the second period increased significantly caused by a 'treatment' (increased tax in the second period) . Unfortunately, it is non stationary data (time series with trend, possibly with cycles). The number of registrations increase anyway. What procedure would be useful for comparing two time periods with non-stationary data?

My first thought was to make the data stationary by computing the difference between each month (or even each day). Then use the t-Test (or U-Test) for the first 6 months and second 6 months (2 groups).

[David Marso]

First thoughts:
How have you deduced that the series is non stationary?
You might wish to review the prerequisites of TSA.
Trend and cycles are not going to be remotely evident without about 3 years of data.
First thoughts are usually bollocks!
--

drfg2008 wrote

A methodological question:

I would like to compare the number of registrations of two periods of a year (first half and second half) and check if the number of applications of the second period increased significantly caused by a 'treatment' (increased tax in the second period) . Unfortunately, it is non stationary data (time series with trend, possibly with cycles). The number of registrations increase anyway. What procedure would be useful for comparing two time periods with non-stationary data?

My first thought was to make the data stationary by computing the difference between each month (or even each day). Then use the t-Test (or U-Test) for the first 6 months and second 6 months (2 groups).

[Li Cao]

Thanks for your email. I am off sick today, Jan 14th, Monday.

 

For ARF, please contact ARF Sample [hidden email]

For SBA, Please contact Springboard US Sample [hidden email]

For SBUK, please contact Springboard UK Sample [hidden email]

 

Thanks!

Li Cao

Sample Analyst | National Panels | Vision Critical 

direct   778.373.0459 

 

[Andy W]

In large agreement with David, IMO you lack enough data to identify the appropriate ARIMA model to begin with. If you seriously pursue this you will either want to get more historical data and wait a few more months (an old advisor of mine gave a rule of thumb of a dozen periods post-intervention to identify any effects) or utilize a smaller time period for the unit of analysis (e.g. weekly) - which tends to come with its own problems.

Differencing tends to be over-used, and can produce some flippantly inaccurate results (both in univariate and multiple regression) - google "over-differencing" for some examples, or see this paper (Bertrand et al., 2002). Robert Nau's webpage is a useful quick tutorial for time series analysis (http://people.duke.edu/~rnau/411arim2.htm), but it is a subject that deserves some serious study with a textbook IMO.

[drfg2008]

Thanks a lot!

The time series is not stationary, has positive trend a periodicity and is autocorrelated (r=0,87).

[David Marso]

With one year of data you have no way to ascertain periodicity.
What do you get as residuals from an AR1?
Where angels fear to tread...
--

drfg2008 wrote

Thanks a lot!

The time series is not stationary, has positive trend a periodicity and is autocorrelated (r=0,87).

[David Marso]

EXSMOOTH

On Tue, Jan 15, 2013 at 9:36 AM, drfg2008 [via SPSSX Discussion]
<[hidden email]> wrote:

> Thanks a lot!
>
> The time series is not stationary, has positive trend a periodicity and is
> autocorrelated (r=0,87).
>
> Does SPSS offer a smooting function, i.e. a moving average?
> Dr. Frank Gaeth
> FU-Berlin
>
>
> ________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://spssx-discussion.1045642.n5.nabble.com/non-stationary-data-tp5717354p5717390.html
> To unsubscribe from non stationary data, click here.
> NAML

[drfg2008]

I have three years of data. So using the expert modeler seems to be an alternative at the first glance (produces a tremendous fit). However, the aim is to find out what independent variables are significant.

Independent Variables:
day (time)
5 Dummy Variables.

Dependent Variable:
newRegistrations


I try to find out, if the 5 dummy variables have any significant influence on the dependent variable.

[David Marso]

So what does the GIGOnosticator tell you?
Review the principles of basic Interrupted Time Series Analysis and report back.
--
A. Model the basic series and reduce to white noise.
B. Regress the white noise on the predictor space.
---

drfg2008 wrote

I have three years of data. So using the expert modeler seems to be an alternative at the first glance (produces a tremendous fit). However, the aim is to find out what independent variables are significant.

Independent Variables:
day (time)
5 Dummy Variables.

Dependent Variable:
newRegistrations


I try to find out, if the 5 dummy variables have any significant influence on the dependent variable.

[drfg2008]

I've never worked with time series problems in this context and tried to follow IBM SPSS Forecasting 20 manual.

Would it make sense to first model the 3 year time series with Expert Modeler (as the only independent variable: time), then save the predicted values, compute the differences between actual values and predicted values, treat this as residuals "RES_" (noise residuals obviously is not the same) and then run a regression on this "RES_" value (as dependent variable)? The idea behind it: The "RES_" value should be free of systematic trend, periodicity, so they can be used in a linear regression. "RES_" is, at least in my data almost N~ distributed with µ~0.

(BTW: The question is if certain events during the time have had in influence on the time series)

[Jon K Peck]

Why not just include the predictors in the original analysis?


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
new phone: [phone]




From:        drfg2008 <[hidden email]>
To:        [hidden email],
Date:        01/16/2013 02:45 AM
Subject:        Re: [SPSSX-L] non stationary data
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

I've never worked with time series problems in this context and tried to
follow IBM SPSS Forecasting 20 manual.

Would it make sense to first model the 3 year time series with Expert
Modeler (as the only independent variable: time), then save the predicted
values, compute the differences between actual values and predicted values,
treat this as residuals "RES_" (noise residuals obviously is not the same)
and then run a regression on this "RES_" value (as dependent variable)? The
idea behind it: The "RES_" value should be free of systematic trend,
periodicity, so they can be used in a linear regression. "RES_" is, at least
in my data almost N~ distributed with µ~0.

(BTW: The question is if certain events during the time have had in
influence on the time series)



-----
Dr. Frank Gaeth
FU-Berlin

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/non-stationary-data-tp5717354p5717417.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[David Marso]

Bravo!  You have the basic ideas!
I posted this link a couple weeks ago which puts much of the Identification issue into perspective:
http://people.duke.edu/~rnau/seasarim.htm
To scratch the surface:
Here is one of the books used in a course I took LONG ago:
http://books.google.com/books/about/Interrupted_Time_Series_Analysis.html?id=oAIuJ2JQIngC
http://books.google.com/books?id=qTStAbJg2mkC&source=gbs_similarbooks
Potentially useful: http://www.gvglass.info/papers/tsx.pdf
http://www.ambpeds.org/specialinterestgroups/QIfolder/2012/Penfold%20-%20101%20ITS_101_PAS_2012.pdf

You will definitely want to do some background reading if you have not worked with this stuff previously.

drfg2008 wrote

I've never worked with time series problems in this context and tried to follow IBM SPSS Forecasting 20 manual.

Would it make sense to first model the 3 year time series with Expert Modeler (as the only independent variable: time), then save the predicted values, compute the differences between actual values and predicted values, treat this as residuals "RES_" (noise residuals obviously is not the same) and then run a regression on this "RES_" value (as dependent variable)? The idea behind it: The "RES_" value should be free of systematic trend, periodicity, so they can be used in a linear regression. "RES_" is, at least in my data almost N~ distributed with µ~0.

(BTW: The question is if certain events during the time have had in influence on the time series)

[drfg2008]

@John: The problem is, I need to know, if the predictor is stat. sig. or not. As far as I can see from the IBM manual (and from the software) the predictors can be taken into the model, but how can you determine if the pred. is stat. sig.? Plus, the size of the effect would matter. Or have I completely overlooked something?

@David: Thank you, I'll check that asap.