autocorrelation in linear regression

[pseudoyou]

Hi, 

What is the proper way to test for the assumption that the residuals are not correlated? I don't have time-series data, so Durbin Watson seems irrelevant. I know there is a visual inspection but it does not seem convincing to certain people. Is there any statistic that can be calculated for a regular multivariate linear regression? 

Thanks

Jon

[David Marso]

"I don't have time-series data..."
So how is autocorrelation even relevant?
Visual inspection? Of what?  And yes, in any case; clearly unconvincing!
Please use statistical concepts correctly!
"multivariate linear regression" implies you have multiple dependent variables.
I suspect the term you seek is Multiple Regression.
--
--

pseudoyou wrote

Hi,

What is the proper way to test for the assumption that the residuals are
not correlated? I don't have time-series data, so Durbin Watson seems
irrelevant. I know there is a visual inspection but it does not seem
convincing to certain people. Is there any statistic that can be calculated
for a regular multivariate linear regression?

Thanks
Jon

[ViAnn Beadle]

I guess the question here is correlation of the residuals with what?

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
David Marso
Sent: Thursday, September 20, 2012 9:57 AM
To: [hidden email]
Subject: Re: autocorrelation in linear regression

"I *don't* have time-series data..."
So how is *autocorrelation *even relevant?
Visual inspection? Of what?  And yes, in any case; clearly *unconvincing*!
Please use statistical concepts correctly!
"multivariate linear regression" implies you have multiple *dependent
*variables.
I suspect the term you seek is Multiple Regression.
--
--

pseudoyou wrote

> Hi,
>
> What is the proper way to test for the assumption that the residuals
> are not correlated? I don't have time-series data, so Durbin Watson
> seems irrelevant. I know there is a visual inspection but it does not
> seem convincing to certain people. Is there any statistic that can be
> calculated for a regular multivariate linear regression?
>
> Thanks
> Jon

-----
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email
me.
--
View this message in context:
http://spssx-discussion.1045642.n5.nabble.com/autocorrelation-in-linear-regr
ession-tp5715185p5715188.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command SIGNOFF SPSSX-L For a list of
commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

I don't think it's crazy to ask if there is any autocorrelation of the errors in ordinary linear regression models (i.e., not time series).  It can also result from mis-specification of the model.  Suppose the true relationship between X and Y was curvilinear -- e.g., inverted U for Yerkes-Dodson law (X = arousal, Y = performance).  If you fit a straight line, the errors will not be independent of each other.


Cheers,
Bruce

David Marso wrote

"I don't have time-series data..."
So how is autocorrelation even relevant?
Visual inspection? Of what?  And yes, in any case; clearly unconvincing!
Please use statistical concepts correctly!
"multivariate linear regression" implies you have multiple dependent variables.
I suspect the term you seek is Multiple Regression.
--
--

pseudoyou wrote

Hi,

What is the proper way to test for the assumption that the residuals are
not correlated? I don't have time-series data, so Durbin Watson seems
irrelevant. I know there is a visual inspection but it does not seem
convincing to certain people. Is there any statistic that can be calculated
for a regular multivariate linear regression?

Thanks
Jon

[David Marso]

I don't believe the term 'autocorrelation' applies outside the realm of time structured data.  At least I have never seen it correctly used outside of time series.

Bruce Weaver wrote

I don't think it's crazy to ask if there is any autocorrelation of the errors in ordinary linear regression models (i.e., not time series).  It can also result from mis-specification of the model.  Suppose the true relationship between X and Y was curvilinear -- e.g., inverted U for Yerkes-Dodson law (X = arousal, Y = performance).  If you fit a straight line, the errors will not be independent of each other.


Cheers,
Bruce

David Marso wrote

"I don't have time-series data..."
So how is autocorrelation even relevant?
Visual inspection? Of what?  And yes, in any case; clearly unconvincing!
Please use statistical concepts correctly!
"multivariate linear regression" implies you have multiple dependent variables.
I suspect the term you seek is Multiple Regression.
--
--

pseudoyou wrote

Hi,

What is the proper way to test for the assumption that the residuals are
not correlated? I don't have time-series data, so Durbin Watson seems
irrelevant. I know there is a visual inspection but it does not seem
convincing to certain people. Is there any statistic that can be calculated
for a regular multivariate linear regression?

Thanks
Jon

[Jon K Peck]

Misspecification may well induce a correlation pattern in the residuals, but it will only show up as autocorrelation if it happens that the misspecified variable values happen to fall in value order.

There are plenty of tests for different types of patterns of correlation in the errors, but you need some sort of hypothesized pattern.  Plotting residuals vs fitted values is a pretty good place to start, bearing in mind influence problems.

For looking at functional form issues pairwise, the STATS REGRESS PLOT extension command gives you a quick, compact way to check for nonlinearities.

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
new phone: [phone]




From:        Bruce Weaver <[hidden email]>
To:        [hidden email]
Date:        09/20/2012 12:42 PM
Subject:        Re: [SPSSX-L] autocorrelation in linear regression
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

I don't think it's crazy to ask if there is any autocorrelation of the errors
in ordinary linear regression models (i.e., not time series).  It can also
result from mis-specification of the model.  Suppose the true relationship
between X and Y was curvilinear -- e.g., inverted U for Yerkes-Dodson law (X
= arousal, Y = performance).  If you fit a straight line, the errors will
not be independent of each other.


Cheers,
Bruce



David Marso wrote
> "I
*
> don't
*
>  have time-series data..."
> So how is
*
> autocorrelation
*
> even relevant?
> Visual inspection? Of what?  And yes, in any case; clearly
*
> unconvincing
*
> !
> Please use statistical concepts correctly!
> "multivariate linear regression" implies you have multiple
*
> dependent
*
> variables.
> I suspect the term you seek is Multiple Regression.
> --
> --
> pseudoyou wrote
>> Hi,
>>
>> What is the proper way to test for the assumption that the residuals are
>> not correlated? I don't have time-series data, so Durbin Watson seems
>> irrelevant. I know there is a visual inspection but it does not seem
>> convincing to certain people. Is there any statistic that can be
>> calculated
>> for a regular multivariate linear regression?
>>
>> Thanks
>> Jon





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/autocorrelation-in-linear-regression-tp5715185p5715190.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Andy W]

You can have spatially auto-correlated data. Another example of non-traditional situation in which auto-correlation was inspected can be found in the following;

Ratcliffe, JH (in press) The spatial extent of criminogenic places on the surrounding environment: A change-point regression of violence around bars, Geographical Analysis.

I bring it up partly because IMO he says he conducted a DW test (see endnotes) and there was no auto-correlation in the residuals, but if you look at his plot of the regression line fits on page 15 there is very clear auto-correlation in the series. I'm not sure what the power of the DW test is offhand, but I wonder in such short series is powerful enough to reject the test. It doesn't have any effect on his findings I believe, but here clear graphical presentation is sufficient to demonstrate auto-correlation in the series.

It is a bit of semantics, but I would agree with David that the term "auto-correlation" is typically only in reference to data that are not "independent" (i.e. the units of analysis have some type of relationship between one another). I suspect you could make some mathematical connections between that and non-linear functional forms, but I don't think such connections will be helpful for literature reviews etc.

Best to hear back from OP though before deciding test (spatial & temporal auto-correlation will certainly require a different test than non-linear functional form, e.g RESET test). Although I don't hold such negativity to visual inspections as do the other posters. See for instance;

Statistical inference for exploratory data analysis and model diagnostics
by: Andreas Buja, Dianne Cook, Heike Hofmann, Michael Lawrence, Eun-Kyung Lee, Deborah F. Swayne, Hadley Wickham. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 367, No. 1906. (13 November 2009), pp. 4361-4383, doi:10.1098/rsta.2009.0120

They suggest a statistical test based on visual inspection of the series under investigation versus 10 other plots of random data. Examining residuals should be conducted regularly IMO. It may not take the place of tests always, but residual plots can be pretty convincing sometimes.

Andy