Calculating BIC for Curve Estimation regressions

[Carol Ma]

Hello,

My student and I are wondering if there is a way of calculating BIC for equations obtained via the curve estimation regression function. We are attempting to model data, and are considering whether it is best to fit a linear, quadratic and cubic model. Obviously the R2 increases from linear to quadratic to cubic. A colleague has suggested that the BIC is the best way to justify which type of model we fit. Can this be calculated in SPSS? Or if not, can anyone suggest a formula we can use that uses outputs provided by SPSS? We have an equation for BIC, however it requires the log liklihood, and we don't seem to have that in the SPSS outputs from the curve estimation regressions.

We are using SPSS version 22. Thanks in advance for any advice you are able to provide.

Carol and Sophia

[Maguin, Eugene]

The BIC is defined in terms of the maximum likelihood value. From Wikipedia: BIC = -2*ln(LL)+k*ln(n), where LL is the log likelihood, k is the number of regressors, counting the intercept, in regression, and n is the sample size. I may well be wrong but I don't think that Curvefit gives a maximum likelihood solution. I haven't used it for quite a while and don't remember.

However, with a tiny bit of effort you can do the analysis using Mixed, which does give a maximum likelihood solution and a BIC value. Look at the documentation but, letting y be the DV and x the IV,

Mixed y with x/print solution. /* either 'solution' or 'parameters'. Spss is flexible that way.

Compute x2=x*x.
Mixed y with x x2/print solution.

Compute x3=x2*x.
... etc.











-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Carol Ma
Sent: Tuesday, June 16, 2015 10:27 PM
To: [hidden email]
Subject: Calculating BIC for Curve Estimation regressions

Hello,

My student and I are wondering if there is a way of calculating BIC for equations obtained via the curve estimation regression function. We are attempting to model data, and are considering whether it is best to fit a linear, quadratic and cubic model. Obviously the R2 increases from linear to quadratic to cubic. A colleague has suggested that the BIC is the best way to justify which type of model we fit. Can this be calculated in SPSS? Or if not, can anyone suggest a formula we can use that uses outputs provided by SPSS? We have an equation for BIC, however it requires the log liklihood, and we don't seem to have that in the SPSS outputs from the curve estimation regressions.

We are using SPSS version 22. Thanks in advance for any advice you are able to provide.

Carol and Sophia



--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Calculating-BIC-for-Curve-Estimation-regressions-tp5729844.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Bruce Weaver]

Hello Carol.  About 5 years ago, a colleague was using CURVEFIT to make projections.  In that case, we wanted to compute the AIC values for all of the models.  IIRC, the way we did it was as follows:

1. We saved the residuals from CURVEFIT to another data set via OMS.
2. In that new data set, we squared the residuals, and used AGGREGATE to sum them.
3. We used the sum of squared residuals as SS in the AIC formula shown on p. 143 of the PRISM 4 REGRESSION BOOK:

    AIC = N*ln(SS/N) + 2K

where N = sample size, SS = sum of squared errors, K = the number of model parameters + 1.  

Judging from what I see on the page linked below, the only change you would have to make to compute BIC is to change the final term in that equation from 2K to ln(N)K.

   http://methodology.psu.edu/eresources/ask/sp07

HTH.

Carol Ma wrote

Hello,

My student and I are wondering if there is a way of calculating BIC for equations obtained via the curve estimation regression function. We are attempting to model data, and are considering whether it is best to fit a linear, quadratic and cubic model. Obviously the R2 increases from linear to quadratic to cubic. A colleague has suggested that the BIC is the best way to justify which type of model we fit. Can this be calculated in SPSS? Or if not, can anyone suggest a formula we can use that uses outputs provided by SPSS? We have an equation for BIC, however it requires the log liklihood, and we don't seem to have that in the SPSS outputs from the curve estimation regressions.

We are using SPSS version 22. Thanks in advance for any advice you are able to provide.

Carol and Sophia

[Jon K Peck]

OMS?  CURVEFIT can save residuals directly to the dataset.  OMS would not see the residuals.


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
phone: [phone]




From:        Bruce Weaver <[hidden email]>
To:        [hidden email]
Date:        06/17/2015 02:51 PM
Subject:        Re: [SPSSX-L] Calculating BIC for Curve Estimation regressions
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

Hello Carol.  About 5 years ago, a colleague was using CURVEFIT to make
projections.  In that case, we wanted to compute the AIC values for all of
the models.  IIRC, the way we did it was as follows:

1. We saved the residuals from CURVEFIT to another data set via OMS.
2. In that new data set, we squared the residuals, and used AGGREGATE to sum
them.
3. We used the sum of squared residuals as SS in the AIC formula shown on p.
143 of the PRISM 4 REGRESSION BOOK
(http://www.graphpad.com/faq/file/Prism4RegressionBook.pdf):

   AIC = N*ln(SS/N) + 2K

where N = sample size, SS = sum of squared errors, K = the number of model
parameters + 1.  

Judging from what I see on the page linked below, the only change you would
have to make to compute BIC is to change the final term in that equation
from 2K to ln(N)K.

  http://methodology.psu.edu/eresources/ask/sp07

HTH.




Carol Ma wrote
> Hello,
>
> My student and I are wondering if there is a way of calculating BIC for
> equations obtained via the curve estimation regression function. We are
> attempting to model data, and are considering whether it is best to fit a
> linear, quadratic and cubic model. Obviously the R2 increases from linear
> to quadratic to cubic. A colleague has suggested that the BIC is the best
> way to justify which type of model we fit. Can this be calculated in SPSS?
> Or if not, can anyone suggest a formula we can use that uses outputs
> provided by SPSS? We have an equation for BIC, however it requires the log
> liklihood, and we don't seem to have that in the SPSS outputs from the
> curve estimation regressions.
>
> We are using SPSS version 22. Thanks in advance for any advice you are
> able to provide.
>
> Carol and Sophia





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Calculating-BIC-for-Curve-Estimation-regressions-tp5729844p5729855.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Maguin, Eugene]

Bruce, thanks for posting that reference to that book. I didn't know that AIC/BIC could be computed with sums of squares.
Gene Maguin

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Bruce Weaver
Sent: Wednesday, June 17, 2015 4:51 PM
To: [hidden email]
Subject: Re: Calculating BIC for Curve Estimation regressions

Hello Carol.  About 5 years ago, a colleague was using CURVEFIT to make projections.  In that case, we wanted to compute the AIC values for all of the models.  IIRC, the way we did it was as follows:

1. We saved the residuals from CURVEFIT to another data set via OMS.
2. In that new data set, we squared the residuals, and used AGGREGATE to sum them.
3. We used the sum of squared residuals as SS in the AIC formula shown on p.
143 of the PRISM 4 REGRESSION BOOK
(http://www.graphpad.com/faq/file/Prism4RegressionBook.pdf):

    AIC = N*ln(SS/N) + 2K

where N = sample size, SS = sum of squared errors, K = the number of model parameters + 1.  

Judging from what I see on the page linked below, the only change you would have to make to compute BIC is to change the final term in that equation from 2K to ln(N)K.

   http://methodology.psu.edu/eresources/ask/sp07

HTH.




Carol Ma wrote

> Hello,
>
> My student and I are wondering if there is a way of calculating BIC
> for equations obtained via the curve estimation regression function.
> We are attempting to model data, and are considering whether it is
> best to fit a linear, quadratic and cubic model. Obviously the R2
> increases from linear to quadratic to cubic. A colleague has suggested
> that the BIC is the best way to justify which type of model we fit. Can this be calculated in SPSS?
> Or if not, can anyone suggest a formula we can use that uses outputs
> provided by SPSS? We have an equation for BIC, however it requires the
> log liklihood, and we don't seem to have that in the SPSS outputs from
> the curve estimation regressions.
>
> We are using SPSS version 22. Thanks in advance for any advice you are
> able to provide.
>
> Carol and Sophia

-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Calculating-BIC-for-Curve-Estimation-regressions-tp5729844p5729855.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Ryan]

Gene,

 

See here:

 

https://listserv.uga.edu/cgi-bin/wa?A2=ind1010&L=SPSSX-L&P=R31683

 

Ryan

On Wed, Jun 17, 2015 at 5:18 PM, Maguin, Eugene <[hidden email]> wrote:

Bruce, thanks for posting that reference to that book. I didn't know that AIC/BIC could be computed with sums of squares.
Gene Maguin

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Bruce Weaver
Sent: Wednesday, June 17, 2015 4:51 PM
To: [hidden email]
Subject: Re: Calculating BIC for Curve Estimation regressions

Hello Carol.  About 5 years ago, a colleague was using CURVEFIT to make projections.  In that case, we wanted to compute the AIC values for all of the models.  IIRC, the way we did it was as follows:

1. We saved the residuals from CURVEFIT to another data set via OMS.
2. In that new data set, we squared the residuals, and used AGGREGATE to sum them.
3. We used the sum of squared residuals as SS in the AIC formula shown on p.
143 of the PRISM 4 REGRESSION BOOK
(http://www.graphpad.com/faq/file/Prism4RegressionBook.pdf):

    AIC = N*ln(SS/N) + 2K

where N = sample size, SS = sum of squared errors, K = the number of model parameters + 1.

Judging from what I see on the page linked below, the only change you would have to make to compute BIC is to change the final term in that equation from 2K to ln(N)K.

   http://methodology.psu.edu/eresources/ask/sp07

HTH.




Carol Ma wrote
> Hello,
>
> My student and I are wondering if there is a way of calculating BIC
> for equations obtained via the curve estimation regression function.
> We are attempting to model data, and are considering whether it is
> best to fit a linear, quadratic and cubic model. Obviously the R2
> increases from linear to quadratic to cubic. A colleague has suggested
> that the BIC is the best way to justify which type of model we fit. Can this be calculated in SPSS?
> Or if not, can anyone suggest a formula we can use that uses outputs
> provided by SPSS? We have an equation for BIC, however it requires the
> log liklihood, and we don't seem to have that in the SPSS outputs from
> the curve estimation regressions.
>
> We are using SPSS version 22. Thanks in advance for any advice you are
> able to provide.
>
> Carol and Sophia





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Calculating-BIC-for-Curve-Estimation-regressions-tp5729844p5729855.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

[Bruce Weaver]

Good catch, Jon.  I'm sure that's exactly what we did.

Jon K Peck wrote

OMS?  CURVEFIT can save residuals directly to the dataset.  OMS would not
see the residuals.


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
phone: [phone]




From:   Bruce Weaver <[hidden email]>
To:     [hidden email]
Date:   06/17/2015 02:51 PM
Subject:        Re: [SPSSX-L] Calculating BIC for Curve Estimation
regressions
Sent by:        "SPSSX(r) Discussion" <[hidden email]>



Hello Carol.  About 5 years ago, a colleague was using CURVEFIT to make
projections.  In that case, we wanted to compute the AIC values for all of
the models.  IIRC, the way we did it was as follows:

1. We saved the residuals from CURVEFIT to another data set via OMS.
2. In that new data set, we squared the residuals, and used AGGREGATE to
sum
them.
3. We used the sum of squared residuals as SS in the AIC formula shown on
p.
143 of the PRISM 4 REGRESSION BOOK
(http://www.graphpad.com/faq/file/Prism4RegressionBook.pdf):

    AIC = N*ln(SS/N) + 2K

where N = sample size, SS = sum of squared errors, K = the number of model
parameters + 1.

Judging from what I see on the page linked below, the only change you
would
have to make to compute BIC is to change the final term in that equation
from 2K to ln(N)K.

   http://methodology.psu.edu/eresources/ask/sp07

HTH.




Carol Ma wrote
> Hello,
>
> My student and I are wondering if there is a way of calculating BIC for
> equations obtained via the curve estimation regression function. We are
> attempting to model data, and are considering whether it is best to fit
a
> linear, quadratic and cubic model. Obviously the R2 increases from
linear
> to quadratic to cubic. A colleague has suggested that the BIC is the
best
> way to justify which type of model we fit. Can this be calculated in
SPSS?
> Or if not, can anyone suggest a formula we can use that uses outputs
> provided by SPSS? We have an equation for BIC, however it requires the
log
> liklihood, and we don't seem to have that in the SPSS outputs from the
> curve estimation regressions.
>
> We are using SPSS version 22. Thanks in advance for any advice you are
> able to provide.
>
> Carol and Sophia





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context:
http://spssx-discussion.1045642.n5.nabble.com/Calculating-BIC-for-Curve-Estimation-regressions-tp5729844p5729855.html

Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD



=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Sophia Ho]

This is great. Thank you all for your help in this!

Sophia.