Minimization function
Locked
 
|
SPSS-L, Suppose we have the following three equations: a + b = 6 2a + b = 10 3a + b = 12 ...and we want to find values for the parameters, a and b, that are as close to the optimal solution as possible.
Rules: 1. a and b must be >0 2. a and b yield totals such that the sum of the squared differences between observations (6, 10, 12) 3.The totals (sum of squared differences) should be as small as possible
Examining the equations above, one can see that the optimal solutions for a and b must be between 3 and 4. With that knowledge, one can manually arrive at the solutions for a and b to the first decimal place (a is 3.0 and b=3.3) without great difficulty:
3.0 + 3.3 = 6.3 2 (3.0) + 3.3 = 9.3 3 (3.0) + 3.3 = 12.3 (6.3 - 6)**2 + (9.3 - 10)**2 + (12.3 - 12)**2 = .67 I'm wondering, however, if there is a way in SPSS to create a program to arrive at the optimal solution for a and b (to the first decimal place) without having to do this manually (e.g., first try a=3.0, b=3.1; then try a=3.0, b=3.2, etc.)
Any hints would be appreciated. Best, Ryan |
|
|
I was given some advice off-list to regress y (6, 10, 12) on a (1, 2, 3) and b (1, 1, 1)-->this produces the optimal solution. Note that it is not necessary to include b since it is a constant. At any rate, the intercept is the value for b (3.333) and the slope is the value for a (3.000).
On Tue, Sep 25, 2012 at 7:22 PM, R B <[hidden email]> wrote:
|
|
|
OLS obviously won't be the solution when OLS estimates violate your given constraints. See the NLR and the CNLR commands, I believe they are what you need to fix such constraints.
Perhaps some of the other statisticians on the list could comment, but I never quite groked such constraints. You could always just estimate the OLS values, and then make some linear transformation to the independent variables and then voila, your beta estimates meet your constraints. Andy |
|
|
Well, no. When you transform the
independent variables, you also have to transform the constraints. But
CNLR can handle inequality constraints. In fact, a statistician formerly
at SPSS was actually able to make CNLR solve a linear programming problem
- not that I would recommend that over a dedicated LP program such as ILOG.
Jon Peck (no "h") aka Kim Senior Software Engineer, IBM [hidden email] new phone: 720-342-5621 From: Andy W <[hidden email]> To: [hidden email] Date: 09/25/2012 07:13 PM Subject: Re: [SPSSX-L] Minimization function Sent by: "SPSSX(r) Discussion" <[hidden email]> OLS obviously won't be the solution when OLS estimates violate your given constraints. See the NLR and the CNLR commands, I believe they are what you need to fix such constraints. Perhaps some of the other statisticians on the list could comment, but I never quite groked such constraints. You could always just estimate the OLS values, and then make some linear transformation to the independent variables and then voila, your beta estimates meet your constraints. Andy ----- Andy W [hidden email] http://andrewpwheeler.wordpress.com/ -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Minimization-function-tp5715269p5715271.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
Administrator
|
Adding a couple more data points to force the OLS solution to have a negative intercept. Followed by example of CNLR to constrain the intercept to be positive.
-- data list free / x y. begin data 1 6 2 10 3 12 4 28 5 30 end data. reg / dep y / method enter x. ** Notice OLS yields intercept -2.60, Bx=6.60 R2=.899 (Adj=.865) **. * NonLinear Regression. MODEL PROGRAM B0=0 BX=0 . COMPUTE PRED_ = B0 + Bx * x. CNLR y /PRED PRED_ /BOUNDS B0-.0001 >= 0; BX - .00001 >= 0 /CRITERIA STEPLIMIT 2 ISTEP 1E+20 . All the derivatives will be calculated numerically. Iteration Residual SS B0 BX 0.2 1963.976320 .000100000 .000010000 1.2 859.6124212 .513247883 1.93326717 2.1 55.34625359 .000100000 5.89331755 3.1 55.34592728 .000100000 5.89088152 4.1 55.34592728 .000100000 5.89088182 Run stopped after 4 major iterations. Optimal solution found. Nonlinear Regression Summary Statistics Dependent Variable Y Source DF Sum of Squares Mean Square Regression 2 1908.65407 954.32704 Residual 3 55.34593 18.44864 Uncorrected Total 5 1964.00000 (Corrected Total) 4 484.80000 R squared = 1 - Residual SS / Corrected SS = .88584 Asymptotic 95 % Asymptotic Confidence Interval Parameter Estimate Std. Error Lower Upper B0 .000100000 4.502473095 -14.32877887 14.328978865 BX 5.890881818 1.356801223 1.572934778 10.208828859 Asymptotic Correlation Matrix of the Parameter Estimates B0 BX B0 1.0000 -.9043 BX -.9043 1.0000
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me. --- "Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis." Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?" |
Automatic reply: Minimization function
|
|
I will be out of the office until Monday, October 1st, with limited access to e-mail.
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
|
In reply to this post by David Marso
Nice example David, thanks for sharing. I don't want to hijack the thread too much, but I guess what I am confused about are reasons why people want such constraints to begin with. Does anybody here mind sharing an example they've come across in their work where such constraints were needed? (RB, David, Jon anyone?)
I've seen a few similar requests over on the CV site (see here and here) but I still don't get the motivation. In those examples I'm still confused why a simple linear transformation of the independent variables wouldn't suffice. It is also confusing to me, as you can arbitrarily change the scale of an independent variables (e.g. divide it by a constant) and hence change its weight given. So what is the motivation for the constraints with the given data? Sorry for going off on a tangent, Andy |
|
|
It's not uncommon to want to impose a sign
on a coefficient, so (positive) scale and location changes would not affect
the constraint. I don't think people actually do that often, though.
One might also want to impose an equality constraint. Of course,
unit changes in one variable would affect that, but that would logically
change the constraint. And that and similar algebraic relationships
could be substituted out, but it may be more convenient to estimate in
a fully parameterized form.
Jon Peck (no "h") aka Kim Senior Software Engineer, IBM [hidden email] new phone: 720-342-5621 From: Andy W <[hidden email]> To: [hidden email] Date: 09/26/2012 01:28 PM Subject: Re: [SPSSX-L] Minimization function Sent by: "SPSSX(r) Discussion" <[hidden email]> Nice example David, thanks for sharing. I don't want to hijack the thread too much, but I guess what I am confused about are reasons why people want such constraints to begin with. Does anybody here mind sharing an example they've come across in their work where such constraints were needed? (RB, David, Jon anyone?) I've seen a few similar requests over on the CV site (see here <http://stats.stackexchange.com/q/21565/1036> and here <http://stats.stackexchange.com/q/24193/1036> ) but I still don't get the motivation. In those examples I'm still confused why a simple linear transformation of the independent variables wouldn't suffice. It is also confusing to me, as you can arbitrarily change the scale of an independent variables (e.g. divide it by a constant) and hence change its weight given. So what is the motivation for the constraints with the given data? Sorry for going off on a tangent, Andy ----- Andy W [hidden email] http://andrewpwheeler.wordpress.com/ -- View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Minimization-function-tp5715269p5715294.html Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD |
|
|
In reply to this post by Andy W
Andy, One example that immediately comes to mind is in the context of multilevel modeling where the random effect variance component estimate is constrained to be non-negative. However, simply bounding the variance can lead to non-convergence, particularly when the variance is near zero. There are ways to constrain the parameter to be non-negative without *directly* constraining the lower bound to be non-negative. Still, a random effect variance which goes negative has a specific interpretable meaning which I do not have time to discuss at the moment. At any rate, as you might imagine, it is very common practice to force a random effect variance to be non-negative--in fact, it's the default in many statistical software programs. Anyway, that's an example of constraining a parameter estimate to be non-negative that's commonly applied in multilevel modeling, for better or worse!
The example I provided previously has absolutely nothing to do with multilevel modeling but rather to do with a very useful illustration provided in a textbook around the idea of underidentification, justidentification, and overidentification in the context of structural equation modeling. Sorry but I can't elaborate further at this point in time.
Best, Ryan On Wed, Sep 26, 2012 at 3:24 PM, Andy W <[hidden email]> wrote: Nice example David, thanks for sharing. I don't want to hijack the thread too |
«
Return to SPSSX Discussion
|
1 view|%1 views
| Free forum by Nabble | Edit this page |