Python: List Index out of range

[drfg2008]

Index out of range

I have problems with a python loop. I try to compute a loop from 1 to 350. However after only 18 loops the program prints an error and stops. If I start the program from 17 on, the loop starts with an error message right from the beginning. I.e. I use the code: COMPUTE filter_$=(sicyear_rec = """ + str(i) + """) or similar codes (see below). Might this be the cause, to concatenate string with a number that is changed into a string ? It works at least the first 17 loops.



Here the Error Message:

Traceback (most recent call last):
  File "<string>", line 53, in <module>
IndexError: list index out of range


Code  excerpt:

BEGIN PROGRAM.
import spss, spssaux
durchlauf = len(spssaux.VariableDict(['SICYear'])[0].ValueLabels) + 1


i = 1

while i < durchlauf:
     print i
     spss.Submit(r"""
     DATASET ACTIVATE Kontroll_t0.
     USE ALL.
     COMPUTE filter_$=(sicyear_rec = """ + str(i) + """).
     FILTER BY filter_$.
     EXECUTE.
     """)

… there is more code …


     spss.Submit(r"""

     IF  (sicyear_rec = """ +str(i) + """) NDAC=(a * TA_1_reverse + b1 * c_REV_c_REC.TA + b2 * PPE.TA) / TA_1.
     EXECUTE.
     formats NDAC (f8.4).
     DELETE VARIABLES a b1 b2.
     EXECUTE .
     """)


     i+=1

END PROGRAM.

[Jon K Peck]

The traceback is telling you that the error occurred at line 53 of your program, but that is not in the code you showed.
Also, though unrelated, you should remove the EXECUTE statements in the code.  They do nothing but slow things down here.
If you can't track down the problem, please show the entire program.

Jon Peck
Senior Software Engineer, IBM
[hidden email]
new phone: [phone]




From:        drfg2008 <[hidden email]>
To:        [hidden email]
Date:        06/03/2011 04:50 AM
Subject:        [SPSSX-L] Python: List Index out of range
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

Index out of range

I have problems with a python loop. I try to compute a loop from 1 to 350.
However after only 18 loops the program prints an error and stops. If I
start the program from 17 on, the loop starts with an error message right
from the beginning. I.e. I use the code: COMPUTE filter_$=(sicyear_rec = """
+ str(i) + """) or similar codes (see below). Might this be the cause, to
concatenate string with a number that is changed into a string ? It works at
least the first 17 loops.



Here the Error Message:

Traceback (most recent call last):
 File "<string>", line 53, in <module>
IndexError: list index out of range


Code  excerpt:

BEGIN PROGRAM.
import spss, spssaux
durchlauf = len(spssaux.VariableDict(['SICYear'])[0].ValueLabels) + 1


i = 1

while i < durchlauf:
    print i
    spss.Submit(r"""
    DATASET ACTIVATE Kontroll_t0.
    USE ALL.
    COMPUTE filter_$=(sicyear_rec = """ + str(i) + """).
    FILTER BY filter_$.
    EXECUTE.
    """)

… there is more code …


    spss.Submit(r"""

    IF  (sicyear_rec = """ +str(i) + """) NDAC=(a * TA_1_reverse + b1 *
c_REV_c_REC.TA + b2 * PPE.TA) / TA_1.
    EXECUTE.
    formats NDAC (f8.4).
    DELETE VARIABLES a b1 b2.
    EXECUTE .
    """)


    i+=1

END PROGRAM.


-----
Dr. Frank Gaeth
FU-Berlin

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Python-List-Index-out-of-range-tp4450774p4450774.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[drfg2008]

Thanks Jon,

I can't still find the problem. The error turns up only after several rounds (17 loops).


The error message:

Traceback (most recent call last):
  File "<string>", line 53, in <module> 
IndexError: list index out of range

here is the complete code (below).




BEGIN PROGRAM.
import spss, spssaux
durchlauf = len(spssaux.VariableDict(['SICYear'])[0].ValueLabels) + 1

print durchlauf

abc = 1

while abc < 30:
     print abc
     spss.Submit(r"""
     DATASET ACTIVATE Kontroll_t0.
     USE ALL.
     COMPUTE filter_$=(sicyear_rec = """ + str(abc) + """).
     FILTER BY filter_$.
     EXECUTE.
     """)

     cmd=r"""
     REGRESSION
       /MISSING LISTWISE
       /STATISTICS COEFF OUTS R ANOVA
       /CRITERIA=PIN(.05) POUT(.10)
       /ORIGIN
       /DEPENDENT TAC.TA
       /METHOD=ENTER TA_1_reverse c_REV_c_REC.TA PPE.TA.
     """

     desc_table,errcode=spssaux.CreateXMLOutput(
                                                                          cmd,
                                                                          omsid="Regression")
     alpha=spssaux.GetValuesFromXMLWorkspace(
                                                                              desc_table,
                                                                              tableSubtype="Coefficients",
                                                                              rowCategory="TA_1_reverse",
                                                                              colCategory="RegressionskoeffizientB",
                                                                              cellAttrib="text")


     beta1=spssaux.GetValuesFromXMLWorkspace(
                                                                              desc_table,
                                                                              tableSubtype="Coefficients",
                                                                              rowCategory="c_REV_c_REC.TA",
                                                                              colCategory="RegressionskoeffizientB",
                                                                              cellAttrib="text")
     beta2=spssaux.GetValuesFromXMLWorkspace(
                                                                              desc_table,
                                                                              tableSubtype="Coefficients",
                                                                              rowCategory="PPE.TA",
                                                                              colCategory="RegressionskoeffizientB",
                                                                              cellAttrib="text")
     a=alpha[0]
     b1=beta1[0]
     b2=beta2[0]

     print "a beträgt: ", a
     print "b1 beträgt: ", b1
     print "b2 beträgt: ", b2

     spss.Submit(r"""DATASET ACTIVATE Stp_t0.
     COMPUTE a=NUMBER('"""+a+"""',f8).
     COMPUTE b1=NUMBER('"""+b1+"""',f8).
     COMPUTE b2=NUMBER('"""+b2+"""',f8).
     EXECUTE.
     formats a (f8.4).
     formats b1 (f8.4).
     formats b2 (f8.4).
     """)


     spss.Submit(r"""

     IF  (sicyear_rec = """ +str(abc) + """) NDAC=(a * TA_1_reverse + b1 * c_REV_c_REC.TA + b2 * PPE.TA) / TA_1.
     EXECUTE.
     formats NDAC (f8.4).
     DELETE VARIABLES a b1 b2.
     """)

     abc+=1

END PROGRAM.

[Jon K Peck]

I can't count the lines, since the email has apparently added blank lines, but I suspect that the problem is that
spssaux.GetValuesFromXMLWorkspace has returned an empty list, so references such as alpha[0] would be invalid.
Check the length of the return value from that call.

If you are having trouble constructing the GetValuesFromXMLWorkspace call, which can be complicated, you might want to get the pivottablestruct module from the SPSS Community site and have it display the structure of the pivot table.

If you still can't figure this out, send the program as an attachment directly to my email address.

HTH,

Jon Peck
Senior Software Engineer, IBM
[hidden email]
new phone: [phone]




From:        drfg2008 <[hidden email]>
To:        [hidden email]
Date:        06/03/2011 08:35 AM
Subject:        Re: [SPSSX-L] Python: List Index out of range
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

Thanks Jon,

I can't still find the problem. The error turns up only after several rounds
(17 loops).


The error message:

Traceback (most recent call last):
 File "<string>", line 53, in <module>
IndexError: list index out of range

here is the complete code (below).




BEGIN PROGRAM.
import spss, spssaux
durchlauf = len(spssaux.VariableDict(['SICYear'])[0].ValueLabels) + 1

print durchlauf

abc = 1

while abc < 30:
    print abc
    spss.Submit(r"""
    DATASET ACTIVATE Kontroll_t0.
    USE ALL.
    COMPUTE filter_$=(sicyear_rec = """ + str(abc) + """).
    FILTER BY filter_$.
    EXECUTE.
    """)

    cmd=r"""
    REGRESSION
      /MISSING LISTWISE
      /STATISTICS COEFF OUTS R ANOVA
      /CRITERIA=PIN(.05) POUT(.10)
      /ORIGIN
      /DEPENDENT TAC.TA
      /METHOD=ENTER TA_1_reverse c_REV_c_REC.TA PPE.TA.
    """

    desc_table,errcode=spssaux.CreateXMLOutput(

cmd,

omsid="Regression")
    alpha=spssaux.GetValuesFromXMLWorkspace(

desc_table,

tableSubtype="Coefficients",

rowCategory="TA_1_reverse",

colCategory="RegressionskoeffizientB",

cellAttrib="text")


    beta1=spssaux.GetValuesFromXMLWorkspace(

desc_table,

tableSubtype="Coefficients",

rowCategory="c_REV_c_REC.TA",

colCategory="RegressionskoeffizientB",

cellAttrib="text")
    beta2=spssaux.GetValuesFromXMLWorkspace(

desc_table,

tableSubtype="Coefficients",

rowCategory="PPE.TA",

colCategory="RegressionskoeffizientB",

cellAttrib="text")
    a=alpha[0]
    b1=beta1[0]
    b2=beta2[0]

    print "a beträgt: ", a
    print "b1 beträgt: ", b1
    print "b2 beträgt: ", b2

    spss.Submit(r"""DATASET ACTIVATE Stp_t0.
    COMPUTE a=NUMBER('"""+a+"""',f8).
    COMPUTE b1=NUMBER('"""+b1+"""',f8).
    COMPUTE b2=NUMBER('"""+b2+"""',f8).
    EXECUTE.
    formats a (f8.4).
    formats b1 (f8.4).
    formats b2 (f8.4).
    """)


    spss.Submit(r"""

    IF  (sicyear_rec = """ +str(abc) + """) NDAC=(a * TA_1_reverse + b1 *
c_REV_c_REC.TA + b2 * PPE.TA) / TA_1.
    EXECUTE.
    formats NDAC (f8.4).
    DELETE VARIABLES a b1 b2.
    """)

    abc+=1

END PROGRAM.

-----
Dr. Frank Gaeth
FU-Berlin

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Python-List-Index-out-of-range-tp4450774p4451317.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[drfg2008]

Yes, that seems to be the problem.

this is "line 53":

     b2=beta2[0]

when I change all coefficients (a, b1, b2) into  constants i.e. 1, then the whole program runs through without problem. Is there a command to tell python to ignore returned empty lists and continue with the next loop? Because if I start with loop number 19 (after the trouble-loop nr.18), the program runs again until it comes later to a second hold. And so on.

Or - as an alternative- is there a loop - program (like: range(from to; from to; from to) where I can omit the problematic loop numbers, which I know?

Frank

[David Marso]

My 2 cents...
What is the ultimate goal of this?
To learn python?
To reinvent the wheel by hacking together something which is already built into SPSS (i.e. saving predicted values)?
IMNSHO:  Python and Macros should only be used for things which cannot be done without them.
If you outline your big picture I am sure there is an easier way to achieve it.
OTOH: Maybe you are doing this to ensure job security.
---

drfg2008 wrote

Yes, that seems to be the problem.

this is "line 53":

     b2=beta2[0]

when I change all coefficients (a, b1, b2) into  constants i.e. 1, then the whole program runs through without problem. Is there a command to tell python to ignore returned empty lists and continue with the next loop? Because if I start with loop number 19 (after the trouble-loop nr.18), the program runs again until it comes later to a second hold. And so on.

Or - as an alternative- is there a loop - program (like: range(from to; from to; from to) where I can omit the problematic loop numbers, which I know?

Frank

[Jon K Peck]

All you need to do is to add into your loop
if len(the-name-of-the-returned-value-variable) == 0:
  continue

That will resume the loop with the next iteration.

Jon Peck
Senior Software Engineer, IBM
[hidden email]
new phone: [phone]




From:        drfg2008 <[hidden email]>
To:        [hidden email]
Date:        06/03/2011 09:56 AM
Subject:        Re: [SPSSX-L] Python: List Index out of range
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

Yes, that seems to be the problem.

this is "line 53":

    b2=beta2[0]

when I change all coefficients (a, b1, b2) into  constants i.e. 1, then the
whole program runs through without problem. Is there a command to tell
python to ignore returned empty lists and continue with the next loop?
Because if I start with loop number 19 (after the trouble-loop nr.18), the
program runs again until it comes later to a second hold. And so on.

Or - as an alternative- is there a loop - program (like: range(from to; from
to; from to) where I can omit the problematic loop numbers, which I know?

Frank

-----
Dr. Frank Gaeth
FU-Berlin

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Python-List-Index-out-of-range-tp4450774p4451662.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[drfg2008]

Thanks Jon,

as a PHP-devotee, but a python dyslexics, I tried your code. Without any success. Can you give a little more hint where to put it exactly, and is this a hopeful approach ...


if len(beta2[0]) == 0:
  continue


or a sign to chage the profession?


Frank

(Thanks David! Yes, life is all about money. PS after I've finished the programming, we can check again, which program is faster, mine or yours ;-)

[Jon K Peck]

You have a call that retrieves something, hopefully not just an empty list, like

results = spssaux.GetValuesFromXmlWorkspace(...)

That could be an empty list, in which case you can't access results[0].  (All indexing in Python starts at 0.)

So you would use as the test
len(results)
If that is 0, you just got back an empty list.

So inside your loop, just put that test

while ...:

    if len(results) == 0:
      continue

That will skip the rest of the body of the loop and resume with the next iteration.

HTH,

Jon Peck
Senior Software Engineer, IBM
[hidden email]
new phone: [phone]




From:        drfg2008 <[hidden email]>
To:        [hidden email]
Date:        06/03/2011 11:16 AM
Subject:        Re: [SPSSX-L] Python: List Index out of range
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

Thanks Jon,

as a PHP-devotee, but a python dyslexics, I tried your code. Without any
success. Can you give a little more hint where to put it exactly, and is
this a hopeful approach ...


if len(beta2[0]) == 0:
 continue


or a sign to chage the profession?


Frank

(Thanks David! Yes, life is all about money. PS after I've finished the
programming, we can check again, which program is faster, mine or yours ;-)

-----
Dr. Frank Gaeth
FU-Berlin

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Python-List-Index-out-of-range-tp4450774p4451886.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[David Marso]

"(Thanks David! Yes, life is all about money. PS after I've finished the programming, we can check again, which program is faster, mine or yours ;-) "

NO!!  If it were I wouldn't be giving out FREE advice and I would be a rich son of a bitch and hoarding my knowledge like a filthy capitalist pig!---Take that to the bank!!!!---

Faster isn't always the most important thing.  Elegance, Simplicity, Maintainability ...are much more important.  OTOH, any code I write will *almost* always run faster than anything anyone else is likely to come up with and will be simpler, much more elegant and self documenting.

No need to slam the Reg coefficients into XML, read them and then roll your own prediction equation.
Seriously, REVIEW the 'fine manual' WRT the capabilities of RAW out of the box SPSS before getting 'fancy'.
If you are trying to get repeated analyses for each of a set of values in a variable or combination of variables see SPLIT FILE (BYE BYE Outer loop).
Have you tried simply saving the predicted values from REGRESSION command?
See SAVE subcommand (Goodbye Rube Goldberg XML/Roll your own prediction scheme).

My version does not support Python, but simply because the new versions do does NOT mean Python code should be substituted for built in functionality.

Begin with an overall plan.
Determine what exists out of the box.
Build that.
use Python or MACRO selectively to add additional functionality.

Definitely learn to walk before you run and respect the KISS principle!!!

It's great that Jon is providing advice.
OTOH, jumping through Python hoops for this is putting the cart before the horse.
I have been f'n around with the SPSS program for over 20 years (probably closer to 25)
and know it inside and out (aside from Python, OMS and GPL, but that would be a quick study).
Not sure we will have a chance to 'race' because I have no intentions of translating the python spaghetti into standard SPSS.


 

[drfg2008]

This is exactly what I did, but it doesn't want to work.


Here is the full syntax:




BEGIN PROGRAM.
import spss, spssaux
durchlauf = len(spssaux.VariableDict(['SICYear'])[0].ValueLabels) + 1

print durchlauf

i = 1

while i < durchlauf:
     print i
     

     
     spss.Submit(r"""
     DATASET ACTIVATE Kontroll_t0.
     USE ALL.
     COMPUTE filter_$=(sicyear_rec = """ + str(i) + """).
     FILTER BY filter_$.
     EXECUTE.
     """)

     cmd=r"""
     REGRESSION
       /MISSING LISTWISE
       /STATISTICS COEFF OUTS R ANOVA
       /CRITERIA=PIN(.05) POUT(.10)
       /ORIGIN
       /DEPENDENT TAC.TA
       /METHOD=ENTER TA_1_reverse c_REV_c_REC.TA PPE.TA.
     """

     desc_table,errcode=spssaux.CreateXMLOutput(
                                                                          cmd,
                                                                          omsid="Regression")
     alpha=spssaux.GetValuesFromXMLWorkspace(
                                                                              desc_table,
                                                                              tableSubtype="Coefficients",
                                                                              rowCategory="TA_1_reverse",
                                                                              colCategory="RegressionskoeffizientB",
                                                                              cellAttrib="text")


     beta1=spssaux.GetValuesFromXMLWorkspace(
                                                                              desc_table,
                                                                              tableSubtype="Coefficients",
                                                                              rowCategory="c_REV_c_REC.TA",
                                                                              colCategory="RegressionskoeffizientB",
                                                                              cellAttrib="text")
     beta2=spssaux.GetValuesFromXMLWorkspace(
                                                                              desc_table,
                                                                              tableSubtype="Coefficients",
                                                                              rowCategory="PPE.TA",
                                                                              colCategory="RegressionskoeffizientB",
                                                                              cellAttrib="text")
     
     if len(beta2) == 0:
          continue

          a=alpha[0]
          b1=beta1[0]
          b2=beta2[0]

          print "a beträgt: ", a
          print "b1 beträgt: ", b1
          print "b2 beträgt: ", b2

          spss.Submit(r"""DATASET ACTIVATE Stp_t0.
          COMPUTE a=NUMBER('"""+a+"""',f8).
          COMPUTE b1=NUMBER('"""+b1+"""',f8).
          COMPUTE b2=NUMBER('"""+b2+"""',f8).
          EXECUTE.
          formats a (f8.4).
          formats b1 (f8.4).
          formats b2 (f8.4).
          """)


          spss.Submit(r"""
 
          IF  (sicyear_rec = """ +str(i) + """) NDAC=(a * TA_1_reverse + b1 * c_REV_c_REC.TA + b2 * PPE.TA) / TA_1.
          EXECUTE.
          formats NDAC (f8.4).
          DELETE VARIABLES a b1 b2.
          """)

i+=1

END PROGRAM.

[David Marso]

SORT CASES BY SICYear.
SPLIT FILE BY SICYear.
REGRESSION
       /VARIABLES TAC.TA TA_1_reverse c_REV_c_REC.TA PPE.TA
       /DEPENDENT TAC.TA
       /METHOD=ENTER TA_1_reverse c_REV_c_REC.TA PPE.TA
       /SAVE PRED(NDAC).

MACRO version...

DEFINE regsave (SPLITS !CHAREND ("/")
                         /DEP    !CHAREND("/")
                         /INDEP !CHAREND("/")
                         /PRED  !CMDEND           ).
SORT CASES BY !SPLITS.
SPLIT FILE BY !SPLITS.
REGRESSION
       /VARIABLES !DEP !INDEP
       /DEPENDENT !DEP
       /METHOD=ENTER !INDEP
       /SAVE PRED(!PRED).
!ENDDEFINE .

regsave SPLITS=SICYear
           /DEP=TAC.TA
           /INDEP=TA_1_reverse c_REV_c_REC.TA PPE.TA
           /PRED=NDAC .
-------------------

drfg2008 wrote

This is exactly what I did, but it doesn't want to work.


Here is the full syntax:




BEGIN PROGRAM.
import spss, spssaux
durchlauf = len(spssaux.VariableDict(['SICYear'])[0].ValueLabels) + 1

print durchlauf

i = 1

while i < durchlauf:
     print i
     

     
     spss.Submit(r"""
     DATASET ACTIVATE Kontroll_t0.
     USE ALL.
     COMPUTE filter_$=(sicyear_rec = """ + str(i) + """).
     FILTER BY filter_$.
     EXECUTE.
     """)

     cmd=r"""
     REGRESSION
       /MISSING LISTWISE
       /STATISTICS COEFF OUTS R ANOVA
       /CRITERIA=PIN(.05) POUT(.10)
       /ORIGIN
       /DEPENDENT TAC.TA
       /METHOD=ENTER TA_1_reverse c_REV_c_REC.TA PPE.TA.
     """

     desc_table,errcode=spssaux.CreateXMLOutput(
                                                                          cmd,
                                                                          omsid="Regression")
     alpha=spssaux.GetValuesFromXMLWorkspace(
                                                                              desc_table,
                                                                              tableSubtype="Coefficients",
                                                                              rowCategory="TA_1_reverse",
                                                                              colCategory="RegressionskoeffizientB",
                                                                              cellAttrib="text")


     beta1=spssaux.GetValuesFromXMLWorkspace(
                                                                              desc_table,
                                                                              tableSubtype="Coefficients",
                                                                              rowCategory="c_REV_c_REC.TA",
                                                                              colCategory="RegressionskoeffizientB",
                                                                              cellAttrib="text")
     beta2=spssaux.GetValuesFromXMLWorkspace(
                                                                              desc_table,
                                                                              tableSubtype="Coefficients",
                                                                              rowCategory="PPE.TA",
                                                                              colCategory="RegressionskoeffizientB",
                                                                              cellAttrib="text")
     
     if len(beta2) == 0:
          continue

          a=alpha[0]
          b1=beta1[0]
          b2=beta2[0]

          print "a beträgt: ", a
          print "b1 beträgt: ", b1
          print "b2 beträgt: ", b2

          spss.Submit(r"""DATASET ACTIVATE Stp_t0.
          COMPUTE a=NUMBER('"""+a+"""',f8).
          COMPUTE b1=NUMBER('"""+b1+"""',f8).
          COMPUTE b2=NUMBER('"""+b2+"""',f8).
          EXECUTE.
          formats a (f8.4).
          formats b1 (f8.4).
          formats b2 (f8.4).
          """)


          spss.Submit(r"""
 
          IF  (sicyear_rec = """ +str(i) + """) NDAC=(a * TA_1_reverse + b1 * c_REV_c_REC.TA + b2 * PPE.TA) / TA_1.
          EXECUTE.
          formats NDAC (f8.4).
          DELETE VARIABLES a b1 b2.
          """)

i+=1

END PROGRAM.

[drfg2008]

I’ve found the solution at last: if len(alpha) ==1  instead of 0.

if len(alpha) ==1  and len(beta1) == 1 and len(beta2) ==1:
…
else:
        continue


Thanks for the hint of the “out of range” problem. I would never have happened on it.


@David: The task was to generate (calibrate) a regression model in file1 and use it in file2. Actually regressions over 351 splits in each file multiplied by 3 different times (t0, t1, t2) multiplied by 2 different regression models (3 variables and 4 variables). I suggested a different approach (multilevel regression) and reducing the split-groups, but the customer insisted of computing about 2100 regressions (because this was the method described in the literature). So, what can I do?

Frank

[David Marso]

drfg2008 wrote

I’ve found the solution at last: if len(alpha) ==1  instead of 0.

if len(alpha) ==1  and len(beta1) == 1 and len(beta2) ==1:
…
else:
        continue


Thanks for the hint of the “out of range” problem. I would never have happened on it.


>@David: The task was to generate (calibrate) a regression model in file1 and use it in file2.
TRIVIAL!  See SELECT subcommand with SAVE PRED(varname)within REGRESSION (assumes your files are stacked using ADD FILES with an identifier). This also respects SPLIT FILE -cases which match the SELECT clause are used to estimate the model.  If /SAVE is specified the unselected cases can have predicted values estimated using the equation calibrated in the SELECT.

Actually regressions over 351 splits in each file multiplied by 3 different times (t0, t1, t2) multiplied by 2 different regression models (3 variables and 4 variables). I suggested a different approach (multilevel regression) and reducing the split-groups, but the customer insisted of computing about 2100 regressions (because this was the method described in the literature). So, what can I do?

Frank,
  It looks like you are getting lost in the forest due to unfamiliarity with a few important trees.  
Notably the following and how the operate together.
ADD FILES
SORT CASES
SPLIT FILE
SELECT/SAVE subcommands within REGRESSION and consider how they work together.
after that do something like the following.
Study the outcome....
THEN AND *ONLY* THEN bring Python or MACRO into the scene.
Allow SPSS to handle as much of the messy repetitive stuff and use Python like a pair of tweezers pulling a splinter from a pinkie finger rather than as a hammer where everything is a nail ;-)
Consider the following:
Consider wrapping the basic meat in a python loop but let SPSS take care of all the crossvalidation and iterative (SPLIT) stuff.  You will have more hair next week .
GET
  FILE='C:\Program Files\SPSS\1991 U.S. General Social Survey.sav'
 /KEEP EDUC HAPPY LIFE SIBS AGE RACE.
SORT CASES BY EDUC RACE.
SPLIT FILE BY EDUC.
MISSING VALUES ALL ().

REGRESSION VARIABLES happy life sibs age
 /SELECT RACE EQ 1
 /DEP happy
 /METHOD ENTER life sibs age
 /SAVE PRED(RACE1).

REGRESSION VARIABLES happy life sibs age
 /SELECT RACE EQ 2
 /DEP happy
 /METHOD ENTER life sibs age
 /SAVE PRED(RACE2).

REGRESSION VARIABLES happy life sibs age
 /SELECT RACE EQ 3
 /DEP happy
 /METHOD ENTER life sibs age
 /SAVE PRED(RACE3).



Frank