Statsitics in a loop

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Statsitics in a loop

Kornbrot, Diana
Statsitics in a loop Hi all experts
I wish to perform various statistics procedures for  a large number of variables
E.g. MIXED linear, so would like to repeat the following syntax for 81different variables ‘pred’

MIXED PMscore BY  pred
  /CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE)
  /FIXED= pred | SSTYPE(3)
  /METHOD=REML
  /REPEATED=pred | SUBJECT(ID) COVTYPE(UN)
  /EMMEANS=TABLES(pred).

Is this possible in SPSS?  Pr does it require python
Loop and do repeat constructions appear only to take data manipulation & transformation commands
Have attempted to put in a macro, !mixlin, as follows, with intention to calling the macro for all 81 variables, if necessary listing them by variable name. As a test have just put in the first variable, v12.

DEFINE !mixlin  (pred = !tokens(1)).
MIXED PMscore BY  pred
  /CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE)
  /FIXED= pred | SSTYPE(3)
  /METHOD=REML
  /REPEATED=pred | SUBJECT(ID) COVTYPE(UN)
  /EMMEANS=TABLES(pred).
!ENDDEFINE.
!mixlin pred = v12.

Does not work & get following error message
Text: pred Command: MIXED
An undefined variable name, or a scratch or system variable was specified in a variable list which accepts only standard variables.  Check spelling and verify the existence of this variable.
Execution of this command stops.

v12, as variable name, works fine if substituted for pred in the syntaxof the body of the macro

All help gratefully received

Best

Diana




  

Emeritus Professor Diana Kornbrot
email:  d.e.kornbrot@...    
 web:    http://dianakornbrot.wordpress.com/
Work
Department of Psychology
School of Life and Medical Sciences
University of Hertfordshire
College Lane, Hatfield, Hertfordshire AL10 9AB, UK
voice:   +44 (0) 170 728 4626
Home
19 Elmhurst Avenue
London N2 0LT, UK
voice:   +44 (0) 208  444 2081
mobile: +44 (0) 740 318 1612


Reply | Threaded
Open this post in threaded view
|

Re: Statsitics in a loop

Bruce Weaver
Administrator
Good morning Diana.  Inside the body of the macro, macro arguments must start with an exclamation point.  (Also, no need for a command terminator on the DEFINE line).  E.g.,

DEFINE !mixlin  (pred = !tokens(1))
MIXED PMscore BY  !pred
  /CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE)
  /FIXED= pred | SSTYPE(3)
  /METHOD=REML
  /REPEATED=pred | SUBJECT(ID) COVTYPE(UN)
  /EMMEANS=TABLES(pred).
!ENDDEFINE.
!mixlin pred = v12.

Given what you've said, you might want to hand the macro a LIST of variables though.

DEFINE !mixlin  (Xlist = !CMDEND)
!DO !X !IN(!Xlist)
MIXED PMscore BY  !X
  /CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE)
  /FIXED= pred | SSTYPE(3)
  /METHOD=REML
  /REPEATED=pred | SUBJECT(ID) COVTYPE(UN)
  /EMMEANS=TABLES(pred).
!DOEND
!ENDDEFINE.

!mixlin Xlist = V12 V13 V42.

HTH.


p.s. - I hope I'm not aiding and abetting a bivariate pre-screening "fishing" expedition!



Kornbrot, Diana wrote
Hi all experts
I wish to perform various statistics procedures for  a large number of variables
E.g. MIXED linear, so would like to repeat the following syntax for 81different variables 'pred'

MIXED PMscore BY  pred
  /CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE)
  /FIXED= pred | SSTYPE(3)
  /METHOD=REML
  /REPEATED=pred | SUBJECT(ID) COVTYPE(UN)
  /EMMEANS=TABLES(pred).

Is this possible in SPSS?  Pr does it require python
Loop and do repeat constructions appear only to take data manipulation & transformation commands
Have attempted to put in a macro, !mixlin, as follows, with intention to calling the macro for all 81 variables, if necessary listing them by variable name. As a test have just put in the first variable, v12.

DEFINE !mixlin  (pred = !tokens(1)).
MIXED PMscore BY  pred
  /CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE)
  /FIXED= pred | SSTYPE(3)
  /METHOD=REML
  /REPEATED=pred | SUBJECT(ID) COVTYPE(UN)
  /EMMEANS=TABLES(pred).
!ENDDEFINE.
!mixlin pred = v12.

Does not work & get following error message
Text: pred Command: MIXED
An undefined variable name, or a scratch or system variable was specified in a variable list which accepts only standard variables.  Check spelling and verify the existence of this variable.
Execution of this command stops.

v12, as variable name, works fine if substituted for pred in the syntaxof the body of the macro

All help gratefully received

Best

Diana





________________________________
Emeritus Professor Diana Kornbrot
email:  [hidden email]
 web:    http://dianakornbrot.wordpress.com/
Work
Department of Psychology
School of Life and Medical Sciences
University of Hertfordshire
College Lane, Hatfield, Hertfordshire AL10 9AB, UK
voice:   +44 (0) 170 728 4626
Home
19 Elmhurst Avenue
London N2 0LT, UK
voice:   +44 (0) 208  444 2081
mobile: +44 (0) 740 318 1612
--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

PLEASE NOTE THE FOLLOWING: 
1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above.
2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/).
Reply | Threaded
Open this post in threaded view
|

Re: Statsitics in a loop

Ruben Geert van den Berg
In reply to this post by Kornbrot, Diana
@Bruce: shouldn't it be

DEFINE !mixlin  (pred = !tokens(1))
MIXED PMscore BY  !pred
  /CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE)
  /FIXED= !pred | SSTYPE(3)
  /METHOD=REML
  /REPEATED=!pred | SUBJECT(ID) COVTYPE(UN)
  /EMMEANS=TABLES(!pred).
!ENDDEFINE.
!mixlin pred = v12.

? I sure hope the 81 variable names can somehow be created in a !DO loop with a !CONCAT because otherwise you'll need to enter all 81 names into the macro call =  no fun. If these variables are adjacent in the data, the "TO" keyword can be used if you'd fix this with Python. Or you could select the variables given a pattern in their names/variable labels/value labels/whatever.

HTH,

Ruben
Reply | Threaded
Open this post in threaded view
|

Re: Statsitics in a loop

Andy W
If mixed works with split files another approach may be to stack the dataset (VARSTOCASES) with an index, and then just split the file by that index.

In addition to Bruce's note about bivariate screening I hope Diana isn't pouring over 80+ tables in all these regression calls!
Andy W
apwheele@gmail.com
http://andrewpwheeler.wordpress.com/
Reply | Threaded
Open this post in threaded view
|

Re: Statsitics in a loop

Albert-Jan Roskam
In reply to this post by Bruce Weaver
<snip>

p.s. - I hope I'm not aiding and abetting a bivariate pre-screening
>"fishing" expedition!

I was thinking something similar, although it involved picking cherries. You might end up with 4 models that reach statistical significance, assuming an alpha of 0.05.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Statistics in a loop

Jon K Peck
In reply to this post by Ruben Geert van den Berg
If the variables that should constitute the BY list can be identified by some combination of patterns in the name, measurement levels, etc and you have installed the Python Essentials and the SPSSINC SELECT VARIABLES (Utilities > Define Variable Macro), you can construct the variable list that way and feed it into the !mixlin macro.

E. g.,
SPSSINC SELECT VARIABLES MACRONAME="!predvars"
/PROPERTIES TYPE=NUMERIC LEVEL=NOMINAL ORDINAL PATTERN = "s.*y$"

would create a macro named !predvars listing all numeric categorical variables whose name starts with s and ends with y.


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
phone: 720-342-5621




From:        Ruben Geert van den Berg <[hidden email]>
To:        [hidden email],
Date:        06/03/2013 05:38 AM
Subject:        Re: [SPSSX-L] Statsitics in a loop
Sent by:        "SPSSX(r) Discussion" <[hidden email]>




@Bruce: shouldn't it be

DEFINE !mixlin  (pred = !tokens(1))
MIXED PMscore BY  *!pred*
 /CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1)
SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE)
PCONVERGE(0.000001, ABSOLUTE)
 /FIXED= *!pred* | SSTYPE(3)
 /METHOD=REML
 /REPEATED=*!pred* | SUBJECT(ID) COVTYPE(UN)
 /EMMEANS=TABLES(*!pred*).
!ENDDEFINE.
!mixlin pred = v12.

? I sure hope the 81 variable names can somehow be created in a !DO loop
with a !CONCAT because otherwise you'll need to enter all 81 names into the
macro call =  no fun. If these variables are adjacent in the data, the "TO"
keyword can be used if you'd fix this with Python. Or you could select the
variables given a pattern in their names/variable labels/value
labels/whatever.

HTH,

Ruben



--
View this message in context:
http://spssx-discussion.1045642.n5.nabble.com/Statsitics-in-a-loop-tp5720528p5720530.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


Reply | Threaded
Open this post in threaded view
|

Re: Statsitics in a loop

Ryan
In reply to this post by Bruce Weaver
General statement (not directed at any specific person):
 
Use of mixed models does not eliminate problems associated with bivariate pre-screening. Nor does it eliminate the serious drawbacks associated with any type of automated selection of a (sub)set of predictors. 
 
In general, it's recommended to perform the following steps when employing a linear mixed model:
 
1. Model the mean structure (i.e., the fixed effects). The analyst should strive to include only *valid* predictors in this step. Inclusion of predictors should be based on theory/evidence accumulated from previous well-constructed studies. Much more could be said on this topic...Again, this is not unique to mixed models.
2. Specify an appropriate covariance structure.
3. Fit the mean model using the covariance structure specified in step 2.
4. Make inferences from the model fitted in step 3. (One may consider simplifying the mean model, in a *judicious* way, during this step)
 
Ryan
 
On Mon, Jun 3, 2013 at 7:26 AM, Bruce Weaver <[hidden email]> wrote:
Good morning Diana.  Inside the body of the macro, macro arguments must start
with an exclamation point.  (Also, no need for a command terminator on the
DEFINE line).  E.g.,

DEFINE !mixlin  (pred = !tokens(1))
MIXED PMscore BY  *!pred*
  /CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1)
SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE)
PCONVERGE(0.000001, ABSOLUTE)
  /FIXED= pred | SSTYPE(3)
  /METHOD=REML
  /REPEATED=pred | SUBJECT(ID) COVTYPE(UN)
  /EMMEANS=TABLES(pred).
!ENDDEFINE.
!mixlin pred = v12.

Given what you've said, you might want to hand the macro a LIST of variables
though.

DEFINE !mixlin  (Xlist = !CMDEND)
*!DO !X !IN(!Xlist)*
MIXED PMscore BY  *!X*
  /CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1)
SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE)
PCONVERGE(0.000001, ABSOLUTE)
  /FIXED= pred | SSTYPE(3)
  /METHOD=REML
  /REPEATED=pred | SUBJECT(ID) COVTYPE(UN)
  /EMMEANS=TABLES(pred).
*!DOEND*
!ENDDEFINE.

!mixlin Xlist = V12 V13 V42.

HTH.


p.s. - I hope I'm not aiding and abetting a bivariate pre-screening
"fishing" expedition!




Kornbrot, Diana wrote
> Hi all experts
> I wish to perform various statistics procedures for  a large number of
> variables
> E.g. MIXED linear, so would like to repeat the following syntax for
> 81different variables 'pred'
>
> MIXED PMscore BY  pred
>   /CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1)
> SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE)
> PCONVERGE(0.000001, ABSOLUTE)
>   /FIXED= pred | SSTYPE(3)
>   /METHOD=REML
>   /REPEATED=pred | SUBJECT(ID) COVTYPE(UN)
>   /EMMEANS=TABLES(pred).
>
> Is this possible in SPSS?  Pr does it require python
> Loop and do repeat constructions appear only to take data manipulation &
> transformation commands
> Have attempted to put in a macro, !mixlin, as follows, with intention to
> calling the macro for all 81 variables, if necessary listing them by
> variable name. As a test have just put in the first variable, v12.
>
> DEFINE !mixlin  (pred = !tokens(1)).
> MIXED PMscore BY  pred
>   /CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1)
> SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE)
> PCONVERGE(0.000001, ABSOLUTE)
>   /FIXED= pred | SSTYPE(3)
>   /METHOD=REML
>   /REPEATED=pred | SUBJECT(ID) COVTYPE(UN)
>   /EMMEANS=TABLES(pred).
> !ENDDEFINE.
> !mixlin pred = v12.
>
> Does not work & get following error message
> Text: pred Command: MIXED
> An undefined variable name, or a scratch or system variable was specified
> in a variable list which accepts only standard variables.  Check spelling
> and verify the existence of this variable.
> Execution of this command stops.
>
> v12, as variable name, works fine if substituted for pred in the syntaxof
> the body of the macro
>
> All help gratefully received
>
> Best
>
> Diana
>
>
>
>
>
> ________________________________
> Emeritus Professor Diana Kornbrot
> email:

> d.e.kornbrot@.ac

>  web:    http://dianakornbrot.wordpress.com/
> Work
> Department of Psychology
> School of Life and Medical Sciences
> University of Hertfordshire
> College Lane, Hatfield, Hertfordshire AL10 9AB, UK
> voice:   <a href="tel:%2B44%20%280%29%20170%20728%204626" target="_blank" value="+441707284626">+44 (0) 170 728 4626
> Home
> 19 Elmhurst Avenue
> London N2 0LT, UK
> voice:   <a href="tel:%2B44%20%280%29%20208%20%20444%202081" target="_blank" value="+442084442081">+44 (0) 208 444 2081
> mobile: <a href="tel:%2B44%20%280%29%20740%20318%201612" target="_blank" value="+447403181612">+44 (0) 740 318 1612





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Statsitics-in-a-loop-tp5720528p5720529.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Reply | Threaded
Open this post in threaded view
|

Re: Statsitics in a loop

Kornbrot, Diana
In reply to this post by Andy W
Re: Statsitics in a loop Thanks to all
Guilty as charged
I am comparing the results of using ordinal regression, raw regression and multinomial regression
That is why I need the 81 variables, and I have sorting methods to help me. This is for real data– not simulations of non-normality
As for the fishing expedition .....
  1. will apply false discovery rate algorithms
  2. have priors as to which variables are expected to be active
Vars to cases would work, but think dumping the entire variable list into the syntax will be easier.

NB the define macro now works fine, no more help needed there
Thanks to all who supplied fast help

Best

Diana


On 03/06/2013 12:48, "Andy W" <apwheele@...> wrote:

If mixed works with split files another approach may be to stack the dataset
(VARSTOCASES) with an index, and then just split the file by that index.

In addition to Bruce's note about bivariate screening I hope Diana isn't
pouring over 80+ tables in all these regression calls!



-----
Andy W
apwheele@...
http://andrewpwheeler.wordpress.com/
--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Statsitics-in-a-loop-tp5720528p5720531.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
LISTSERV@... (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD



Emeritus Professor Diana Kornbrot
email:  d.e.kornbrot@...    
 web:    http://dianakornbrot.wordpress.com/
Work
Department of Psychology
School of Life and Medical Sciences
University of Hertfordshire
College Lane, Hatfield, Hertfordshire AL10 9AB, UK
voice:   +44 (0) 170 728 4626
Home
19 Elmhurst Avenue
London N2 0LT, UK
voice:   +44 (0) 208  444 2081
mobile: +44 (0) 740 318 1612


Reply | Threaded
Open this post in threaded view
|

Re: Statsitics in a loop

Bruce Weaver
Administrator
In reply to this post by Ruben Geert van den Berg
Mea culpa.  

Good catch, Ruben.  I was in a hurry to get somewhere, and did not check carefully enough before posting.


Ruben Geert van den Berg wrote
@Bruce: shouldn't it be

DEFINE !mixlin  (pred = !tokens(1))
MIXED PMscore BY  !pred
  /CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE)
  /FIXED= !pred | SSTYPE(3)
  /METHOD=REML
  /REPEATED=!pred | SUBJECT(ID) COVTYPE(UN)
  /EMMEANS=TABLES(!pred).
!ENDDEFINE.
!mixlin pred = v12.

? I sure hope the 81 variable names can somehow be created in a !DO loop with a !CONCAT because otherwise you'll need to enter all 81 names into the macro call =  no fun. If these variables are adjacent in the data, the "TO" keyword can be used if you'd fix this with Python. Or you could select the variables given a pattern in their names/variable labels/value labels/whatever.

HTH,

Ruben
--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

PLEASE NOTE THE FOLLOWING: 
1. My Hotmail account is not monitored regularly. To send me an e-mail, please use the address shown above.
2. The SPSSX Discussion forum on Nabble is no longer linked to the SPSSX-L listserv administered by UGA (https://listserv.uga.edu/).