VARSTOCASES bug?

[gcnau]

Hi all,

I'm new around here and around SPSS in general, so please be nice.
I've discovered that when I try to run VARSTOCASES on an empty set (a skipped section from a survey) I get the number or rows I'm expecting but with rubbish/junk data in it. What's more interesting is that it doesn't happen every time I run the syntax. I have this running daily and only 1 out of 5-6 runs will produce this non-data.

Is this known and is there any workaround for it?

I'll try to post a shortened version of the syntax, currently standing at 3457 lines...

Much appreciated
George

[David Marso]

George,
PLEASE do not post a 3K line syntax file.  
Nobody is going to read it (much or less run it).
Better yet, what ver of SPSS are you using?
Have you applied all relevant patches (if 21 a new fixpack was released testerday).
With a huge 3K+ line syntax file your best option is to divide and conquer.
Chop that B into multiple files and investigate the INSERT command.
Being new to SPSS I suspect you have inherited someone else's code.
Do the right thing and refactor/D&C/Optimize moving forward!
David (me being nice!)
--

gcnau wrote

Hi all,

I'm new around here and around SPSS in general, so please be nice.
I've discovered that when I try to run VARSTOCASES on an empty set (a skipped section from a survey) I get the number or rows I'm expecting but with rubbish/junk data in it. What's more interesting is that it doesn't happen every time I run the syntax. I have this running daily and only 1 out of 5-6 runs will produce this non-data.

Is this known and is there any workaround for it?

I'll try to post a shortened version of the syntax, currently standing at 3457 lines...

Much appreciated
George

[gcnau]

Thanks David!

I will not post 3k lines file, no worries :) I'm using v17 - don't ask why. But I've tested the same syntax in v19 and v20 and sometimes I get the results I don't want (ie junk data). Hopefully soon, we'll move to newest version, but I'm scared of what other bugs will be there :)

To explain a bit better what I'm doing: I have 6 sections pretty much the same and I need to fold them, but before that, each section needs to be folded first. So in total I'm doing 12 VARSTOCASES. Each section has a few thousand variables and it was tested on its own, all works fine.
I think the problem appears from the /INDEX and /COUNT commands, each of the VARSTOCASES have them, but I'm not 100% sure. One other problem is that I can't replicate it, it seems to depend on the memory load or something like that, the junk data I'm getting looks like memory dump. Would using INSERT commands free up memory? Because it will run pretty much the same number of lines, and as far as I know SPSS is single threaded.

I'll keep digging

Cheers
George

[David Marso]

Hi George,
Maybe see /NULL=DROP subcommand on VARSTOCASES (V2C).
or
only retain the needed variables at a particular instance of V2C (see KEEP and DROP on GET FILE).
Rather difficult to provide more input.
Re: Divide and Conquer and INSERT.
That was advice directed more at goals of preservation of sanity and hair retention ;-)

gcnau wrote

Thanks David!

I will not post 3k lines file, no worries :) I'm using v17 - don't ask why. But I've tested the same syntax in v19 and v20 and sometimes I get the results I don't want (ie junk data). Hopefully soon, we'll move to newest version, but I'm scared of what other bugs will be there :)

To explain a bit better what I'm doing: I have 6 sections pretty much the same and I need to fold them, but before that, each section needs to be folded first. So in total I'm doing 12 VARSTOCASES. Each section has a few thousand variables and it was tested on its own, all works fine.
I think the problem appears from the /INDEX and /COUNT commands, each of the VARSTOCASES have them, but I'm not 100% sure. One other problem is that I can't replicate it, it seems to depend on the memory load or something like that, the junk data I'm getting looks like memory dump. Would using INSERT commands free up memory? Because it will run pretty much the same number of lines, and as far as I know SPSS is single threaded.

I'll keep digging

Cheers
George