aggregate outfile=* with no dataset activated very occasionally crashes in production mode

[Simon Phillip Freidin]

I'll document this very occasional bug in case it helps someone else.

The random halts in the SPSS parts of my production system were disconcerting and took a long time to track down.

 

Affected systems: SPSS for Windows Versions 14 to 20 in production mode.

 

Symptoms

========

Executing production jobs randomly halt.

Most commonly the job suspends without logging.

Under some circumstances it halts with the error message "JVMDUMP006I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" - please wait." etc and writes snap, headdump and javacore files.

Either way it remains as an executing but inactive process until it is force ended.

 

Affected commands

=================

aggregate outfile=*/breakvar=stringID ...

 

aggregate outfile=* mode=replace/breakvar=stringID ...

 

I can't get numeric breakvars to fail but SPSS support says this also halts on their systems

aggregate outfile=*/breakvar=numericID ... 

 

Unaffected commands

===================

/* mode=addvar does not halt, but this does not reduce the number of cases in the file */

aggregate outfile=* mode=addvar/breakvar=stringID ...     

 

/* my workaround: write aggregated dataset to disk, then get file */

aggregate outfile='c:\file location'/breakvar=stringID ...

get file='c:\file location'.

 

If you have enough aggregate outfile=* commands it will look like the production jobs are halting erratically due to some system-wide problem and it won't be clear if it is occurring in hardware (memory/disk/CPU/network) or software (Windows/SPSS).

 

SPSS Supports's resolution/workaround (PMR 40070 999 616) is to name and explicitly activate a dataset before the aggregate command.

dataset name agg01.

dataset activate agg01.

aggregate outfile=*/breakvar=ID

 

The official explanation is

From the Command Syntax Reference "OUTFILE=* with no additional keywords on the OUTFILE subcommand will replace the active dataset with the aggregated results."                

From the PMR: "The key word from that description is 'ACTIVE' dataset. Since this was not defined beforehand, Statistics encountered difficulty in associating the (currently open) file with the AGGREGATE OUTFILE=* subcommand, hence the crashes."

If "aggregate without a named dataset" fails around once in every 250-350 calls this would accord with the behaviour I saw in my production processes.

[Jarrod Teo-2]

Hi Simon,
 
That is an interesting observation. Thanks for the info and I will take note of this the next time I am working with a production job.
 
If I am not wrong, you are trying to use production job mode for the PC's task scheduler to produce reports on a timely basis right?
 
I will strongly advise not to use * in other files commands then since aggregate outfile=* is causing this issue. In my case, I would create a new folder call processing file folder to save all the processing data files. I am not sure if this will cause the whole automation process to slow down but at least it will keep your project's data folder neater without those processing data files messing up the data folder. I know this is not the best way but at least it might also not cause the production jobs to randomly halt.
 
Warmest Regards
Dorraj Oet
 

---

Date: Fri, 13 Jul 2012 06:48:20 +0000
From: [hidden email]
Subject: aggregate outfile=* with no dataset activated very occasionally crashes in production mode
To: [hidden email]

I'll document this very occasional bug in case it helps someone else.

The random halts in the SPSS parts of my production system were disconcerting and took a long time to track down.

 

Affected systems: SPSS for Windows Versions 14 to 20 in production mode.

 

Symptoms

========

Executing production jobs randomly halt.

Most commonly the job suspends without logging.

Under some circumstances it halts with the error message "JVMDUMP006I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" - please wait." etc and writes snap, headdump and javacore files.

Either way it remains as an executing but inactive process until it is force ended.

 

Affected commands

=================

aggregate outfile=*/breakvar=stringID ...

 

aggregate outfile=* mode=replace/breakvar=stringID ...

 

I can't get numeric breakvars to fail but SPSS support says this also halts on their systems

aggregate outfile=*/breakvar=numericID ... 

 

Unaffected commands

===================

/* mode=addvar does not halt, but this does not reduce the number of cases in the file */

aggregate outfile=* mode=addvar/breakvar=stringID ...     

 

/* my workaround: write aggregated dataset to disk, then get file */

aggregate outfile='c:\file location'/breakvar=stringID ...

get file='c:\file location'.

 

If you have enough aggregate outfile=* commands it will look like the production jobs are halting erratically due to some system-wide problem and it won't be clear if it is occurring in hardware (memory/disk/CPU/network) or software (Windows/SPSS).

 

SPSS Supports's resolution/workaround (PMR 40070 999 616) is to name and explicitly activate a dataset before the aggregate command.

dataset name agg01.

dataset activate agg01.

aggregate outfile=*/breakvar=ID

 

The official explanation is

From the Command Syntax Reference "OUTFILE=* with no additional keywords on the OUTFILE subcommand will replace the active dataset with the aggregated results."                

From the PMR: "The key word from that description is 'ACTIVE' dataset. Since this was not defined beforehand, Statistics encountered difficulty in associating the (currently open) file with the AGGREGATE OUTFILE=* subcommand, hence the crashes."

If "aggregate without a named dataset" fails around once in every 250-350 calls this would accord with the behaviour I saw in my production processes.

[Albert-Jan Roskam]

Hi Simon,

 

Are you 100 % sure it is related to aggregate outfile? I am asking because I reported a Production Facility bug before and I mistakenly linked it to using INSERT /INCLUDE to a non-existing file.

I pasted the IBM report below. It is still under investigation (SPSS v21 or FP2 do not yet exist).

 

Regards,
Albert-Jan

 

APAR status

• OPEN

Error description

• It was thought that from time to time production jobs fail with
  java errors when the command syntax file underlying the *.spj,
  command syntax inserted/included, or data files opened with GET
  at one of the INSERT/INCLUDE levels, do not exist at the
  specified path.  But upon further inspection, it seems that even
  correctly specified paths end in cryptic java traces from time
  to time.
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * Users of IBM SPSS Statistics running multiple Production     *
  * Facility jobs from command line.                             *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * From time to time, correctly specified Production Facility   *
  * command line jobs would end in cryptic java traces without   *
  * the job completing.                                          *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Please apply Statistics 20, Fixpack 2 or upgrade to          *
  * Statistics 21.                                               *
  ****************************************************************
  

 

 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a
fresh water system, and public health, what have the Romans ever done for us?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

From: Simon Phillip Freidin <[hidden email]>
To: [hidden email]
Sent: Friday, July 13, 2012 8:48 AM
Subject: [SPSSX-L] aggregate outfile=* with no dataset activated very occasionally crashes in production mode

I'll document this very occasional bug in case it helps someone else.

The random halts in the SPSS parts of my production system were disconcerting and took a long time to track down.

 

Affected systems: SPSS for Windows Versions 14 to 20 in production mode.

 

Symptoms

========

Executing production jobs randomly halt.

Most commonly the job suspends without logging.

Under some circumstances it halts with the error message "JVMDUMP006I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" - please wait." etc and writes snap, headdump and javacore files.

Either way it remains as an executing but inactive process until it is force ended.

 

Affected commands

=================

aggregate outfile=*/breakvar=stringID ...

 

aggregate outfile=* mode=replace/breakvar=stringID ...

 

I can't get numeric breakvars to fail but SPSS support says this also halts on their systems

aggregate outfile=*/breakvar=numericID ... 

 

Unaffected commands

===================

/* mode=addvar does not halt, but this does not reduce the number of cases in the file */

aggregate outfile=* mode=addvar/breakvar=stringID ...     

 

/* my workaround: write aggregated dataset to disk, then get file */

aggregate outfile='c:\file location'/breakvar=stringID ...

get file='c:\file location'.

 

If you have enough aggregate outfile=* commands it will look like the production jobs are halting erratically due to some system-wide problem and it won't be clear if it is occurring in hardware (memory/disk/CPU/network) or software (Windows/SPSS).

 

SPSS Supports's resolution/workaround (PMR 40070 999 616) is to name and explicitly activate a dataset before the aggregate command.

dataset name agg01.

dataset activate agg01.

aggregate outfile=*/breakvar=ID

 

The official explanation is

From the Command Syntax Reference "OUTFILE=* with no additional keywords on the OUTFILE subcommand will replace the active dataset with the aggregated results."                

From the PMR: "The key word from that description is 'ACTIVE' dataset. Since this was not defined beforehand, Statistics encountered difficulty in associating the (currently open) file with the AGGREGATE OUTFILE=* subcommand, hence the crashes."

If "aggregate without a named dataset" fails around once in every 250-350 calls this would accord with the behaviour I saw in my production processes.

[Jon K Peck]

A dataset does not necessarily have a file name and location associated with it.  It is better not to wire in specific locations as it makes the job dependent on a particular directory structure.  Just declaring and using a dataset name would be sufficient.

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
new phone: [phone]




From:        DorraJ Oet <[hidden email]>
To:        [hidden email]
Date:        07/13/2012 01:13 AM
Subject:        Re: [SPSSX-L] aggregate outfile=* with no dataset activated very              occasionally crashes in production mode
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

Hi Simon,

That is an interesting observation. Thanks for the info and I will take note of this the next time I am working with a production job.

If I am not wrong, you are trying to use production job mode for the PC's task scheduler to produce reports on a timely basis right?

I will strongly advise not to use * in other files commands then since aggregate outfile=* is causing this issue. In my case, I would create a new folder call processing file folder to save all the processing data files. I am not sure if this will cause the whole automation process to slow down but at least it will keep your project's data folder neater without those processing data files messing up the data folder. I know this is not the best way but at least it might also not cause the production jobs to randomly halt.

Warmest Regards
Dorraj Oet

---

Date: Fri, 13 Jul 2012 06:48:20 +0000
From: [hidden email]
Subject: aggregate outfile=* with no dataset activated very occasionally crashes in production mode
To: [hidden email]


I'll document this very occasional bug in case it helps someone else.
The random halts in the SPSS parts of my production system were disconcerting and took a long time to track down.
 
Affected systems: SPSS for Windows Versions 14 to 20 in production mode.
 
Symptoms
========
Executing production jobs randomly halt.
Most commonly the job suspends without logging.
Under some circumstances it halts with the error message "JVMDUMP006I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" - please wait." etc and writes snap, headdump and javacore files.
Either way it remains as an executing but inactive process until it is force ended.
 
Affected commands
=================
aggregate outfile=*/breakvar=stringID ...
 
aggregate outfile=* mode=replace/breakvar=stringID ...
 
I can't get numeric breakvars to fail but SPSS support says this also halts on their systems
aggregate outfile=*/breakvar=numericID ...  
 
Unaffected commands
===================
/* mode=addvar does not halt, but this does not reduce the number of cases in the file */
aggregate outfile=* mode=addvar/breakvar=stringID ...      
 
/* my workaround: write aggregated dataset to disk, then get file */
aggregate outfile='c:\file location'/breakvar=stringID ...
get file='c:\file location'.
 
If you have enough aggregate outfile=* commands it will look like the production jobs are halting erratically due to some system-wide problem and it won't be clear if it is occurring in hardware (memory/disk/CPU/network) or software (Windows/SPSS).
 
SPSS Supports's resolution/workaround (PMR 40070 999 616) is to name and explicitly activate a dataset before the aggregate command.
dataset name agg01.
dataset activate agg01.
aggregate outfile=*/breakvar=ID
 
The official explanation is
From the Command Syntax Reference "OUTFILE=* with no additional keywords on the OUTFILE subcommand will replace the active dataset with the aggregated results."                
From the PMR: "The key word from that description is 'ACTIVE' dataset. Since this was not defined beforehand, Statistics encountered difficulty in associating the (currently open) file with the AGGREGATE OUTFILE=* subcommand, hence the crashes."
If "aggregate without a named dataset" fails around once in every 250-350 calls this would accord with the behaviour I saw in my production processes.