SET SORT = EXTERNAL

[Albert-Jan Roskam]

Hi,

 

Has anybody successfully used an external sorting mechanism? What characteristics should such a mechanism have in order for spss to be able to use it? When would an external algorithm offer advantages? (a quicksort for nearly-sorted data?)

 

SORT Subcommand
By default, IBM® SPSS® Statistics tries to use an external, third-party sorting mechanism, which
may reduce processing time with large data sources. The third-party sorting option is available
only if you have SPSS Statistics Server. The specific sorting engine is defined by your server
administrator. If you are not connected to the SPSS Statistics Server or the SPSS Statistics Server
cannot find the third-party sort engine, the built-in sorting mechanism is used.
EXTERNAL Use the external, third-party sort engine if available. This setting is the default. If
the third-party sort engine is not available, this setting is ignored, and the built-in
sorting mechanism is used. COXREG and CURVEFIT use built-in sorting regardless
of the SORT setting.
INTERNAL Use the built-in sorting mechanism.
SS This setting is deprecated. It has the same effect as EXTERNAL.

 

Regards,
Albert-Jan

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a
fresh water system, and public health, what have the Romans ever done for us?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

[Jon K Peck]

The original inspiration for an external sort had SYNCSORT in mind.  That is a product really dedicated specifically to high performance sorting.  Quicksort for nearly sorted data is not an example, though, where this would apply, since real QS implementations, including SPSS, deal with the potential bad problems there.  But SYNCSORT plays a lot of other tricks with multiprocessing and other tricks.

So the bottom line is if sorting in SPSS is taking a really long time, consider SYNCSORT et al.

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
new phone: [phone]




From:        Albert-Jan Roskam <[hidden email]>
To:        [hidden email]
Date:        08/15/2012 03:00 AM
Subject:        [SPSSX-L] SET SORT = EXTERNAL
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

Hi,
 
Has anybody successfully used an external sorting mechanism? What characteristics should such a mechanism have in order for spss to be able to use it? When would an external algorithm offer advantages? (a quicksort for nearly-sorted data?)
 
SORT Subcommand
By default, IBM® SPSS® Statistics tries to use an external, third-party sorting mechanism, which
may reduce processing time with large data sources. The third-party sorting option is available
only if you have SPSS Statistics Server. The specific sorting engine is defined by your server
administrator. If you are not connected to the SPSS Statistics Server or the SPSS Statistics Server
cannot find the third-party sort engine, the built-in sorting mechanism is used.
EXTERNAL Use the external, third-party sort engine if available. This setting is the default. If
the third-party sort engine is not available, this setting is ignored, and the built-in
sorting mechanism is used. COXREG and CURVEFIT use built-in sorting regardless
of the SORT setting.
INTERNAL Use the built-in sorting mechanism.
SS This setting is deprecated. It has the same effect as EXTERNAL.
 
Regards,
Albert-Jan


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a
fresh water system, and public health, what have the Romans ever done for us?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~