Number of cases restriction for Chart Builder

[Hector Maletta]

I tried to obtain a simple 2-variable scatterplot with GGRAPH and a data set
of about 2 million cases, though the number of observed value combinations
was more limited (about 3500). SPSS refused to run the procedure with an
error message indicating that the number of cases was restricted to one
million in the graph data set. Is this related to my available workspace?
Can this restriction be eased?

Hector

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Hortensia Carrillo]

Please, I need help: if you are doing a questionary and are trying to know how many factors you can have, in a Factor Analisys: you must see the factors as correlated o as independent?.
Thanks.
_________________________________________________________________
News, entertainment and everything you care about at Live.com. Get it now!
http://www.live.com/getstarted.aspx
====================To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Hector Maletta]

Jon,
thanks for the rapid feedback. Of course I don't want 2 million point
symbols in the chart. The limit is one million cases, but I do not want one
million points either. In fact,  in my 2 million cases dataset there are
only 3500 distinct value combinations (with different frequencies each).
Good advice about hex binning, but I don't have v.17 as yet. How do I manage
it with 15 or 16?

Hector

  _____

From: Peck, Jon [mailto:[hidden email]]
Sent: 16 November 2008 20:25
To: Hector Maletta
Subject: RE: [SPSSX-L] Number of cases restriction for Chart Builder



I don't think you would want to wait for the graphics engine to draw two
million point symbols in a chart.  And the result might well be useless.



Perhaps you should try doing the scatter with hex binning (hex.bin).  You
can't specify that directly in Chart Builder, though.  You would need to use
GPL to get it.



If you have 17, you could try the GraphBoard Binned Scatterplot.  I don't
know whether it has the same case limit, though.

HTH,

Jon Peck



-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Hector Maletta
Sent: Sunday, November 16, 2008 2:03 PM
To: [hidden email]
Subject: [SPSSX-L] Number of cases restriction for Chart Builder



I tried to obtain a simple 2-variable scatterplot with GGRAPH and a data set

of about 2 million cases, though the number of observed value combinations

was more limited (about 3500). SPSS refused to run the procedure with an

error message indicating that the number of cases was restricted to one

million in the graph data set. Is this related to my available workspace?

Can this restriction be eased?



Hector



=====================

To manage your subscription to SPSSX-L, send a message to

[hidden email] (not to SPSSX-L), with no body text except the

command. To leave the list, send the command

SIGNOFF SPSSX-L

For a list of commands to manage subscriptions, send the command

INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Lemon, John S.]

Hector

When I'm presented with problems like that I always aggregate the file and then plot the aggregated file it looks the same whether there are 5 million or 500 points !! in fact in many cases it looks better !!

Best Wishes

John S. Lemon
DIT ( Directorate of Information Technology ) - Student Liaison Officer
University of Aberdeen
Edward Wright Building: Room G51

Tel:  [phone]
Fax: [phone]

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Hector Maletta
Sent: 17 November 2008 02:07
To: [hidden email]
Subject: Re: Number of cases restriction for Chart Builder

Jon,
thanks for the rapid feedback. Of course I don't want 2 million point
symbols in the chart. The limit is one million cases, but I do not want one
million points either. In fact,  in my 2 million cases dataset there are
only 3500 distinct value combinations (with different frequencies each).
Good advice about hex binning, but I don't have v.17 as yet. How do I manage
it with 15 or 16?

Hector

  _____

From: Peck, Jon [mailto:[hidden email]]
Sent: 16 November 2008 20:25
To: Hector Maletta
Subject: RE: [SPSSX-L] Number of cases restriction for Chart Builder



I don't think you would want to wait for the graphics engine to draw two
million point symbols in a chart.  And the result might well be useless.



Perhaps you should try doing the scatter with hex binning (hex.bin).  You
can't specify that directly in Chart Builder, though.  You would need to use
GPL to get it.



If you have 17, you could try the GraphBoard Binned Scatterplot.  I don't
know whether it has the same case limit, though.

HTH,

Jon Peck



-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Hector Maletta
Sent: Sunday, November 16, 2008 2:03 PM
To: [hidden email]
Subject: [SPSSX-L] Number of cases restriction for Chart Builder



I tried to obtain a simple 2-variable scatterplot with GGRAPH and a data set

of about 2 million cases, though the number of observed value combinations

was more limited (about 3500). SPSS refused to run the procedure with an

error message indicating that the number of cases was restricted to one

million in the graph data set. Is this related to my available workspace?

Can this restriction be eased?



Hector



=====================

To manage your subscription to SPSSX-L, send a message to

[hidden email] (not to SPSSX-L), with no body text except the

command. To leave the list, send the command

SIGNOFF SPSSX-L

For a list of commands to manage subscriptions, send the command

INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


The University of Aberdeen is a charity registered in Scotland, No SC013683.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[ViAnn Beadle]

Hex binning does this with the added option of actually providing a
mini-heatmap of the overlapping points. Here's a simple example of hex
binning.

GGRAPH
  /GRAPHDATASET NAME="graphdataset" VARIABLES=salbegin salary
MISSING=LISTWISE REPORTMISSING=NO
  /GRAPHSPEC SOURCE=INLINE.
BEGIN GPL
  SOURCE: s=userSource(id("graphdataset"))
  DATA: salbegin=col(source(s), name("salbegin"))
  DATA: salary=col(source(s), name("salary"))
  GUIDE: axis(dim(1), label("Beginning Salary"))
  GUIDE: axis(dim(2), label("Current Salary"))
GUIDE: legend(aesthetic(aesthetic.color), label("Count"))
ELEMENT: polygon(position(bin.hex(salbegin*salary, dim(1,2),
binWidth(4000,4000))),
                 color(summary.count()))
END GPL.

Note that this is derived from an example in the GPL help. I've added the
binWidth function to control the x and y bin widths.

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Lemon, John S.
Sent: Monday, November 17, 2008 12:40 AM
To: [hidden email]
Subject: Re: Number of cases restriction for Chart Builder

Hector

When I'm presented with problems like that I always aggregate the file and
then plot the aggregated file it looks the same whether there are 5 million
or 500 points !! in fact in many cases it looks better !!

Best Wishes

John S. Lemon
DIT ( Directorate of Information Technology ) - Student Liaison Officer
University of Aberdeen
Edward Wright Building: Room G51

Tel:  [phone]
Fax: [phone]

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Hector Maletta
Sent: 17 November 2008 02:07
To: [hidden email]
Subject: Re: Number of cases restriction for Chart Builder

Jon,
thanks for the rapid feedback. Of course I don't want 2 million point
symbols in the chart. The limit is one million cases, but I do not want one
million points either. In fact,  in my 2 million cases dataset there are
only 3500 distinct value combinations (with different frequencies each).
Good advice about hex binning, but I don't have v.17 as yet. How do I manage
it with 15 or 16?

Hector

  _____

From: Peck, Jon [mailto:[hidden email]]
Sent: 16 November 2008 20:25
To: Hector Maletta
Subject: RE: [SPSSX-L] Number of cases restriction for Chart Builder



I don't think you would want to wait for the graphics engine to draw two
million point symbols in a chart.  And the result might well be useless.



Perhaps you should try doing the scatter with hex binning (hex.bin).  You
can't specify that directly in Chart Builder, though.  You would need to use
GPL to get it.



If you have 17, you could try the GraphBoard Binned Scatterplot.  I don't
know whether it has the same case limit, though.

HTH,

Jon Peck



-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Hector Maletta
Sent: Sunday, November 16, 2008 2:03 PM
To: [hidden email]
Subject: [SPSSX-L] Number of cases restriction for Chart Builder



I tried to obtain a simple 2-variable scatterplot with GGRAPH and a data set

of about 2 million cases, though the number of observed value combinations

was more limited (about 3500). SPSS refused to run the procedure with an

error message indicating that the number of cases was restricted to one

million in the graph data set. Is this related to my available workspace?

Can this restriction be eased?



Hector



=====================

To manage your subscription to SPSSX-L, send a message to

[hidden email] (not to SPSSX-L), with no body text except the

command. To leave the list, send the command

SIGNOFF SPSSX-L

For a list of commands to manage subscriptions, send the command

INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


The University of Aberdeen is a charity registered in Scotland, No SC013683.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Hector Maletta]

Thanks, Ray, Jon and others that sent help on this issue. The aggregate +
weight solution works fine with ordinary scatterplots, and also hex binned
scatterplots in recent versions.



Hector





---------- Forwarded message ----------
From: Raynald Levesque <[hidden email]>
Date: Sun, Nov 16, 2008 at 5:29 PM
Subject: Re: Number of cases restriction for Chart Builder
To: [hidden email]


Hi Hector,

The 1 million cases is based on the number of unweighted cases.
So an easy way to avoid the limit is to aggregate data using the 2 variables
as BREAK then weight the file by N.
The following example has 2 million cases (but less than 4,000 unique
combination). The scatter plot is produced without problems.

Note that variable n could be used to modify the size of each circle and
thus illustrate the number of cases in each combination.

NEW FILE.
INPUT PROGRAM.
LOOP id=1 TO 2000000.
  COMPUTE var1=RND(UNIFORM(60)).
  COMPUTE var2=RND(UNIFORM(60)).
  END CASE.
END LOOP.
END FILE.
END INPUT PROGRAM.
AGGREGATE OUTFILE=*
   /BREAK=var1 var2
   /n=N.
WEIGHT BY n.
SHOW N.

* Chart Builder.
GGRAPH
  /GRAPHDATASET NAME="graphdataset" VARIABLES=var1 var2 MISSING=LISTWISE
REPORTMISSING=NO
  /GRAPHSPEC SOURCE=INLINE.
BEGIN GPL
  SOURCE: s=userSource(id("graphdataset"))
  DATA: var1=col(source(s), name("var1"))
  DATA: var2=col(source(s), name("var2"))
  GUIDE: axis(dim(1), label("var1"))
  GUIDE: axis(dim(2), label("var2"))
  ELEMENT: point(position(var1*var2))
END GPL.

Regards

Ray






On Sun, Nov 16, 2008 at 4:02 PM, Hector Maletta <[hidden email]>
wrote:

I tried to obtain a simple 2-variable scatterplot with GGRAPH and a data set
of about 2 million cases, though the number of observed value combinations
was more limited (about 3500). SPSS refused to run the procedure with an
error message indicating that the number of cases was restricted to one
million in the graph data set. Is this related to my available workspace?
Can this restriction be eased?

Hector

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD





--
Raynald Levesque
www.spsstools.net <http://www.spsstools.net/>




--
Raynald Levesque
www.spsstools.net

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD