number of up front in a file

[Ornelas, Fermin]

I am asking SPSS to tell me how many records is bringing in but have not
had no luck.

 

GET FILE= IN1

SHOW N

 

 

 The answer

 

N= UNKNOWN

 

For comparison I ran the job in SAS and the number of records brought is
usually in the log. No bashing intended, but I think  there needs to be
more documentation in the steps taken when processing data. Especially
when the data sets are large.

 

 

Fermin Ornelas
   
NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR CONFIDENTIAL information and is intended only for the use of the specific individual(s) to whom it is addressed.  It may contain information that is privileged and confidential under state and federal law.  This information may be used or disclosed only in accordance with law, and you may be subject to penalties under law for improper use or further disclosure of the information in this e-mail and its attachments. If you have received this e-mail in error, please immediately notify the person named above by reply e-mail, and then delete the original e-mail.  Thank you.

[Melissa Ives]

Per the SPSS help: N displays UNKNOWN if a active dataset has not yet
been created.
Check to see that the file was actually opened--usually a file needs to
be identified with the full pathname and be surrounded by quotes.

GET FILE='C:\mypath\IN1.sav'.


Melissa

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
Ornelas, Fermin
Sent: Friday, September 14, 2007 12:40 PM
To: [hidden email]
Subject: [SPSSX-L] number of up front in a file

I am asking SPSS to tell me how many records is bringing in but have not
had no luck.



GET FILE= IN1

SHOW N





 The answer



N= UNKNOWN



For comparison I ran the job in SAS and the number of records brought is
usually in the log. No bashing intended, but I think  there needs to be
more documentation in the steps taken when processing data. Especially
when the data sets are large.





Fermin Ornelas

NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR
CONFIDENTIAL information and is intended only for the use of the
specific individual(s) to whom it is addressed.  It may contain
information that is privileged and confidential under state and federal
law.  This information may be used or disclosed only in accordance with
law, and you may be subject to penalties under law for improper use or
further disclosure of the information in this e-mail and its
attachments. If you have received this e-mail in error, please
immediately notify the person named above by reply e-mail, and then
delete the original e-mail.  Thank you.


PRIVILEGED AND CONFIDENTIAL INFORMATION
This transmittal and any attachments may contain PRIVILEGED AND
CONFIDENTIAL information and is intended only for the use of the
addressee. If you are not the designated recipient, or an employee
or agent authorized to deliver such transmittals to the designated
recipient, you are hereby notified that any dissemination,
copying or publication of this transmittal is strictly prohibited. If
you have received this transmittal in error, please notify us
immediately by replying to the sender and delete this copy from your
system. You may also call us at [phone] for assistance.

[Richard Ristow]

At 01:39 PM 9/14/2007, Ornelas, Fermin wrote:

>I am asking SPSS to tell me how many records is bringing in but have
>had no luck.
>
>GET FILE= IN1
>SHOW N
>
>  The answer
>N= UNKNOWN

Hmmmmm. Melissa may be right, that the file wasn't properly or
completely opened. The following is SPSS 15 draft output (WRR:not saved
separately). The listed commands were the only ones run following
startup of SPSS:

GET
   FILE='C:\Documents and Settings\Richard\My Documents' +
        '\Professional\SCEC\SAV files\TDS March.sav'.
SHOW N.

SHOW
|----------------------------|---------------------------|
|Output Created              |14-SEP-2007 18:55:42       |
|----------------------------|---------------------------|
C:\Documents and Settings\Richard\My Documents
   \Professional\SCEC\SAV files\TDS March.sav

System Settings
|-------|---------------|-------|
|Keyword|Description    |Setting|
|-------|---------------|-------|
|N      |Number of cases|65     |
|       |in the working |       |
|       |data file      |       |
|-------|---------------|-------|



>For comparison I ran the job in SAS and the number of records brought
>is usually in the log. No bashing intended, but I think there needs to
>be more documentation in the steps taken when processing data.
>Especially when the data sets are large.

SAS is more "talkative" about processing statistics, though I'm not
sure just how much - I haven't used SAS in quite a while, and can't lay
hands on any of my old SAS code offhand. However, in this case it
doesn't look like SPSS does badly. The number of cases is apparently in
the file attributes; at least, notice that the number of cases is given
correctly, although there hasn't been a data pass.

SPSS's design of holding transformations and other processing until a
data pass is needed can produce paradoxes if you're not clear what's
happening. Note, in the following, that after the SELECT IF is issued
but before the next data pass, the number of cases appears unchanged.
(And I'm embarrassed every time I post an "EXECUTE"; but, this time,
it's necessary to force a data pass to show the effect.) Again, SPSS
draft output (not saved separately):

GET
   FILE='C:\Documents and Settings\Richard\My Documents' +
        '\Professional\SCEC\SAV files\TDS March.sav'.
SHOW N.

SHOW
|----------------------------|---------------------------|
|Output Created              |14-SEP-2007 19:05:32       |
|----------------------------|---------------------------|
C:\Documents and Settings\Richard\My Documents
   \Professional\SCEC\SAV files\TDS March.sav

System Settings
|-------|---------------|-------|
|Keyword|Description    |Setting|
|-------|---------------|-------|
|N      |Number of cases|65     |
|       |in the working |       |
|       |data file      |       |
|-------|---------------|-------|


SELECT IF NOT MISSING(team).
SHOW N.

SHOW
|----------------------------|---------------------------|
|Output Created              |14-SEP-2007 19:05:32       |
|----------------------------|---------------------------|
C:\Documents and Settings\Richard\My Documents
   \Professional\SCEC\SAV files\TDS March.sav

System Settings
|-------|---------------|-------|
|Keyword|Description    |Setting|
|-------|---------------|-------|
|N      |Number of cases|65     |
|       |in the working |       |
|       |data file      |       |
|-------|---------------|-------|


EXECUTE.
SHOW N.

SHOW
|-----------------------------|---------------------------|
|Output Created               |14-SEP-2007 19:05:32       |
|-----------------------------|---------------------------|
C:\Documents and Settings\Richard\My Documents
   \Professional\SCEC\SAV files\TDS March.sav

System Settings
|-------|---------------|-------|
|Keyword|Description    |Setting|
|-------|---------------|-------|
|N      |Number of cases|32     |
|       |in the working |       |
|       |data file      |       |
|-------|---------------|-------|

[Albert-Jan Roskam]

I get that message sometimes when large files aren't
completely cached. So yep, I also think that the file
is rather large and isn't completely opened yet. Maybe
spss can't read the header info when the file isn't
entirely read yet?

Albert-Jan

--- Richard Ristow <[hidden email]> wrote:

> At 01:39 PM 9/14/2007, Ornelas, Fermin wrote:
>
> >I am asking SPSS to tell me how many records is
> bringing in but have
> >had no luck.
> >
> >GET FILE= IN1
> >SHOW N
> >
> >  The answer
> >N= UNKNOWN
>
> Hmmmmm. Melissa may be right, that the file wasn't
> properly or
> completely opened. The following is SPSS 15 draft
> output (WRR:not saved
> separately). The listed commands were the only ones
> run following
> startup of SPSS:
>
> GET
>    FILE='C:\Documents and Settings\Richard\My
> Documents' +
>         '\Professional\SCEC\SAV files\TDS
> March.sav'.
> SHOW N.
>
> SHOW
>

|----------------------------|---------------------------|
> |Output Created              |14-SEP-2007 18:55:42
>     |
>
|----------------------------|---------------------------|

> C:\Documents and Settings\Richard\My Documents
>    \Professional\SCEC\SAV files\TDS March.sav
>
> System Settings
> |-------|---------------|-------|
> |Keyword|Description    |Setting|
> |-------|---------------|-------|
> |N      |Number of cases|65     |
> |       |in the working |       |
> |       |data file      |       |
> |-------|---------------|-------|
>
>
>
> >For comparison I ran the job in SAS and the number
> of records brought
> >is usually in the log. No bashing intended, but I
> think there needs to
> >be more documentation in the steps taken when
> processing data.
> >Especially when the data sets are large.
>
> SAS is more "talkative" about processing statistics,
> though I'm not
> sure just how much - I haven't used SAS in quite a
> while, and can't lay
> hands on any of my old SAS code offhand. However, in
> this case it
> doesn't look like SPSS does badly. The number of
> cases is apparently in
> the file attributes; at least, notice that the
> number of cases is given
> correctly, although there hasn't been a data pass.
>
> SPSS's design of holding transformations and other
> processing until a
> data pass is needed can produce paradoxes if you're
> not clear what's
> happening. Note, in the following, that after the
> SELECT IF is issued
> but before the next data pass, the number of cases
> appears unchanged.
> (And I'm embarrassed every time I post an "EXECUTE";
> but, this time,
> it's necessary to force a data pass to show the
> effect.) Again, SPSS
> draft output (not saved separately):
>
> GET
>    FILE='C:\Documents and Settings\Richard\My
> Documents' +
>         '\Professional\SCEC\SAV files\TDS
> March.sav'.
> SHOW N.
>
> SHOW
>

|----------------------------|---------------------------|
> |Output Created              |14-SEP-2007 19:05:32
>     |
>
|----------------------------|---------------------------|

> C:\Documents and Settings\Richard\My Documents
>    \Professional\SCEC\SAV files\TDS March.sav
>
> System Settings
> |-------|---------------|-------|
> |Keyword|Description    |Setting|
> |-------|---------------|-------|
> |N      |Number of cases|65     |
> |       |in the working |       |
> |       |data file      |       |
> |-------|---------------|-------|
>
>
> SELECT IF NOT MISSING(team).
> SHOW N.
>
> SHOW
>

|----------------------------|---------------------------|
> |Output Created              |14-SEP-2007 19:05:32
>     |
>
|----------------------------|---------------------------|

> C:\Documents and Settings\Richard\My Documents
>    \Professional\SCEC\SAV files\TDS March.sav
>
> System Settings
> |-------|---------------|-------|
> |Keyword|Description    |Setting|
> |-------|---------------|-------|
> |N      |Number of cases|65     |
> |       |in the working |       |
> |       |data file      |       |
> |-------|---------------|-------|
>
>
> EXECUTE.
> SHOW N.
>
> SHOW
>

|-----------------------------|---------------------------|
> |Output Created               |14-SEP-2007 19:05:32
>      |
>
|-----------------------------|---------------------------|

> C:\Documents and Settings\Richard\My Documents
>    \Professional\SCEC\SAV files\TDS March.sav
>
> System Settings
> |-------|---------------|-------|
> |Keyword|Description    |Setting|
> |-------|---------------|-------|
> |N      |Number of cases|32     |
> |       |in the working |       |
> |       |data file      |       |
> |-------|---------------|-------|
>

Cheers!
Albert-Jan

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Did you know that 87.166253% of all statistics claim a precision of results that is not justified by the method employed? [HELMUT RICHTER]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


      ____________________________________________________________________________________
Tonight's top picks. What will you watch tonight? Preview the hottest shows on Yahoo! TV.
http://tv.yahoo.com/

[Richard Ristow]

At 01:12 PM 9/15/2007, Albert-jan Roskam wrote:

>I get that message sometimes when large files aren't completely
>cached. So yep, I also think that the file is rather large and isn't
>completely opened yet. Maybe spss can't read the header info when the
>file isn't entirely read yet?

It's a good question. It worked fine with the largest one I have handy,
which is 49 megabytes and 623,000 records, but that isn't huge these
days.

On the other hand, SPSS has to read most of the header info
immediately. The largest part of this is the data dictionary, and SPSS
can't process a transformation program without access to that.

[Peck, Jon]

If you are reading a sav file, it know the case count from the header, but if you are reading another source, SPSS won't know the case count until the data have been passed.

Regards,
Jon Peck


-----Original Message-----
From: SPSSX(r) Discussion on behalf of Richard Ristow
Sent: Sat 9/15/2007 12:24 PM
To: [hidden email]
Subject:      Re: [SPSSX-L] number of up front in a file
 
At 01:12 PM 9/15/2007, Albert-jan Roskam wrote:

>I get that message sometimes when large files aren't completely
>cached. So yep, I also think that the file is rather large and isn't
>completely opened yet. Maybe spss can't read the header info when the
>file isn't entirely read yet?

It's a good question. It worked fine with the largest one I have handy,
which is 49 megabytes and 623,000 records, but that isn't huge these
days.

On the other hand, SPSS has to read most of the header info
immediately. The largest part of this is the data dictionary, and SPSS
can't process a transformation program without access to that.

[Richard Ristow]

Thank you, Jon,

As I'm catching up some long-pending correspondence

At 10:41 AM 9/16/2007, Peck, Jon wrote:

>If you are reading a sav file, it know the case count from the header,

I'd hoped that would be the case. (Darned hard to find, though; perhaps
I need a better hex editor than frhed. I assume it's a 32-bit signed
integer,

>but if you are reading another source, SPSS won't know the case count
>until the data have been passed.

No, of course not. Is the clairvoyance module due out in release 17?
Darned, they're already freezing the specs for 17, and I'll bet they
forgot this one.

Pity. The clairvoyance model could be used for missing-data imputation,
too. No need to worry about reliability of the imputation technique.

-Cheers and onward,
  Richard

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD