SPSSX Discussion

Overwriting a dataset

Classic

List

Threaded

16 messages Options

paulm

Overwriting a dataset

Hi, I am somewhat new to SPSS and I am trying to resolve an issue that I have when I run my syntax on the SPSS Server vs. running the syntax locally. Basically I am retrieving a dataset, doing some manipulations and then saving the dataset (overwriting it). Here is the syntax I am using:

DEFINE !newdate (!positional = !TOKENS(1) / !positional = !TOKENS(1))
* RETRIEVE THE CURRENT DATASET.
GET FILE=!QUOTE(!CONCAT('e:\se\',!1,'\',!2,'\Data\POST\post_data.sav')).
DATASET NAME DataSet1 WINDOW=FRONT.

dataset activate dataset1.
dataset copy dataset2.
dataset close dataset1.
compute mydate=!maxdate .
execute.

*FILTER OUT THE LAST DATE IN THE DATASET.
DATASET ACTIVATE DataSet2.
FILTER OFF.
USE ALL.
SELECT IF (date < mydate) .
EXECUTE.
save outfile=!QUOTE(!CONCAT('e:\se\',!league,'\',!venue,'\Data\POST\post_data.sav'))
/ compressed .
!ENDDEFINE.
!newdate level1 level2 .

When I run this locally the syntax works. However, when I run it on the SPSS Server I get the following error: "Due to contention for the specified file, the data has been saved to a file with a different name -
e:\se\level1\level2\data\post\post_data_1.sav"

Can anyone tell me a more efficient way to accomplish this task.

John F Hall

Re: Overwriting a dataset

This is probably a safety device to protect your original data. If you use
GUI, double click on the *.sav file you want to open and perform your
operations on the working file. Then it's good practice to click on:

File > Save as

. . . and save your working file with a different filename (name1.sav,
name2.sav etc). That way you don't risk losing your original data if you
make a mistake.

John F Hall

[hidden email]
www.surveyresearch.weebly.com

PS Since you're new to SPSS there are 400+ pages of tutorials on my site,
plus links to dozens of useful SPSS resources.

-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of
paulm
Sent: 16 December 2011 17:38
To: [hidden email]
Subject: Overwriting a dataset

Hi, I am somewhat new to SPSS and I am trying to resolve an issue that I
have when I run my syntax on the SPSS Server vs. running the syntax locally.
Basically I am retrieving a dataset, doing some manipulations and then
saving the dataset (overwriting it). Here is the syntax I am using:

DEFINE !newdate (!positional = !TOKENS(1) / !positional = !TOKENS(1))
* RETRIEVE THE CURRENT DATASET.
GET FILE=!QUOTE(!CONCAT('e:\se\',!1,'\',!2,'\Data\POST\post_data.sav')).
DATASET NAME DataSet1 WINDOW=FRONT.

dataset activate dataset1.
dataset copy dataset2.
dataset close dataset1.
compute mydate=!maxdate .
execute.

*FILTER OUT THE LAST DATE IN THE DATASET.
DATASET ACTIVATE DataSet2.
FILTER OFF.
USE ALL.
SELECT IF (date < mydate) .
EXECUTE.
save
outfile=!QUOTE(!CONCAT('e:\se\',!league,'\',!venue,'\Data\POST\post_data.sav
'))
/ compressed .
!ENDDEFINE.
!newdate level1 level2 .

When I run this locally the syntax works. However, when I run it on the
SPSS Server I get the following error: "Due to contention for the specified
file, the data has been saved to a file with a different name -
e:\se\level1\level2\data\post\post_data_1.sav"

Can anyone tell me a more efficient way to accomplish this task.

--
View this message in context:
http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp508076
8p5080768.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

David Marso

Re: Overwriting a dataset

Administrator

In reply to this post by paulm

Rather dodgy convoluted code!!!
Where are !maxdate, !league, !venue defined?
I presume they are set up as other macros somewhere else in the code (dubious practice IMNSHO).
This whole mess could be recasted as :
---
DEFINE !newdate (!positional = !TOKENS(1) / !positional = !TOKENS(1))
* RETRIEVE THE CURRENT DATASET.
GET FILE=!QUOTE(!CONCAT('e:\se\',!1,'\',!2,'\Data\POST\post_data.sav')).
SELECT IF (date < !maxdate) .
SAVE OUTFILE=!QUOTE(!CONCAT('e:\se\',!1,'\',!2,'\Data\POST\post_data.sav')).
!ENDDEFINE.
!newdate level1 level2 .

On the other hand, I *DEPLORE* the use of macro constants within other macros and positionals make the code difficult to read. In general, *WHY BOTHER with a macro when you have all this hard coded baggage?* Totally defeats the purpose of having a macro in the first place. Better off just passing the full file path and be done with it! Not sure if the following will yield a file contention issue (no way to test as I don't have access to a server version). But it certainly goes for the jugular and if it does you may conclude that you are SOL and should run it locally!
HTH, David
--
DEFINE !newdate ( !filepath !ENCLOSE("[","]") / maxdate !CMDEND ).
GET FILE !QUOTE(!filepath) .
SELECT IF date LT NUMBER(!maxdate,ADATE) .
SAVE OUTFILE !QUOTE(!Filepath).
!ENDDEFINE.

!newdate filepath ['e:\se\level1\level2\Data\POST\post_data.sav'] maxdate="12/15/2011" .

paulm wrote

Hi, I am somewhat new to SPSS and I am trying to resolve an issue that I have when I run my syntax on the SPSS Server vs. running the syntax locally. Basically I am retrieving a dataset, doing some manipulations and then saving the dataset (overwriting it). Here is the syntax I am using:

DEFINE !newdate (!positional = !TOKENS(1) / !positional = !TOKENS(1))
* RETRIEVE THE CURRENT DATASET.
GET FILE=!QUOTE(!CONCAT('e:\se\',!1,'\',!2,'\Data\POST\post_data.sav')).
DATASET NAME DataSet1 WINDOW=FRONT.

dataset activate dataset1.
dataset copy dataset2.
dataset close dataset1.
compute mydate=!maxdate .
execute.

*FILTER OUT THE LAST DATE IN THE DATASET.
DATASET ACTIVATE DataSet2.
FILTER OFF.
USE ALL.
SELECT IF (date < mydate) .
EXECUTE.
save outfile=!QUOTE(!CONCAT('e:\se\',!league,'\',!venue,'\Data\POST\post_data.sav'))
/ compressed .
!ENDDEFINE.
!newdate level1 level2 .

When I run this locally the syntax works. However, when I run it on the SPSS Server I get the following error: "Due to contention for the specified file, the data has been saved to a file with a different name -
e:\se\level1\level2\data\post\post_data_1.sav"

Can anyone tell me a more efficient way to accomplish this task.

Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me.
---
"Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis."
Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?"

Art Kendall

Re: Overwriting a dataset

In reply to this post by paulm

Dangerous! If you overwrite a dataset will you still be able to go back and do the process over in the way you want versus the way you said to do it?

Art Kendall
Social Research Consultants

On 12/16/2011 11:38 AM, paulm wrote:

Hi,  I am somewhat new to SPSS and I am trying to resolve an issue that I
have when I run my syntax on the SPSS Server vs. running the syntax locally.
Basically I am retrieving a dataset, doing some manipulations and then
saving the dataset (overwriting it).  Here is the syntax I am using:


DEFINE !newdate (!positional = !TOKENS(1) / !positional = !TOKENS(1))
* RETRIEVE THE CURRENT DATASET.
GET FILE=!QUOTE(!CONCAT('e:\se\',!1,'\',!2,'\Data\POST\post_data.sav')).
DATASET NAME DataSet1 WINDOW=FRONT.

dataset activate dataset1.
dataset copy dataset2.
dataset close dataset1.
compute mydate=!maxdate .
execute.

*FILTER OUT THE LAST DATE IN THE DATASET.
DATASET ACTIVATE DataSet2.
FILTER OFF.
USE ALL.
SELECT IF (date < mydate) .
EXECUTE.
save
outfile=!QUOTE(!CONCAT('e:\se\',!league,'\',!venue,'\Data\POST\post_data.sav'))
/ compressed .
!ENDDEFINE.
!newdate level1 level2  .

When I run this locally the syntax works.  However, when I run it on the
SPSS Server I get the following error: "Due to contention for the specified
file, the data has been saved to a file with a different name -
e:\se\level1\level2\data\post\post_data_1.sav"

Can anyone tell me a more efficient way to accomplish this task.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5080768.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

Art Kendall
Social Research Consultants

Dale

Re: Overwriting a dataset

I tend to use (and suggest) the age-old archived main data set and then work from a “working data set” to address such cases.

Dale

==========================================================================
CONFIDENTIALITY NOTICE: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, you may not review, copy or distribute this message. If you have received this email in error, please notify the sender immediately and delete the original message. Neither the sender nor the company for which he or she works accepts any liability for any damage caused by any virus transmitted by this email.
==========================================================================

Dale Pietrzak, Ed.D., LPC-MH, CCMHC
Director, Office of Academic Evaluation & Assessment
University of South Dakota
414 East Clark Street
Vermillion, SD 57069
(605) 677-6497

We are continually faced with a series of great opportunities brilliantly disguised as insoluble problems. - John W. Gardner

Problems cannot be solved at the same level of awareness that created them. - A. Einstein

From: SPSSX(r) Discussion [mailto:[hidden email]] On Behalf Of Art Kendall
Sent: Friday, December 16, 2011 11:39 AM
To: [hidden email]
Subject: Re: Overwriting a dataset

Dangerous! If you overwrite a dataset will you still be able to go back and do the process over in the way you want versus the way you said to do it?

Art Kendall

Social Research Consultants

On 12/16/2011 11:38 AM, paulm wrote:

Hi,  I am somewhat new to SPSS and I am trying to resolve an issue that I

have when I run my syntax on the SPSS Server vs. running the syntax locally.

Basically I am retrieving a dataset, doing some manipulations and then

saving the dataset (overwriting it).  Here is the syntax I am using:

DEFINE !newdate (!positional = !TOKENS(1) / !positional = !TOKENS(1))

* RETRIEVE THE CURRENT DATASET.

GET FILE=!QUOTE(!CONCAT('e:\se\',!1,'\',!2,'\Data\POST\post_data.sav')).

DATASET NAME DataSet1 WINDOW=FRONT.

dataset activate dataset1.

dataset copy dataset2.

dataset close dataset1.

compute mydate=!maxdate .

execute.

*FILTER OUT THE LAST DATE IN THE DATASET.

DATASET ACTIVATE DataSet2.

FILTER OFF.

USE ALL.

SELECT IF (date < mydate) .

EXECUTE.

save

outfile=!QUOTE(!CONCAT('e:\se\',!league,'\',!venue,'\Data\POST\post_data.sav'))

/ compressed .

!ENDDEFINE.

!newdate level1 level2  .

When I run this locally the syntax works.  However, when I run it on the

SPSS Server I get the following error: "Due to contention for the specified

file, the data has been saved to a file with a different name -

e:\se\level1\level2\data\post\post_data_1.sav"

Can anyone tell me a more efficient way to accomplish this task.

--

View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5080768.html

Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================

To manage your subscription to SPSSX-L, send a message to

[hidden email] (not to SPSSX-L), with no body text except the

command. To leave the list, send the command

SIGNOFF SPSSX-L

For a list of commands to manage subscriptions, send the command

INFO REFCARD

David Marso

Re: Overwriting a dataset

Administrator

In reply to this post by Art Kendall

One should *NEVER* overwrite data unless one can recreate it from original sources or some well defined documented processing milestone. Sometimes accidents happen and it can be *VERY EXPENSIVE* to recreate a crucial data set. Once upon a time in a land far far away I was working on a project with 4 other consultants. We had one shared dataset which was built from several other files after sorts/merges/complex flag calculations etc... All in all it took about 3 hours to build the master file from source data using the complex data preparation syntax. It took about an HOUR to sort the damned thing (500K cases, 400+ variables .. back in mid 90's when EVERYTHING was *SLOOOWWW*).
One afternoon I grabbed the data, ran a bit of code for the part I was working on and was QUITE distressed to find there were only 10000 cases in the file. Turns out that one of the team members had closed out of SPSS and clicked the F'ing button asking Save changes before closing....
Consequently 4 people were sitting on their a$$es for 3 hours while I rebuilt the file from source files.
After that, EVERYTHING got locked down (set ALL file permissions to be READ only).

Art Kendall wrote

Dangerous! If you overwrite a dataset will you
still be able to go back and do the process over in the way you
want versus the way you said to do it?
Art Kendall
Social Research Consultants

On 12/16/2011 11:38 AM, paulm wrote:

Hi, I am somewhat new to SPSS and I am trying to resolve an issue that I
have when I run my syntax on the SPSS Server vs. running the syntax locally.
Basically I am retrieving a dataset, doing some manipulations and then
saving the dataset (overwriting it). Here is the syntax I am using:

DEFINE !newdate (!positional = !TOKENS(1) / !positional = !TOKENS(1))
* RETRIEVE THE CURRENT DATASET.
GET FILE=!QUOTE(!CONCAT('e:\se\',!1,'\',!2,'\Data\POST\post_data.sav')).
DATASET NAME DataSet1 WINDOW=FRONT.

dataset activate dataset1.
dataset copy dataset2.
dataset close dataset1.
compute mydate=!maxdate .
execute.

*FILTER OUT THE LAST DATE IN THE DATASET.
DATASET ACTIVATE DataSet2.
FILTER OFF.
USE ALL.
SELECT IF (date < mydate) .
EXECUTE.
save
outfile=!QUOTE(!CONCAT('e:\se\',!league,'\',!venue,'\Data\POST\post_data.sav'))
/ compressed .
!ENDDEFINE.
!newdate level1 level2 .

When I run this locally the syntax works. However, when I run it on the
SPSS Server I get the following error: "Due to contention for the specified
file, the data has been saved to a file with a different name -
e:\se\level1\level2\data\post\post_data_1.sav"

Can anyone tell me a more efficient way to accomplish this task.

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5080768.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Art Kendall

Re: Overwriting a dataset

Art Kendall
Social Research Consultants

On 12/16/2011 2:06 PM, David Marso wrote:

One should *NEVER* overwrite data unless one can recreate it from original
sources or some well defined documented processing milestone.   Sometimes
accidents happen and it can be *VERY EXPENSIVE* to recreate a crucial data
set.  Once upon a time in a land far far away I was working on a project
with 4 other consultants.  We had one shared dataset which was built from
several other files after sorts/merges/complex flag calculations etc... All
in all it took about 3 hours to build the master file from source data using
the complex data preparation syntax.  It took about an HOUR to sort the
damned thing (500K cases, 400+ variables .. back in mid 90's when EVERYTHING
was *SLOOOWWW*).
One afternoon I grabbed the data, ran a bit of code for the part I was
working on and was QUITE distressed to find there were only 10000 cases in
the file.  Turns out that one of the team members had closed out of SPSS and
clicked the F'ing button asking Save changes before closing....
Consequently 4 people were sitting on their a$$es for 3 hours while I
rebuilt the file from source files.
After that, EVERYTHING got locked down (set ALL file permissions to be READ
only).


Art Kendall wrote

Dangerous!&nbsp; If you overwrite a dataset will you
      still be able to go back and do the process over in the way you
      want versus the way you said to do it?
    Art Kendall
Social Research Consultants

    On 12/16/2011 11:38 AM, paulm wrote:

      Hi,  I am somewhat new to SPSS and I am trying to resolve an issue
that I
have when I run my syntax on the SPSS Server vs. running the syntax
locally.
Basically I am retrieving a dataset, doing some manipulations and then
saving the dataset (overwriting it).  Here is the syntax I am using:


DEFINE !newdate (!positional = !TOKENS(1) / !positional = !TOKENS(1))
* RETRIEVE THE CURRENT DATASET.
GET FILE=!QUOTE(!CONCAT('e:\se\',!1,'\',!2,'\Data\POST\post_data.sav')).
DATASET NAME DataSet1 WINDOW=FRONT.

dataset activate dataset1.
dataset copy dataset2.
dataset close dataset1.
compute mydate=!maxdate .
execute.

*FILTER OUT THE LAST DATE IN THE DATASET.
DATASET ACTIVATE DataSet2.
FILTER OFF.
USE ALL.
SELECT IF (date &lt; mydate) .
EXECUTE.
save
outfile=!QUOTE(!CONCAT('e:\se\',!league,'\',!venue,'\Data\POST\post_data.sav'))
/ compressed .
!ENDDEFINE.
!newdate level1 level2  .

When I run this locally the syntax works.  However, when I run it on the
SPSS Server I get the following error: "Due to contention for the
specified
file, the data has been saved to a file with a different name -
e:\se\level1\level2\data\post\post_data_1.sav"

Can anyone tell me a more efficient way to accomplish this task.

--
View this message in context:
http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5080768.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email]  (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD






=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5081115.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Art Kendall
Social Research Consultants

Jon K Peck

Re: Overwriting a dataset

Apropos of which, I would remind people that Statistics has a feature to set file permissions.

Using SAVE, you can include /PERMISSIONS=READONLY
or you can use the PERMISSIONS command:
PERMISSIONS FILE=filespec /PERMISSIONS READONLY.

If you are using C & DS (Collaboration and Deployment Services), if you save to the repository, previous generations of the file are preserved, so you can go back to earlier versions. That works for syntax and other file types, too. Preserving previous versions of your syntax file can be as important as preserving your data.

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
new phone: 720-342-5621

From: Art Kendall <[hidden email]>
To: [hidden email]
Date: 12/16/2011 01:01 PM
Subject: Re: [SPSSX-L] Overwriting a dataset
Sent by: "SPSSX(r) Discussion" <[hidden email]>

I have seen things like this happen dozens of times over the years. Losing 3 hours is a pain. The worse situation I have seen was when someone wiped her data out with an "IF (something impossible);" in SAS. (SAS's version of SELECT IF). This was an hour after she shredded the paper copy of her questionnaires to protect privacy. Two years work gone.

In 1971 after a couple of weeks experience, I realized that computers do what they are told. Which is often not what you want them to do.
Art Kendall Social Research Consultants

On 12/16/2011 2:06 PM, David Marso wrote:
One should *NEVER* overwrite data unless one can recreate it from original sources or some well defined documented processing milestone. Sometimes accidents happen and it can be *VERY EXPENSIVE* to recreate a crucial data set. Once upon a time in a land far far away I was working on a project with 4 other consultants. We had one shared dataset which was built from several other files after sorts/merges/complex flag calculations etc... All in all it took about 3 hours to build the master file from source data using the complex data preparation syntax. It took about an HOUR to sort the damned thing (500K cases, 400+ variables .. back in mid 90's when EVERYTHING was *SLOOOWWW*). One afternoon I grabbed the data, ran a bit of code for the part I was working on and was QUITE distressed to find there were only 10000 cases in the file. Turns out that one of the team members had closed out of SPSS and clicked the F'ing button asking Save changes before closing.... Consequently 4 people were sitting on their a$$es for 3 hours while I rebuilt the file from source files. After that, EVERYTHING got locked down (set ALL file permissions to be READ only). Art Kendall wrote
Dangerous!  If you overwrite a dataset will you still be able to go back and do the process over in the way you want versus the way you said to do it? Art Kendall Social Research Consultants On 12/16/2011 11:38 AM, paulm wrote: Hi, I am somewhat new to SPSS and I am trying to resolve an issue that I have when I run my syntax on the SPSS Server vs. running the syntax locally. Basically I am retrieving a dataset, doing some manipulations and then saving the dataset (overwriting it). Here is the syntax I am using: DEFINE !newdate (!positional = !TOKENS(1) / !positional = !TOKENS(1)) * RETRIEVE THE CURRENT DATASET. GET FILE=!QUOTE(!CONCAT('e:\se\',!1,'\',!2,'\Data\POST\post_data.sav')). DATASET NAME DataSet1 WINDOW=FRONT. dataset activate dataset1. dataset copy dataset2. dataset close dataset1. compute mydate=!maxdate . execute. *FILTER OUT THE LAST DATE IN THE DATASET. DATASET ACTIVATE DataSet2. FILTER OFF. USE ALL. SELECT IF (date < mydate) . EXECUTE. save outfile=!QUOTE(!CONCAT('e:\se\',!league,'\',!venue,'\Data\POST\post_data.sav')) / compressed . !ENDDEFINE. !newdate level1 level2 . When I run this locally the syntax works. However, when I run it on the SPSS Server I get the following error: "Due to contention for the specified file, the data has been saved to a file with a different name - e:\se\level1\level2\data\post\post_data_1.sav" Can anyone tell me a more efficient way to accomplish this task. -- View this message in context:http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5080768.htmlSent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message toLISTSERV@.UGA (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message toLISTSERV@.UGA(not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
-- View this message in context:http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5081115.htmlSent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message toLISTSERV@...(not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

David Marso

Re: Overwriting a dataset

Administrator

In reply to this post by Art Kendall

That was actually 3 hrs x 4 people = 12 hrs on a billable project approx $1200 because of one stupid mistake! Ultimately it was *MY* mistake because I didn't lock down the file immediately after creation.
I was under the mistaken impression that I was working with fellow professionals.
I should have had 2 backups too (actually was backed up on the system but Systems/IT wanted 24 hrs to pull it from tape and we were totally under the gun to get the project completed).
" This was an hour after she shredded the paper copy of her questionnaires to protect privacy.."
What a F'ing idiot! That is what LOCKED FILE cabinets are for!!!!!

Art Kendall wrote

I have seen things like this happen dozens of times
over the years. Losing 3 hours is a pain. The worse situation I
have seen was when someone wiped her data out with an "IF
(something impossible);" in SAS. (SAS's version of SELECT IF).
This was an hour after she shredded the paper copy of her
questionnaires to protect privacy. Two years work gone.

In 1971 after a couple of weeks experience, I realized that
computers do what they are told. Which is often not what you want
them to do.
Art Kendall
Social Research Consultants

On 12/16/2011 2:06 PM, David Marso wrote:

One should *NEVER* overwrite data unless one can recreate it from original
sources or some well defined documented processing milestone. Sometimes
accidents happen and it can be *VERY EXPENSIVE* to recreate a crucial data
set. Once upon a time in a land far far away I was working on a project
with 4 other consultants. We had one shared dataset which was built from
several other files after sorts/merges/complex flag calculations etc... All
in all it took about 3 hours to build the master file from source data using
the complex data preparation syntax. It took about an HOUR to sort the
damned thing (500K cases, 400+ variables .. back in mid 90's when EVERYTHING
was *SLOOOWWW*).
One afternoon I grabbed the data, ran a bit of code for the part I was
working on and was QUITE distressed to find there were only 10000 cases in
the file. Turns out that one of the team members had closed out of SPSS and
clicked the F'ing button asking Save changes before closing....
Consequently 4 people were sitting on their a$$es for 3 hours while I
rebuilt the file from source files.
After that, EVERYTHING got locked down (set ALL file permissions to be READ
only).

Art Kendall wrote

Dangerous!  If you overwrite a dataset will you
still be able to go back and do the process over in the way you
want versus the way you said to do it?
Art Kendall
Social Research Consultants

On 12/16/2011 11:38 AM, paulm wrote:

Hi, I am somewhat new to SPSS and I am trying to resolve an issue
that I
have when I run my syntax on the SPSS Server vs. running the syntax
locally.
Basically I am retrieving a dataset, doing some manipulations and then
saving the dataset (overwriting it). Here is the syntax I am using:

DEFINE !newdate (!positional = !TOKENS(1) / !positional = !TOKENS(1))
* RETRIEVE THE CURRENT DATASET.
GET FILE=!QUOTE(!CONCAT('e:\se\',!1,'\',!2,'\Data\POST\post_data.sav')).
DATASET NAME DataSet1 WINDOW=FRONT.

dataset activate dataset1.
dataset copy dataset2.
dataset close dataset1.
compute mydate=!maxdate .
execute.

*FILTER OUT THE LAST DATE IN THE DATASET.
DATASET ACTIVATE DataSet2.
FILTER OFF.
USE ALL.
SELECT IF (date < mydate) .
EXECUTE.
save
outfile=!QUOTE(!CONCAT('e:\se\',!league,'\',!venue,'\Data\POST\post_data.sav'))
/ compressed .
!ENDDEFINE.
!newdate level1 level2 .

When I run this locally the syntax works. However, when I run it on the
SPSS Server I get the following error: "Due to contention for the
specified
file, the data has been saved to a file with a different name -
e:\se\level1\level2\data\post\post_data_1.sav"

Can anyone tell me a more efficient way to accomplish this task.

--
View this message in context:
http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5080768.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
LISTSERV@.UGA (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
LISTSERV@.UGA (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5081115.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

Jara Kampmann-2

Re: Overwriting a dataset

In reply to this post by paulm

I totally agree with all the warnings and the precautions one should take.
Nevertheless, I am wondering whether there is a possibility to save changes into the active data set via syntax. My annoyance is: if I create an outfile with aggregate, get the file and perform some changes, I always have to create a new file (with new data set name) to be able to save the changes. Which causes additional work of rewriting when having syntax to be run on several data sets as well as the need for more storing capacity.

So does one of you know of any syntax snippet that would do the trick (if necessary)?

Jara

----- Ursprüngliche Nachricht -----

Von: Jon K Peck

Gesendet: 16.12.11 21:13 Uhr

An: [hidden email]

Betreff: Re: Overwriting a dataset

Apropos of which, I would remind people that Statistics has a feature to set file permissions.

Using SAVE, you can include /PERMISSIONS=READONLY
or you can use the PERMISSIONS command:
PERMISSIONS FILE=filespec /PERMISSIONS READONLY.

If you are using C & DS (Collaboration and Deployment Services), if you save to the repository, previous generations of the file are preserved, so you can go back to earlier versions. That works for syntax and other file types, too. Preserving previous versions of your syntax file can be as important as preserving your data.

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
new phone: 720-342-5621

From: Art Kendall <[hidden email]>
To: [hidden email]
Date: 12/16/2011 01:01 PM
Subject: Re: [SPSSX-L] Overwriting a dataset
Sent by: "SPSSX(r) Discussion" <[hidden email]>

I have seen things like this happen dozens of times over the years. Losing 3 hours is a pain. The worse situation I have seen was when someone wiped her data out with an "IF (something impossible);" in SAS. (SAS's version of SELECT IF). This was an hour after she shredded the paper copy of her questionnaires to protect privacy. Two years work gone.

In 1971 after a couple of weeks experience, I realized that computers do what they are told. Which is often not what you want them to do.
Art Kendall Social Research Consultants

On 12/16/2011 2:06 PM, David Marso wrote:
One should *NEVER* overwrite data unless one can recreate it from original sources or some well defined documented processing milestone. Sometimes accidents happen and it can be *VERY EXPENSIVE* to recreate a crucial data set. Once upon a time in a land far far away I was working on a project with 4 other consultants. We had one shared dataset which was built from several other files after sorts/merges/complex flag calculations etc... All in all it took about 3 hours to build the master file from source data using the complex data preparation syntax. It took about an HOUR to sort the damned thing (500K cases, 400+ variables .. back in mid 90's when EVERYTHING was *SLOOOWWW*). One afternoon I grabbed the data, ran a bit of code for the part I was working on and was QUITE distressed to find there were only 10000 cases in the file. Turns out that one of the team members had closed out of SPSS and clicked the F'ing button asking Save changes before closing.... Consequently 4 people were sitting on their a$$es for 3 hours while I rebuilt the file from source files. After that, EVERYTHING got locked down (set ALL file permissions to be READ only). Art Kendall wrote

Dangerous!  If you overwrite a dataset will you still be able to go back and do the process over in the way you want versus the way you said to do it? Art Kendall Social Research Consultants On 12/16/2011 11:38 AM, paulm wrote: Hi, I am somewhat new to SPSS and I am trying to resolve an issue that I have when I run my syntax on the SPSS Server vs. running the syntax locally. Basically I am retrieving a dataset, doing some manipulations and then saving the dataset (overwriting it). Here is the syntax I am using: DEFINE !newdate (!positional = !TOKENS(1) / !positional = !TOKENS(1)) * RETRIEVE THE CURRENT DATASET. GET FILE=!QUOTE(!CONCAT('e:\se\',!1,'\',!2,'\Data\POST\post_data.sav')). DATASET NAME DataSet1 WINDOW=FRONT. dataset activate dataset1. dataset copy dataset2. dataset close dataset1. compute mydate=!maxdate . execute. *FILTER OUT THE LAST DATE IN THE DATASET. DATASET ACTIVATE DataSet2. FILTER OFF. USE ALL. SELECT IF (date < mydate) . EXECUTE. save outfile=!QUOTE(!CONCAT('e:\se\',!league,'\',!venue,'\Data\POST\post_data.sav')) / compressed . !ENDDEFINE. !newdate level1 level2 . When I run this locally the syntax works. However, when I run it on the SPSS Server I get the following error: "Due to contention for the specified file, the data has been saved to a file with a different name - e:\se\level1\level2\data\post\post_data_1.sav" Can anyone tell me a more efficient way to accomplish this task. -- View this message in context:
http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5080768.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to
[hidden email](not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

-- View this message in context:http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5081115.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to
[hidden email](not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

Art Kendall

Re: Overwriting a dataset

I am not sure I understand what you would like to do.� Please give us more detail.

Art Kendall
Social Research Consultants

On 12/16/2011 3:49 PM, Jara Kampmann wrote:

I totally agree with all the warnings and the precautions one should take.
Nevertheless, I am wondering whether there is a possibility to save changes into the active data set via syntax. My annoyance is: if I create an outfile with aggregate, get the file and perform some changes, I always have to create a new file (with new data set name) to be able to save the changes. Which causes additional work of rewriting when having syntax to be run on several data sets as well as the need for more storing capacity.

So does one of you know of any syntax snippet that would do the trick (if necessary)?

Jara

�

----- Ursprüngliche Nachricht -----

Von: Jon K Peck

Gesendet: 16.12.11 21:13 Uhr

An: [hidden email]

Betreff: Re: Overwriting a dataset

Apropos of which, I would remind people that Statistics has a feature to set file permissions.

Using SAVE, you can include /PERMISSIONS=READONLY
or you can use the PERMISSIONS command:
PERMISSIONS FILE=filespec /PERMISSIONS READONLY.

If you are using C & DS (Collaboration and Deployment Services), if you save to the repository, previous generations of the file are preserved, so � you can go � back to earlier versions. � That works for syntax and other file types, too. � Preserving previous versions of your syntax file can be as important as preserving your data.

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
new phone: 720-342-5621

From: � � � � Art Kendall [hidden email]
To: � � � � [hidden email]
Date: � � � � 12/16/2011 01:01 PM
Subject: � � � � Re: [SPSSX-L] Overwriting a dataset
Sent by: � � � � "SPSSX(r) Discussion" [hidden email]

I have seen things like this happen dozens of times over the years. � Losing 3 hours is a pain. � The worse situation I have seen was when someone wiped her data out with an "IF (something impossible);" in SAS. (SAS's version of SELECT IF). � This was an hour after she shredded the paper copy of her questionnaires to protect privacy. � Two years work gone.

In 1971 after a couple of weeks experience, I realized that computers do what they are told. Which is often not what you want them to do.
Art Kendall Social Research Consultants

On 12/16/2011 2:06 PM, David Marso wrote:
One should *NEVER* overwrite data unless one can recreate it from original sources or some well defined documented processing milestone. � Sometimes accidents happen and it can be *VERY EXPENSIVE* to recreate a crucial data set. � Once upon a time in a land far far away I was working on a project with 4 other consultants. � We had one shared dataset which was built from several other files after sorts/merges/complex flag calculations etc... All in all it took about 3 hours to build the master file from source data using the complex data preparation syntax. � It took about an HOUR to sort the damned thing (500K cases, 400+ variables .. back in mid 90's when EVERYTHING was *SLOOOWWW*). One afternoon I grabbed the data, ran a bit of code for the part I was working on and was QUITE distressed to find there were only 10000 cases in the file. � Turns out that one of the team members had closed out of SPSS and clicked the F'ing button asking Save changes before closing.... Consequently 4 people were sitting on their a$$es for 3 hours while I rebuilt the file from source files. After that, EVERYTHING got locked down (set ALL file permissions to be READ only). Art Kendall wrote

Dangerous!  If you overwrite a dataset will you � � � still be able to go back and do the process over in the way you � � � want versus the way you said to do it? � � Art Kendall Social Research Consultants � � On 12/16/2011 11:38 AM, paulm wrote: � � � Hi, � I am somewhat new to SPSS and I am trying to resolve an issue that I have when I run my syntax on the SPSS Server vs. running the syntax locally. Basically I am retrieving a dataset, doing some manipulations and then saving the dataset (overwriting it). � Here is the syntax I am using: DEFINE !newdate (!positional = !TOKENS(1) / !positional = !TOKENS(1)) * RETRIEVE THE CURRENT DATASET. GET FILE=!QUOTE(!CONCAT('e:\se\',!1,'\',!2,'\Data\POST\post_data.sav')). DATASET NAME DataSet1 WINDOW=FRONT. dataset activate dataset1. dataset copy dataset2. dataset close dataset1. compute mydate=!maxdate . execute. *FILTER OUT THE LAST DATE IN THE DATASET. DATASET ACTIVATE DataSet2. FILTER OFF. USE ALL. SELECT IF (date < mydate) . EXECUTE. save outfile=!QUOTE(!CONCAT('e:\se\',!league,'\',!venue,'\Data\POST\post_data.sav')) / compressed . !ENDDEFINE. !newdate level1 level2 � . When I run this locally the syntax works. � However, when I run it on the SPSS Server I get the following error: "Due to contention for the specified file, the data has been saved to a file with a different name - e:\se\level1\level2\data\post\post_data_1.sav" Can anyone tell me a more efficient way to accomplish this task. -- View this message in context:
http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5080768.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to
[hidden email]� (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to
[hidden email](not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

-- View this message in context:http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5081115.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to
[hidden email](not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

�

Art Kendall
Social Research Consultants

Jara Kampmann-2

Re: Overwriting a dataset

In reply to this post by paulm

Art,

I would like to be able to perform via syntax:
get file="....sav". /*get a file.
flip. /*perform any change to the file.
"save file." /*save changes into the active data set, just as in GUI using the "disket"-Button.

The rest of my earlier posting was to explain why I would like to be able to do this despite the risks connected to it, but that is not important. So, for special occasions, I do want to overwrite my data via syntax.

----- Ursprüngliche Nachricht -----

Von: Art Kendall

Gesendet: 16.12.11 22:25 Uhr

An: [hidden email]

Betreff: Re: Overwriting a dataset
I am not sure I understand what you would like to do.? Please give us more detail.
 
Art Kendall 
Social Research Consultants
On 12/16/2011 3:49 PM, Jara Kampmann wrote:

I totally agree with all the warnings and the precautions one should take.
Nevertheless, I am wondering whether there is a possibility to save changes into the active data set via syntax. My annoyance is: if I create an outfile with aggregate, get the file and perform some changes, I always have to create a new file (with new data set name) to be able to save the changes. Which causes additional work of rewriting when having syntax to be run on several data sets as well as the need for more storing capacity.

So does one of you know of any syntax snippet that would do the trick (if necessary)?

Jara

?

----- Ursprüngliche Nachricht -----

Von: Jon K Peck

Gesendet: 16.12.11 21:13 Uhr

An: [hidden email]

Betreff: Re: Overwriting a dataset

Apropos of which, I would remind people that Statistics has a feature to set file permissions.

Using SAVE, you can include /PERMISSIONS=READONLY
or you can use the PERMISSIONS command:
PERMISSIONS FILE=filespec /PERMISSIONS READONLY.

If you are using C & DS (Collaboration and Deployment Services), if you save to the repository, previous generations of the file are preserved, so ? you can go ? back to earlier versions. ? That works for syntax and other file types, too. ? Preserving previous versions of your syntax file can be as important as preserving your data.

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
new phone: 720-342-5621

From: ? ? ? ? Art Kendall [hidden email]
To: ? ? ? ? [hidden email]
Date: ? ? ? ? 12/16/2011 01:01 PM
Subject: ? ? ? ? Re: [SPSSX-L] Overwriting a dataset
Sent by: ? ? ? ? "SPSSX(r) Discussion" [hidden email]

I have seen things like this happen dozens of times over the years. ? Losing 3 hours is a pain. ? The worse situation I have seen was when someone wiped her data out with an "IF (something impossible);" in SAS. (SAS's version of SELECT IF). ? This was an hour after she shredded the paper copy of her questionnaires to protect privacy. ? Two years work gone.

In 1971 after a couple of weeks experience, I realized that computers do what they are told. Which is often not what you want them to do.
Art Kendall Social Research Consultants

On 12/16/2011 2:06 PM, David Marso wrote:
One should *NEVER* overwrite data unless one can recreate it from original sources or some well defined documented processing milestone. ? Sometimes accidents happen and it can be *VERY EXPENSIVE* to recreate a crucial data set. ? Once upon a time in a land far far away I was working on a project with 4 other consultants. ? We had one shared dataset which was built from several other files after sorts/merges/complex flag calculations etc... All in all it took about 3 hours to build the master file from source data using the complex data preparation syntax. ? It took about an HOUR to sort the damned thing (500K cases, 400+ variables .. back in mid 90's when EVERYTHING was *SLOOOWWW*). One afternoon I grabbed the data, ran a bit of code for the part I was working on and was QUITE distressed to find there were only 10000 cases in the file. ? Turns out that one of the team members had closed out of SPSS and clicked the F'ing button asking Save changes before closing.... Consequently 4 people were sitting on their a$$es for 3 hours while I rebuilt the file from source files. After that, EVERYTHING got locked down (set ALL file permissions to be READ only). Art Kendall wrote

Dangerous!  If you overwrite a dataset will you ? ? ? still be able to go back and do the process over in the way you ? ? ? want versus the way you said to do it? ? ? Art Kendall Social Research Consultants ? ? On 12/16/2011 11:38 AM, paulm wrote: ? ? ? Hi, ? I am somewhat new to SPSS and I am trying to resolve an issue that I have when I run my syntax on the SPSS Server vs. running the syntax locally. Basically I am retrieving a dataset, doing some manipulations and then saving the dataset (overwriting it). ? Here is the syntax I am using: DEFINE !newdate (!positional = !TOKENS(1) / !positional = !TOKENS(1)) * RETRIEVE THE CURRENT DATASET. GET FILE=!QUOTE(!CONCAT('e:\se\',!1,'\',!2,'\Data\POST\post_data.sav')). DATASET NAME DataSet1 WINDOW=FRONT. dataset activate dataset1. dataset copy dataset2. dataset close dataset1. compute mydate=!maxdate . execute. *FILTER OUT THE LAST DATE IN THE DATASET. DATASET ACTIVATE DataSet2. FILTER OFF. USE ALL. SELECT IF (date < mydate) . EXECUTE. save outfile=!QUOTE(!CONCAT('e:\se\',!league,'\',!venue,'\Data\POST\post_data.sav')) / compressed . !ENDDEFINE. !newdate level1 level2 ? . When I run this locally the syntax works. ? However, when I run it on the SPSS Server I get the following error: "Due to contention for the specified file, the data has been saved to a file with a different name - e:\se\level1\level2\data\post\post_data_1.sav" Can anyone tell me a more efficient way to accomplish this task. -- View this message in context:
http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5080768.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to
[hidden email]? (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to
[hidden email](not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

-- View this message in context:http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5081115.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to
[hidden email](not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

?
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

David Marso

Re: Overwriting a dataset

Administrator

Jara,
So...., what happens when you run:
GET FILE ="valid file path...".
FLIP.
SAVE OUTFILE="valid file path..."
er... me thinks you have not RTFM'd to a sufficient degree.
I doubt whether you will receive any error!

Jara Kampmann-2 wrote

Art,

I would like to be able to perform via syntax:
get file="....sav". /*get a file.
flip. /*perform any change to the file.
*"save file." */*save changes into the active data set, just as in GUI using the "disket"-Button.

The rest of my earlier posting was to explain why I would like to be able to do this despite the risks connected to it, but that is not important. So, for special occasions, I do want to overwrite my data via syntax.

----- Ursprüngliche Nachricht -----
Von: Art Kendall
Gesendet: 16.12.11 22:25 Uhr
An: [hidden email]
Betreff: Re: Overwriting a dataset

I am not sure I understand what you would like to do.? Please give us more detail.
Art Kendall Social Research Consultants
On 12/16/2011 3:49 PM, Jara Kampmann wrote:
I totally agree with all the warnings and the precautions one should take.
Nevertheless, I am wondering whether there is a possibility to save changes into the active data set via syntax. My annoyance is: if I create an outfile with aggregate, get the file and perform some changes, I always have to create a new file (with new data set name) to be able to save the changes. Which causes additional work of rewriting when having syntax to be run on several data sets as well as the need for more storing capacity.

So does one of you know of any syntax snippet that would do the trick (if necessary)?

Jara

?
----- Ursprüngliche Nachricht -----
Von: Jon K Peck
Gesendet: 16.12.11 21:13 Uhr
An: [hidden email]
Betreff: Re: Overwriting a dataset

Apropos of which, I would remind people that Statistics has a feature to set file permissions.

Using SAVE, you can include /PERMISSIONS=READONLY
or you can use the PERMISSIONS command:
PERMISSIONS FILE=filespec /PERMISSIONS READONLY.

If you are using C & DS (Collaboration and Deployment Services), if you save to the repository, previous generations of the file are preserved, so ? you can go ? back to earlier versions. ? That works for syntax and other file types, too. ? Preserving previous versions of your syntax file can be as important as preserving your data.

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
new phone: 720-342-5621

From: ? ? ? ? Art Kendall [hidden email]
To: ? ? ? ? [hidden email]
Date: ? ? ? ? 12/16/2011 01:01 PM
Subject: ? ? ? ? Re: [SPSSX-L] Overwriting a dataset
Sent by: ? ? ? ? "SPSSX(r) Discussion" [hidden email]
-----------------------------------------------------------------

I have seen things like this happen dozens of times over the years. ? Losing 3 hours is a pain. ? The worse situation I have seen was when someone wiped her data out with an "IF (something impossible);" in SAS. (SAS's version of SELECT IF). ? This was an hour after she shredded the paper copy of her questionnaires to protect privacy. ? Two years work gone.

In 1971 after a couple of weeks experience, I realized that computers do what they are told. Which is often not what you want them to do.
Art Kendall
Social Research Consultants

On 12/16/2011 2:06 PM, David Marso wrote:
One should *NEVER* overwrite data unless one can recreate it from original
sources or some well defined documented processing milestone. ? Sometimes
accidents happen and it can be *VERY EXPENSIVE* to recreate a crucial data
set. ? Once upon a time in a land far far away I was working on a project
with 4 other consultants. ? We had one shared dataset which was built from
several other files after sorts/merges/complex flag calculations etc... All
in all it took about 3 hours to build the master file from source data using
the complex data preparation syntax. ? It took about an HOUR to sort the
damned thing (500K cases, 400+ variables .. back in mid 90's when EVERYTHING
was *SLOOOWWW*).
One afternoon I grabbed the data, ran a bit of code for the part I was
working on and was QUITE distressed to find there were only 10000 cases in
the file. ? Turns out that one of the team members had closed out of SPSS and
clicked the F'ing button asking Save changes before closing....
Consequently 4 people were sitting on their a$$es for 3 hours while I
rebuilt the file from source files.
After that, EVERYTHING got locked down (set ALL file permissions to be READ
only).

Art Kendall wrote

Dangerous! If you overwrite a dataset will you
? ? ? still be able to go back and do the process over in the way you
? ? ? want versus the way you said to do it?
? ? Art Kendall
Social Research Consultants

? ? On 12/16/2011 11:38 AM, paulm wrote:

? ? ? Hi, ? I am somewhat new to SPSS and I am trying to resolve an issue
that I
have when I run my syntax on the SPSS Server vs. running the syntax
locally.
Basically I am retrieving a dataset, doing some manipulations and then
saving the dataset (overwriting it). ? Here is the syntax I am using:

DEFINE !newdate (!positional = !TOKENS(1) / !positional = !TOKENS(1))
* RETRIEVE THE CURRENT DATASET.
GET FILE=!QUOTE(!CONCAT('e:\se\',!1,'\',!2,'\Data\POST\post_data.sav')).
DATASET NAME DataSet1 WINDOW=FRONT.

dataset activate dataset1.
dataset copy dataset2.
dataset close dataset1.
compute mydate=!maxdate .
execute.

*FILTER OUT THE LAST DATE IN THE DATASET.
DATASET ACTIVATE DataSet2.
FILTER OFF.
USE ALL.
SELECT IF (date < mydate) .
EXECUTE.
save
outfile=!QUOTE(!CONCAT('e:\se\',!league,'\',!venue,'\Data\POST\post_data.sav'))
/ compressed .
!ENDDEFINE.
!newdate level1 level2 ? .

When I run this locally the syntax works. ? However, when I run it on the
SPSS Server I get the following error: "Due to contention for the
specified
file, the data has been saved to a file with a different name -
e:\se\level1\level2\data\post\post_data_1.sav"

Can anyone tell me a more efficient way to accomplish this task.

--
View this message in context:
http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5080768.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
LISTSERV@.UGA ? (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
LISTSERV@.UGA (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5081115.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
?

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

Art Kendall

Re: Overwriting a dataset

In reply to this post by Jara Kampmann-2

If you save all of your syntax and a separate copy of the original, you would retain the ability to refine your process.
With the capacity of today's machines even when working with files of 100,000 cases and a few hundred variables, I copy teh original and save all of the syntax.� I often do not save intermediate files.� As I redraft the syntax I keep copies of each segment of syntax until I get it the way I want it.� I may have several versions of a syntax segment until that segment is thoroughly developed.

One thing is to use the GUI and paste the syntax.

To control use of disk space (storage) and RAM (memory) you can use the DATASET commands and HOST commands to your OS to rename or delete files in your syntax.

The DATASET commands (DATASET NAME, DATASET ACTIVATE, DATASET DECLARE, DATASET COPY, DATASET CLOSE) provide the ability to have multiple data sources open at the same time and control which open data source is active at any point in the session.

below there is some syntax that I started by using the GUI and copying syntax from the output file into a syntax file.
I used the "SAVE AS" which comes up as "speichern unter" in Google translate.� I do not know what "disket" means.
Google gives "sparen" for "save".

HTH

Art Kendall
Social Research Consultants

GET
� � FILE='C:\Program Files\IBM\SPSS\Statistics\20\Samples\English\car_sales.sav'.
dataset name original.
dataset copy scratch.
dataset activate scratch.
dataset close original.
XSAVE OUTFILE='C:\project\car_sales scratch.sav'
� � /COMPRESSED.
compute loss4year=� price- resale.
formats loss4year (f8.3) .
descriptives variables = loss4year.
GET
� � FILE='C:\project\car_sales scratch.sav'.
aggregate outfile=* mode=addvariables overwrite = yes
� � /break = type
� � /typemaxprice = max(price).
compute price.pct.maxprice = 100*price/typemaxprice.
rank variables = price resale price.pct.maxprice by type.

On 12/16/2011 4:55 PM, Jara Kampmann wrote:

Art,

I would like to be able to perform via syntax:
get file="....sav". /*get a file.
flip. /*perform any change to the file.
"save file." /*save changes into the active data set, just as in GUI using the "disket"-Button.

The rest of my earlier posting was to explain why I would like to be able to do this despite the risks connected to it, but that is not important. So, for special occasions, I do want to overwrite my data via syntax.

�

----- Ursprüngliche Nachricht -----

Von: Art Kendall

Gesendet: 16.12.11 22:25 Uhr

An: [hidden email]

Betreff: Re: Overwriting a dataset

I am not sure I understand what you would like to do.? Please give us more detail.

Art Kendall Social Research Consultants

On 12/16/2011 3:49 PM, Jara Kampmann wrote:

I totally agree with all the warnings and the precautions one should take.
Nevertheless, I am wondering whether there is a possibility to save changes into the active data set via syntax. My annoyance is: if I create an outfile with aggregate, get the file and perform some changes, I always have to create a new file (with new data set name) to be able to save the changes. Which causes additional work of rewriting when having syntax to be run on several data sets as well as the need for more storing capacity.

So does one of you know of any syntax snippet that would do the trick (if necessary)?

Jara

?

----- Ursprüngliche Nachricht -----

Von: Jon K Peck

Gesendet: 16.12.11 21:13 Uhr

An: [hidden email]

Betreff: Re: Overwriting a dataset

Apropos of which, I would remind people that Statistics has a feature to set file permissions.

Using SAVE, you can include /PERMISSIONS=READONLY
or you can use the PERMISSIONS command:
PERMISSIONS FILE=filespec /PERMISSIONS READONLY.

If you are using C & DS (Collaboration and Deployment Services), if you save to the repository, previous generations of the file are preserved, so ? you can go ? back to earlier versions. ? That works for syntax and other file types, too. ? Preserving previous versions of your syntax file can be as important as preserving your data.

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
new phone: 720-342-5621

From: ? ? ? ? Art Kendall [hidden email]
To: ? ? ? ? [hidden email]
Date: ? ? ? ? 12/16/2011 01:01 PM
Subject: ? ? ? ? Re: [SPSSX-L] Overwriting a dataset
Sent by: ? ? ? ? "SPSSX(r) Discussion" [hidden email]

I have seen things like this happen dozens of times over the years. ? Losing 3 hours is a pain. ? The worse situation I have seen was when someone wiped her data out with an "IF (something impossible);" in SAS. (SAS's version of SELECT IF). ? This was an hour after she shredded the paper copy of her questionnaires to protect privacy. ? Two years work gone.

In 1971 after a couple of weeks experience, I realized that computers do what they are told. Which is often not what you want them to do.
Art Kendall Social Research Consultants

On 12/16/2011 2:06 PM, David Marso wrote:
One should *NEVER* overwrite data unless one can recreate it from original sources or some well defined documented processing milestone. ? Sometimes accidents happen and it can be *VERY EXPENSIVE* to recreate a crucial data set. ? Once upon a time in a land far far away I was working on a project with 4 other consultants. ? We had one shared dataset which was built from several other files after sorts/merges/complex flag calculations etc... All in all it took about 3 hours to build the master file from source data using the complex data preparation syntax. ? It took about an HOUR to sort the damned thing (500K cases, 400+ variables .. back in mid 90's when EVERYTHING was *SLOOOWWW*). One afternoon I grabbed the data, ran a bit of code for the part I was working on and was QUITE distressed to find there were only 10000 cases in the file. ? Turns out that one of the team members had closed out of SPSS and clicked the F'ing button asking Save changes before closing.... Consequently 4 people were sitting on their a$$es for 3 hours while I rebuilt the file from source files. After that, EVERYTHING got locked down (set ALL file permissions to be READ only). Art Kendall wrote

Dangerous!  If you overwrite a dataset will you ? ? ? still be able to go back and do the process over in the way you ? ? ? want versus the way you said to do it? ? ? Art Kendall Social Research Consultants ? ? On 12/16/2011 11:38 AM, paulm wrote: ? ? ? Hi, ? I am somewhat new to SPSS and I am trying to resolve an issue that I have when I run my syntax on the SPSS Server vs. running the syntax locally. Basically I am retrieving a dataset, doing some manipulations and then saving the dataset (overwriting it). ? Here is the syntax I am using: DEFINE !newdate (!positional = !TOKENS(1) / !positional = !TOKENS(1)) * RETRIEVE THE CURRENT DATASET. GET FILE=!QUOTE(!CONCAT('e:\se\',!1,'\',!2,'\Data\POST\post_data.sav')). DATASET NAME DataSet1 WINDOW=FRONT. dataset activate dataset1. dataset copy dataset2. dataset close dataset1. compute mydate=!maxdate . execute. *FILTER OUT THE LAST DATE IN THE DATASET. DATASET ACTIVATE DataSet2. FILTER OFF. USE ALL. SELECT IF (date < mydate) . EXECUTE. save outfile=!QUOTE(!CONCAT('e:\se\',!league,'\',!venue,'\Data\POST\post_data.sav')) / compressed . !ENDDEFINE. !newdate level1 level2 ? . When I run this locally the syntax works. ? However, when I run it on the SPSS Server I get the following error: "Due to contention for the specified file, the data has been saved to a file with a different name - e:\se\level1\level2\data\post\post_data_1.sav" Can anyone tell me a more efficient way to accomplish this task. -- View this message in context:
http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5080768.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to
[hidden email]? (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to
[hidden email](not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

-- View this message in context:http://spssx-discussion.1045642.n5.nabble.com/Overwriting-a-dataset-tp5080768p5081115.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com. ===================== To manage your subscription to SPSSX-L, send a message to
[hidden email](not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

?

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

�

Art Kendall
Social Research Consultants

Rich Ulrich

Re: Overwriting a dataset

In reply to this post by Dale

The professional standard that I was introduced to 30 some years
ago, in the days of mainframe computing, was to have at least two
backups (on tape, in those days), with one of them at a remote
physical location. The University of Pittsburgh computer center
provided that service in a pretty timely fashion by taking backups
regularly, and storing off-site.

I wonder - Does NIH, or other government agencies that fund
grants, lay out standards for data security of this sort?

--
Rich Ulrich

Date: Fri, 16 Dec 2011 12:08:09 -0600
From: [hidden email]
Subject: Re: Overwriting a dataset
To: [hidden email]

I tend to use (and suggest) the age-old archived main data set and then work from a “working data set” to address such cases.

Dale

[snip, previous]

Tony

Re: Overwriting a dataset

This post was updated on .

In reply to this post by Jara Kampmann-2

Hi Jara,

Irrespective of whether you know what you are doing or not, it seems possible to overwrite a data file by the following command:

save outfile = '<FileName1>' /compressed.

So, not much of a snippet, I guess.

The problem arises when the datafile <FileName1> you want to overwrite is still open. You have to close the Dataset with the name that is intended to be overwritten. So if the dataset with the filename '<FileName1>' is DataSet1, and you want it to be overwritten by the contents of DataSet2, type:

dataset close DataSet1.
dataset activate DataSet2.
save outfile = '<FileName1>' /compressed.

Happy overwriting,
Tony