SPSS error: regression coefficient/standard error does not fit with significance level

[jules2013]

I have noticed a major problem with the results of some linear regression and ordinal regression analyses that were computed with SPSS 14-17 some years ago. There is a really strange error. The regression coefficients and standard errors do not fit with the significance levels for some independent variables.
 
Do you have any information about a SPSS bug or other error? There were two different versions of SPSS on the same machine (e.g., 15 and 16), and no service packs of SPSS were installed at that time. Any suggestion would be much appreciated!

[Bruce Weaver]

Please provide specific examples of the problem.

jules2013 wrote

I have noticed a major problem with the results of some linear regression and ordinal regression analyses that were computed with SPSS 14-17 some years ago. There is a really strange error. The regression coefficients and standard errors do not fit with the significance levels for some independent variables.
 
Do you have any information about a SPSS bug or other error? There were two different versions of SPSS on the same machine (e.g., 15 and 16), and no service packs of SPSS were installed at that time. Any suggestion would be much appreciated!

[jules2013]

Here is an example: the following is the wrong result for one of several independent variables in an OLS regression.

Unstandardized regression coefficient: .37
Standard error: .31
Significance: .05

It is mathematically impossible. At least one of the three numbers is wrong. There are missing values for some other variables, and pairwise exclusion is used, but that should not be an issue...

[Rich Ulrich]

I certainly agree that those values are inconsistent.  That kind of
problem - which you report as coming from multiple machines -
certainly would have been noticed an immediately fixed, and not,
I think, without some notoriety.   No one has jumped in with an
explanation here.

I was following SPSS on the Usenet group and not the List a few
years ago,  and I don't remember it from there, either.

So - the likely explanation is what I have discovered a few times
in my experience as consultant:  The printout is being mis-read.
Either (a) that standard error does not go with that statistic, or (b) that
p-value is not a p-value or (c) that p-value is not the value for that test.
(How sure are you?)

Assuming that the problem is noted on a printout, the easiest solution
is to find your own nearby expert to consult.  Sometimes, a non-expert
can help, just by providing new and critical eyes.  Otherwise, you might
scan the listing and upload it to somewhere that the rest of us can see
it.   Nabble?

--
Rich Ulrich

> Date: Fri, 15 Feb 2013 05:50:23 -0800

> From: [hidden email]
> Subject: SPSS error: regression coefficient/standard error does not fit with significance level
> To: [hidden email]
>
> I have noticed a major problem with the results of some linear regression and
> ordinal regression analyses that were computed with SPSS 14-17 some years
> ago. There is a really strange error. The regression coefficients and
> standard errors do not fit with the significance levels for some independent
> variables.
>
> Do you have any information about a SPSS bug or other error? There were two
> different versions of SPSS on the same machine (e.g., 15 and 16), and no
> service packs of SPSS were installed at that time. Any suggestion would be
> much appreciated!
>

[snip, rest]
FROM YOUR OTHER POST

Here is an example: the following is the wrong result for one of several
independent variables in an OLS regression.
 
Unstandardized regression coefficient: .37
Standard error: .31
Significance: .05
 
It is mathematically impossible. At least one of the three numbers is wrong.
There are missing values for some other variables, and pairwise exclusion is
used, but that should not be an issue...
 

[jules2013]

Unfortunately, the SPSS output is not available anymore. The results have been transferred to Excel, and the Excel sheet this is the only thing that I have.

I can basically exclude any misreading of the output. The problems occured on one computer, but with multiple versions of SPSS on that computer. If you have any further ideas concerning the origin of the errors, it would be most appreciated!

By the way, is there any public list of errors of different SPSS versions?


<quote author="Rich Ulrich-2">
I certainly agree that those values are inconsistent.  That kind of
problem - which you report as coming from multiple machines -
certainly would have been noticed an immediately fixed, and not,
I think, without some notoriety.   No one has jumped in with an
explanation here.

I was following SPSS on the Usenet group and not the List a few
years ago,  and I don't remember it from there, either.

So - the likely explanation is what I have discovered a few times
in my experience as consultant:  The printout is being mis-read.
Either (a) that standard error does not go with that statistic, or (b) that
p-value is not a p-value or (c) that p-value is not the value for that test.
(How sure are you?)

Assuming that the problem is noted on a printout, the easiest solution
is to find your own nearby expert to consult.  Sometimes, a non-expert
can help, just by providing new and critical eyes.  Otherwise, you might
scan the listing and upload it to somewhere that the rest of us can see
it.   Nabble?

--
Rich Ulrich

[Kornbrot, Diana]

Is there any simple code to obtain partial eta square for predictors in procedure mixed?
Help much appreciated
Best
Diana

[Bruce Weaver]

Does "transferred to Excel" mean exported to Excel?  (That could be done for older versions of SPSS using a basic script.)  I ask, because if there were manual steps involved in transferring results to Excel, errors could easily have occurred.  If it was truly "exported", it would be useful for others to see the output.  As Rich noted, one can attach files when posting to this list via Nabble.   If you don't use Nabble, you could send me the file (using the e-mail address in my sig file below), and I'll upload it.

HTH.

<quote author="jules2013">
Unfortunately, the SPSS output is not available anymore. The results have been transferred to Excel, and the Excel sheet this is the only thing that I have.

I can basically exclude any misreading of the output. The problems occured on one computer, but with multiple versions of SPSS on that computer. If you have any further ideas concerning the origin of the errors, it would be most appreciated!

By the way, is there any public list of errors of different SPSS versions?

Rich Ulrich-2 wrote

I certainly agree that those values are inconsistent.  That kind of
problem - which you report as coming from multiple machines -
certainly would have been noticed an immediately fixed, and not,
I think, without some notoriety.   No one has jumped in with an
explanation here.

I was following SPSS on the Usenet group and not the List a few
years ago,  and I don't remember it from there, either.

So - the likely explanation is what I have discovered a few times
in my experience as consultant:  The printout is being mis-read.
Either (a) that standard error does not go with that statistic, or (b) that
p-value is not a p-value or (c) that p-value is not the value for that test.
(How sure are you?)

Assuming that the problem is noted on a printout, the easiest solution
is to find your own nearby expert to consult.  Sometimes, a non-expert
can help, just by providing new and critical eyes.  Otherwise, you might
scan the listing and upload it to somewhere that the rest of us can see
it.   Nabble?

--
Rich Ulrich

[Jon K Peck]

I have searched through our bug database back through V16 and checked with our QA person who manages the backend test bed, and there is no sign of a bug anything like what is described below in either development or released versions.  If there had been something like that found in the normal SPSS QA process or reported through Technical Support, it would have set off alarm bells all over development.

Furthermore, I ran some regressions in V17 and exported the results to Excel without seeing any problems.


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
[hidden email]
new phone: [phone]




From:        jules2013 <[hidden email]>
To:        [hidden email],
Date:        02/17/2013 12:28 PM
Subject:        Re: [SPSSX-L] SPSS error: regression coefficient/standard error              does not              fit with significance level
Sent by:        "SPSSX(r) Discussion" <[hidden email]>

---

Unfortunately, the SPSS output is not available anymore. The results have
been transferred to Excel, and the Excel sheet this is the only thing that I
have.

I can basically exclude any misreading of the output. The problems occured
on one computer, but with multiple versions of SPSS on that computer. If you
have any further ideas concerning the origin of the errors, it would be most
appreciated!

By the way, is there any public list of errors of different SPSS versions?



I certainly agree that those values are inconsistent.  That kind of
problem - which you report as coming from multiple machines -
certainly would have been noticed an immediately fixed, and not,
I think, without some notoriety.   No one has jumped in with an
explanation here.

I was following SPSS on the Usenet group and not the List a few
years ago,  and I don't remember it from there, either.

So - the likely explanation is what I have discovered a few times
in my experience as consultant:  The printout is being mis-read.
Either (a) that standard error does not go with that statistic, or (b) that
p-value is not a p-value or (c) that p-value is not the value for that test.
(How sure are you?)

Assuming that the problem is noted on a printout, the easiest solution
is to find your own nearby expert to consult.  Sometimes, a non-expert
can help, just by providing new and critical eyes.  Otherwise, you might
scan the listing and upload it to somewhere that the rest of us can see
it.   Nabble?

--
Rich Ulrich




--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/SPSS-error-regression-coefficient-standard-error-does-not-fit-with-significance-level-tp5718081p5718107.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

[Ryan]

For a fixed effects only model, or including random effects? Please post code or describe the model.

On Sun, Feb 17, 2013 at 2:16 PM, Kornbrot, Diana <[hidden email]> wrote:

Is there any simple code to obtain partial eta square for predictors in procedure mixed?
Help much appreciated
Best
Diana

[Ryan]

Even though I did not receive a response to my question, my guess is that some people on this list might be interested in calculating partial eta squared from a linear mixed model. For this post, I will demonstrate how to estimate partial eta squared for a fully balanced fixed effects design using two approaches.

 

First, let's generate some data:

*Generate Data.

 

set seed 98765432.
new file.

inp pro.

loop ID= 1 to 1000.

     if (ID<=500) FactorA=0.
     if (ID>500 and ID<=1000) FactorA=1.
     if (ID<=250 | ID>750) FactorB=0.
     if (ID>250 and ID<=750) FactorB=1.
     comp #b0 = -1.5.
     comp #b1 = 0.9.
     comp #b2 = 0.5.
     comp #error = rv.normal(0,1).
     comp y  = #b0 + #b1*FactorA + #b2*FactorB + #error.

     end case.
   end loop.
end file.
end inp pro.
exe.

 

CROSSTABS
  /TABLES=FactorB BY FactorA
  /FORMAT=AVALUE TABLES
  /CELLS=COUNT ROW COLUMN
  /COUNT ROUND CELL.

 

UNIANOVA y BY FactorA FactorB
  /METHOD=SSTYPE(3)
  /INTERCEPT=INCLUDE
  /PRINT=ETASQ
  /CRITERIA=ALPHA(.05)
  /DESIGN=FactorA FactorB.

 

MIXED y BY FactorA FactorB
  /FIXED=FactorA FactorB | SSTYPE(3)
  /METHOD=REML.

 

*Calculate partial-eta squared from MIXED results using GLM partial-eta squared formula:
*Partial eta squared Factor a = Fa*DFa / (Fa*DFa + DFerror)
*Partial eta squared Factor b = Fb*DFb / (Fb*DFb + DFerror).

 

compute partial_eta_squared_a_GLM_formula =258.55739235480064*1 / (258.55739235480064*1 + 997) .
compute partial_eta_squared_b_GLM_formula = 77.49055195297237*1 / (77.49055195297237*1 + 997). 
execute.

 

*Calculate partial eta squared from MIXED results using using linear mixed model formula:
*Proportion variance explained = 1 - (Full model residual variance  / Reduced model residual variance)
*Note: Models employed below are based on ML estimation method.

 

*Fit full model.

MIXED y BY FactorA FactorB
  /FIXED=FactorA FactorB | SSTYPE(3)
  /METHOD=ML.

 

*Fit reduced model to obtain residual variance to calculate partial eta squared for Factor A.

MIXED y BY FactorB
  /FIXED=FactorB | SSTYPE(3)
  /METHOD=ML.

 

compute partial_eta_squared_a_Mixed_formula = 1 - (.9851997823936827 / 1.2406969606125673).
execute.
 
*Fit reduced model to obtain residual variance to calculate partial eta squared for Factor B.

MIXED y BY FactorA
  /FIXED=FactorA | SSTYPE(3)
  /METHOD=ML.

 

compute partial_eta_squared_b_Mixed_formula =  1 - (.9851997823936827 / 1.0617731775006374).
execute.

Best,

 

Ryan

On Tue, Feb 19, 2013 at 2:52 PM, R B <[hidden email]> wrote:

For a fixed effects only model, or including random effects? Please post code or describe the model.

On Sun, Feb 17, 2013 at 2:16 PM, Kornbrot, Diana <[hidden email]> wrote:

Is there any simple code to obtain partial eta square for predictors in procedure mixed?
Help much appreciated
Best
Diana