Help, need feedback

[Debra Caywood-Rukas]

I am a doctoral student trying to pull together my diss topic and how to analyze my data.  It's been a few years since my Stats courses and would like input.

I my looking to see if 2 different sets of reading assessment (IV's)  administered in Kindergarten and 1st grade (categorical) are predictive of scores received on tests given in 3rd ( categorical) and 4th grade (continuous: percentiles, but can convert to categorical), DV's . Can I keep continuous or must I convert to categorical?

My data are longitudinal and consists of 153 students who took the 1st grade assessments as well as the 3rd and 4th grade tests and a subset of that group are  64 students who took the Kindergarten and 1st grade assessment as well as the 3rd and 4th grade tests.

I am thinking that I need to analyze the data using Multiple Regression multivariate analysis. Including examining if there are any interaction effects between the 2 IV's.  Am I on the right track?  Any suggestions?

Please give me your thoughts.

Thanks,
Debra Caywood-Rukas, M.Ed., Ed. S, NCSP
Educational-School Psychologist

[paulandpen]

Hi Debra,

I see there are a few things to considers in this evaluation.  A predictor variable by my top of mind definition needs to have a 'causal' influence on a dependent variable.  I am not sure how an assessment (without some form of intervention) can be considered a predictor variable (i.e. causes scores on later tests)

Are you actually looking at the 'predictive validity' of the test over time, in other words, will scores found at the first/second point of time actually predict similar scores later on.  This is a somewhat different matter for mind, and my first thoughts are test-retest reliability.

Given this is a time-series repeated measures evaluation, you will need to factor in a time-series component to your analysis (that is each person has multiple points of data collected - test scores recorded).

I am thinking as a start that since you have categorical variables, unless I am mistaken, multiple regression is not suitable because the dv is assumed to be continuous in mr, and also that unless you use some specific adjustments to the analysis (time-series regression), results will be distorted because you have repeated measures.

As a start for consideration, I am thinking a repeated measures GLM would be useful (assuming the algorithm is robust enought to handle categorical data).  The fixed variable entered into the GLM would be the group exposed to a certain reading assessment at stage 1 or 2, and the time series would be their scores (WITHIN SUBJECT FACTORS) for each of the different test periods.  At the very least, this will give you a visual representation via plots of patterns across each time period.

Regards Paul

> Debra Caywood-Rukas <[hidden email]> wrote:
>
> I am a doctoral student trying to pull together my diss topic and how to
> analyze my data.  It's been a few years since my Stats courses and would
> like input.
>
> I my looking to see if 2 different sets of reading assessment (IV's)
> administered in Kindergarten and 1st grade (categorical) are predictive
> of scores received on tests given in 3rd ( categorical) and 4th grade
> (continuous: percentiles, but can convert to categorical), DV's . Can I
> keep continuous or must I convert to categorical?
>
> My data are longitudinal and consists of 153 students who took the 1st
> grade assessments as well as the 3rd and 4th grade tests and a subset of
> that group are  64 students who took the Kindergarten and 1st grade
> assessment as well as the 3rd and 4th grade tests.
>
> I am thinking that I need to analyze the data using Multiple Regression
> multivariate analysis. Including examining if there are any interaction
> effects between the 2 IV's.  Am I on the right track?  Any suggestions?
>
> Please give me your thoughts.
>
> Thanks,
> Debra Caywood-Rukas, M.Ed., Ed. S, NCSP
> Educational-School Psychologist

[statisticsdoc]

Debra,

A few initial thoughts and questions:

1. I assume that your categorical variables are ordinal - like rubric scores
or grades.  Is this correct?  If you are predicting outcomes on an ordinal
variable, it would be helpful to use the SPSS PLUM package
( see http://www.norusis.com/pdf/ASPC_v13.pdf ).

2. It strikes me that the 3rd grade and 4th grade assessments are on
fundamentally different metrics (ordinal and interval).
This is even more so if the 3rd grade scores are rubric scores that are
anchored on a criterion-referenced scoring system, rather a norm-referenced
one.  You can conduct separate analyses for each outcome, of course.

3. Be very careful about collinearity in the predictors - there is probably
a lot of this with the test scores.

All for now - must dash - hope this helps,

Stephen Brand

For personalized and professional consultation in statistics and research
design, visit
www.statisticsdoc.com


-----Original Message-----
From: SPSSX(r) Discussion [mailto:[hidden email]]On Behalf Of
Debra Caywood-Rukas
Sent: Monday, January 29, 2007 7:21 PM
To: [hidden email]
Subject: Help, need feedback


I am a doctoral student trying to pull together my diss topic and how to
analyze my data.  It's been a few years since my Stats courses and would
like input.

I my looking to see if 2 different sets of reading assessment (IV's)
administered in Kindergarten and 1st grade (categorical) are predictive of
scores received on tests given in 3rd ( categorical) and 4th grade
(continuous: percentiles, but can convert to categorical), DV's . Can I keep
continuous or must I convert to categorical?

My data are longitudinal and consists of 153 students who took the 1st grade
assessments as well as the 3rd and 4th grade tests and a subset of that
group are  64 students who took the Kindergarten and 1st grade assessment as
well as the 3rd and 4th grade tests.

I am thinking that I need to analyze the data using Multiple Regression
multivariate analysis. Including examining if there are any interaction
effects between the 2 IV's.  Am I on the right track?  Any suggestions?

Please give me your thoughts.

Thanks,
Debra Caywood-Rukas, M.Ed., Ed. S, NCSP
Educational-School Psychologist