design question

[Zdaniuk, Bozena-3]

If I want to test impact of new math teaching method on kids' learning i can do one of two designs:

1. I pick 20 classes and use the new method in ten of them and teaching as usual in the other ten. I do pretest and post-test and then compare the change scores between the two groups of classes (kids)

2. I pick 20 classes and use new method for HALF of the students of each class and teaching as usual for the other half of students in the class. I do pretest and post-test and compare the change scores between the two groups of kids. 

Is one design preferred to another? My colleague claims that the second design allows me to not worry about the effect of the classroom. What do people think? Any reference to written discussions on the topic are appreciated.

bozena 

[Art Kendall]

WRT the second design.  With today's software why not build classroom into the analysis.

Is it really practical to split a class into two methods of teaching?
On 9/29/2013 1:00 AM, Zdaniuk, Bozena-3 [via SPSSX Discussion] wrote:

If I want to test impact of new math teaching method on kids' learning i can do one of two designs:

1. I pick 20 classes and use the new method in ten of them and teaching as usual in the other ten. I do pretest and post-test and then compare the change scores between the two groups of classes (kids)

2. I pick 20 classes and use new method for HALF of the students of each class and teaching as usual for the other half of students in the class. I do pretest and post-test and compare the change scores between the two groups of kids. 

Is one design preferred to another? My colleague claims that the second design allows me to not worry about the effect of the classroom. What do people think? Any reference to written discussions on the topic are appreciated.

bozena 

---

If you reply to this email, your message will be added to the discussion below:

http://spssx-discussion.1045642.n5.nabble.com/design-question-tp5722312.html

To start a new topic under SPSSX Discussion, email [hidden email]
To unsubscribe from SPSSX Discussion, click here.
NAML

[Andy W]

I agree with Art. The second option (randomize within classrooms) doesn't per se get rid of the classroom effects anyway. (Classroom effects could be from teachers, or certain people self-selecting into classrooms, or contamination between students among others I'm sure.) So I'm not sure what you gain from it besides head-aches in implementation. (On-line classes it seems doable, in person off-hand it seems very difficult.)

Also two presents a confound with the status quo of teaching that could have undesirable effects. Say for instance both experiment and control do worse than prior data would suggest. It could be from splitting the class in half it causes adverse effects.

I'd suggest picking up any book on multi-level modelling. Experiments of this kind were one of the main motivations for development of the field. Controlling for classroom isn't difficult with current software (as Art mentioned), and it can potentially add useful insight into the experimental analysis. (It certainly is more true to how it would be implemented in practice anyway.)

[Ryan]

I also do not see how students in the same physical class could be taught different methods.  

A variation I would consider would be to have the same teachers administer both methods to different classes. Such a design would allow you to evaluate the teacher-BY-method interaction effect. You would then likely have a greater risk for diffusion of methods. 

I agree that you ought to learn about and/or consult with an expert in research designs for hierarchical designs. 

Ryan

On Sun, Sep 29, 2013 at 7:27 AM, Art Kendall <[hidden email]> wrote:

WRT the second design.  With today's software why not build classroom into the analysis.

Is it really practical to split a class into two methods of teaching?

Art Kendall
Social Research Consultants

On 9/29/2013 1:00 AM, Zdaniuk, Bozena-3 [via SPSSX Discussion] wrote:

If I want to test impact of new math teaching method on kids' learning i can do one of two designs:

1. I pick 20 classes and use the new method in ten of them and teaching as usual in the other ten. I do pretest and post-test and then compare the change scores between the two groups of classes (kids)

2. I pick 20 classes and use new method for HALF of the students of each class and teaching as usual for the other half of students in the class. I do pretest and post-test and compare the change scores between the two groups of kids. 

Is one design preferred to another? My colleague claims that the second design allows me to not worry about the effect of the classroom. What do people think? Any reference to written discussions on the topic are appreciated.

bozena 

---

If you reply to this email, your message will be added to the discussion below:

http://spssx-discussion.1045642.n5.nabble.com/design-question-tp5722312.html

To start a new topic under SPSSX Discussion, email [hidden email]
To unsubscribe from SPSSX Discussion, click here.
NAML

Art Kendall
Social Research Consultants

---

View this message in context: Re: design question
Sent from the SPSSX Discussion mailing list archive at Nabble.com.

[Melissa Ives-2]

This makes much more sense in terms of eliminating the bias due to a specific teacher. Although it does seem that some measure of implementation would be useful since the same teacher may excel at one method and not do well at the other (e.g. s/he prefers/understands/resonates with one method over the other.)

Melissa

On Sun, Sep 29, 2013 at 2:54 PM, Ryan Black <[hidden email]> wrote:

I also do not see how students in the same physical class could be taught different methods.  

A variation I would consider would be to have the same teachers administer both methods to different classes. Such a design would allow you to evaluate the teacher-BY-method interaction effect. You would then likely have a greater risk for diffusion of methods. 

I agree that you ought to learn about and/or consult with an expert in research designs for hierarchical designs. 

Ryan

On Sun, Sep 29, 2013 at 7:27 AM, Art Kendall <[hidden email]> wrote:

WRT the second design.  With today's software why not build classroom into the analysis.

Is it really practical to split a class into two methods of teaching?

Art Kendall
Social Research Consultants

On 9/29/2013 1:00 AM, Zdaniuk, Bozena-3 [via SPSSX Discussion] wrote:

If I want to test impact of new math teaching method on kids' learning i can do one of two designs:

1. I pick 20 classes and use the new method in ten of them and teaching as usual in the other ten. I do pretest and post-test and then compare the change scores between the two groups of classes (kids)

2. I pick 20 classes and use new method for HALF of the students of each class and teaching as usual for the other half of students in the class. I do pretest and post-test and compare the change scores between the two groups of kids. 

Is one design preferred to another? My colleague claims that the second design allows me to not worry about the effect of the classroom. What do people think? Any reference to written discussions on the topic are appreciated.

bozena 

---

If you reply to this email, your message will be added to the discussion below:

http://spssx-discussion.1045642.n5.nabble.com/design-question-tp5722312.html

To start a new topic under SPSSX Discussion, email [hidden email]
To unsubscribe from SPSSX Discussion, click here.
NAML

Art Kendall
Social Research Consultants

---

View this message in context: Re: design question
Sent from the SPSSX Discussion mailing list archive at Nabble.com.