[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Why MCT's? 2nd Try


On Mar 7, 2007, at 11:44 PM, Richard Hake wrote:


2. Peter wrote:

CCCCCCCCCCCCCCCCCCCCCCCCCCC
You argue in response to the question "Why MCT's?":
"So that the tests can be given to thousands of students in hundreds of courses under varying conditions in such a manner that meta-analyses can be performed, thus establishing general causal relationships in a convincing manner. "
I'm not convinced by anything other than the fact that the students who did well were good at taking MCT's." 
CCCCCCCCCCCCCCCCCCCCCCCCCCC
Peter's reaction might well be justified for the results of most MCT evaluations. However for my own survey [Hake (1998a,b)] and subsequent confirming work by many other physics education research groups [for references see Hake (2007b)] IT WOULD NOT BE EASY TO ARGUE THAT THE APPROXIMATELY TWO-STANDARD DEVIATION SUPERIORITY IN NORMALIZED GAINS OF INTERACTIVE ENGAGEMENT (IE) OVER TRADITIONAL (T) COURSES, WAS DUE TO THE FACT THAT STUDENTS IN THE IE COURSES JUST HAPPENED TO BE A LOT BETTER AT TAKING MCT'S THAN STUDENTS IN THE T COURSES.
Richard - You're arguing that students who were taught using "interactive engagement" scored better on MCT's than students who were taught using "traditional" course methods. You use this as proof that MCT's reflect the superiority of "interactive engagement" over "traditional" course methods. Is that a fair summary? 

If so, here are the problems I see:
1) You're establishing a causal relationship between "interactive engagement" and higher scores. 2) You're also establishing a causal relationship between "traditional" course methods and lower scores. 3) You're arguing that MCT's can reflect the superiority of "interactive engagement."
Because 1 and 2 cannot be established, you cannot, therefore, argue that MCT's can measure something that cannot be established. Here's a rather crude analogy: 

Babies fed soy milk are happy.
Babies fed breast milk are not happy.
The babies fed soy milk gained 20 pounds, whereas the breast fed babies gained only 5 pounds.
Conclusion: the 15-pound difference between the two groups shows the superiority of soy milk as a nutritional supplement for babies.
4) It's entirely possible that students who were taught using "interactive engagement" scored better on MCT's than students who were taught using "traditional" course methods due to something in the design of the "interactive engagement" curriculum/pedagogy. In other words, these students might have been better trained and prepared to take MCT's. You do not seem to control for this variable.
5) Finally, it's also entirely possible that students who were taught using "interactive engagement" scored better on MCT's than students who were taught using "traditional" course methods due to the composition of the students in the "interactive engagement" group. Were the students randomly selected to each group? How large were the groups? 

Peter