Re: Lauren Resnick and higher-order thinking skills

[aburke5054@aol.com]

 I fail to see either tragic or comic dimensions to presenting third graders with one test item that associates good citizenship with the proper disposal of trash. And while I share wholeheartedly your belief that test items magically shape all of children's experience, I doubt that very many Tennessee kids will grow up thinking that tossing trash in the trash can is all there is to good citizenship. 
 
 Art 
  
 -----Original Message-----
 From: campbellp@mail.montclair.edu
 To: arn-l@interversity.org
 Sent: Sat, 27 Jan 2007 12:08 PM
 Subject: Re: [arn-l] Lauren Resnick and higher-order thinking skills
 
  Just wanted to add a couple post-scripts to this: 
 
 1) What makes the example I gave below from Tennessee's 2005 state test in Governance and Civics both tragic and comic is that it's intended -- with no irony or humor involved -- to measure the extent to which 3rd graders have met the following standard: 
 
 "3.4.spi.2 Determine the representative acts of a good citizen (i.e., obeying speed limit, not littering, walking within the crosswalk)." 
 
 So, presumably those third graders in Tennessee who chose Answer B - " A boy puts his litter in a trash can." - are now able to determine the representative acts of a good citizen. The most offensive aspect of this is that measuring citizenship is reduced to a multiple choice question that students either get right or wrong. In addition, the students could have easily ruled out the other choices as being obviously wrong and were left with only one answer - B. So what this means is that students may not know what "the representative acts of a good citizen" ARE - they simply know what they are NOT. Of course, because these tests are standards-based, Tennessee officials can sleep at night (and get re-elected), knowing they have definitive, psychometrically-backed proof that their state's 3rd graders are good citizens. 
 
 2) The assumption/assertion about the MCT's that assess higher-order thinking is that the results show "general causal relationships in a convincing manner" is misleading because it overlooks the experience of taking these tests, i.e., that students either stress out, over-think, etc., while taking them or they don't. This means that the results must first be analyzed through this lens. In other words, you would have to be able to measure the students' affective response to test-taking first and then, for those students who have a negative affective response, be able to account for that in some way. Personally, the way I would account for that is to throw out the results and look at other measures. For those students who had a positive or neutral affective response to test-taking, you would then have to determine the extent to which this affective disposition to test-taking skewed the results and, ultimately, make these students seem "smarter" than they might actually be -- and certainly much, much "smarter" than the students who have a negative affective disposition. 
 
 Have any studies ever been done that controlled for this affective disposition to test-taking? 
 
 The only thing that I know of that comes close is the studies that Claude Steele did RE: "stereotype threat." (Here's a quick overview here: http://www.pbs.org/wgbh/pages/frontline/shows/sats/interviews/steele.html) 
 
 What's interesting in light of this research is that minority kids might actually deal with a double-dose of affective dispositions to test taking that negatively affect their results, i.e, they might -- independent of their race -- feel apprehensive and anxious about taking tests and not test well AND also experience this "stereotype threat." 
 
 Peter C. 
 
 On Jan 27, 2007, at 12:10 PM, Peter Campbell wrote: 
 
 > For example, here's a real question from Tennessee's 2005 state test > in Governance and Civics given to students in Grade 3: 
 > 
 > "Which of these is an example of someone being a good citizen?" 
 > 
 > a) A girl steals candy from a store. 
 > b) A boy puts his litter in a trash can. 
 > c) A man lets his dog run loose on the street. 
 > d) A woman drives faster than the speed limit. 
 
 > You argue in response to the question, "Why MCT's?", "So that the > tests can be given to thousands of students in hundreds of courses > under varying conditions in such a manner that meta-analyses can be > performed, thus establishing general causal relationships in a > convincing manner." I'm not convinced by anything other than the fact > that the students who did well were good at taking MCT's. 
 
 ------------------------------------------------ 
 Direct list questions to listmom@interversity.net 
   
________________________________________________________________________
Check out the new AOL.  Most comprehensive set of free safety and security tools, free access to millions of high-quality videos from across the web, free AOL Mail and more.