Re: bias in tests

[Peter Campbell <campbellp@mail.montclair.edu>]

> <x-tad-bigger>From: George Hein </x-tad-bigger>
>
> <x-tad-bigger>But, he essential point is that there is no objective way of selecting questions in the standard format of current testing regimes.  That's one little realized crucial flaw of psychometrics.  The practitioners of this art can provide exquisitely detailed analyses of the validity and reliability of items compared to to other items and tests, but the very choice of any item in the first pace is the issue. That they are "aligned with the standards" is simply a statement.  No one, to my knowledge,  ever provides any empirical evidence, based on experimental work that would let me know what "alignment" means in a quantitative way.   Items are generated by a process that has no theoretical basis and no experimental base that adheres to the kind of "scientific" rigor that  the tests are supposed to exemplify.
> </x-tad-bigger>


I posted this earlier, but thought it was relevant again here. I think we need to expose what "aligning standardized tests to state standards" actually means, esp. in light of George Hein's comments.

Here's a real question from Tennessee's 2005 state test from Governance and Civics section given to students in Grade 3. It's meant to measure the extent to which students have met this standard: "3.4.spi.2 Determine the representative acts of a  good citizen (i.e., obeying speed limit, not littering,  walking within the crosswalk)."

"Which of these is an example of someone being a good citizen?"

a) A girl steals candy from a store.
b) A boy puts his litter in a trash can.
c) A man lets his dog run loose on the street.
d) A woman drives faster than the speed limit.

So, presumably those third graders in Tennessee who chose Answer B - " A boy puts his litter in a trash can." - are now able to determine the representative acts of a  good citizen. The most offensive aspect of this is that measuring citizenship is reduced to a multiple choice question that students either get right or wrong. In addition, the students could have easily ruled out the other choices as being obviously wrong and were left with only one answer - B. So what this means is that students may not know what "the representative acts of a  good citizen" ARE - they simply know what they are NOT. Of course, because these tests are standards-based, Tennessee officials can sleep at night (and get re-elected), knowing they have definitive, psychometrically-backed proof that their state's 3rd graders are good citizens.

Not.

Peter C.