[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: test invalidity

I was surprised to see that these sort of items were included on a
practice test. I have done a lot of test development and been on
committees screening items and it has always been my experience that the
reviewers are hyper sensitive to any item that gives the impression that
it would be unfair to any group for any reason. No one has a reason to
want to such items included and they just cause all sort of PR problems.
Therefore, any dubious items are eliminated. I can't believe such
obviously questionable items would be included on an actual test.

There are also statistical methods for identifying biased items, which
essentially flag items that are more difficult for some subgroup than
the sample as a whole. The frustrating thing is that often items that
appear to be biased are not, based on statistical indicators, and items
that seem entirely innocuous are statistically biased.

George K. Cunningham
Professor, Ret.

>>> monty@fairtest.org 01/13/06 10:08 AM >>>
How about the items in the NY test that are clearly culturally based
(the Palast piece)? If there were a balance in which questions more
likely to relate to the experiences of low-income inner city kids were
also on the test, that would be one thing. I have not seen that test,
but I'd bet there are no such items.

Ok, this was a practice test. Maybe the final is better scrutinized for
this sort of bias - test makers started doing that after pressure in the
60s and 70s. But reviews of tests that I have been part of that have
included African Americans, for example, usually end up with them noting
both some subtle biases in favor of the experiences of the white and
affluent and no such biases tilt the other way.

Susan Ohanian has the piece I am referring to on her website:

No Child's Behind Left: The Test
Greg Palast

Greg Palast provides shocking questions from a New York practice test
for third graders.
Monty Neill, Ed.D.
Executive Director
342 Broadway
Cambridge, MA 02139
617-864-4810 fax 617-497-2224