Re: The elimination of items when too many students get them correct

[George Cunningham <gkc@LOUISVILLE.EDU>]

Victor,

Sure random responses will give you a bell shaped curve.  That is why that
is not a primary goal of test developers.  They are far more interrested in
item-total correlations and coefficient alphas.  Appropriate p-values are a
necessary but not sufficient condition for maximizing reliability on NRTs.
Random responses will give you a coefficient alpha of zero.


George Cunningham
University of Louisville
----- Original Message -----
From: Victor Steinbok <Victor.Steinbok@VERIZON.NET>
To: <ARN-L@LISTS.CUA.EDU>
Sent: Sunday, March 31, 2002 12:34 PM
Subject: Re: The elimination of items when too many students get them
correct


> At 10:01 PM -0500 3/29/02, George Cunningham wrote:
> >It is not the bell curve that test developers are seeking as much as
> >high item-total score correlations, which only occur when there is
> >variability in student responses, which does not happen if the item
> >is too easy.  Good item total correlations lead directly to high
> >coefficient alpha reliability which is the holy grail of the
> >developers of NRTs.
>
> This has absolutely nothing to do with intent. It is simple math. The
> aggregate of a large number of binary distributions (assuming they
> are truly binary, close to 50-50 split), will ALWAYS resemble normal
> distribution, no matter what the content, context or intent of the
> individual items is. It is truly hypocritical to claim that the
> intent is to use highly discriminating items, then feign surprise
> when the result comes out as a bell-curve. A list 100 true false
> questions to which no subject knows an answer or even simply a bubble
> sheet with 100 two-choice lines to be filled in, in the end, will
> have virtually the same distribution of "scores" as an NRT with 100
> items that have been selected for their "discrimination" value. As I
> said, this is simply math, not any kind of testing expertise. What
> "knowledge" can do is change the standard deviation on the scores,
> not the nature of the distribution.

--------------------------------------------------------------------------
To unsubscribe from the ARN-L list, send command SIGNOFF ARN-L
to LISTSERV@LISTS.CUA.EDU.