Re: did anyone see frontline?

[Victor Steinbok <Victor.Steinbok@VERIZON.NET>]

At 9:28 AM -0800 3/29/02, Art Burke wrote:
>  >>> kber@EARTHLINK.NET 03/29 9:22 AM >>>
>
> Teresa or J Glenn wrote:  There was a great teacher (missed her name)
> who talked about how tests in many states eliminate questions that "too
> many" kids get right in order to create a Bell curve... thought that was
> a pretty powerful point.
> ____________________________
> They also eliminate questions that too many kids get wrong, which
> leaves, like the beds of the three bears, the ones that are just
> about right.
>
> Art

This is truly idiotic even for you, Art. What's "just right"? 30%
passing rate? 50%? All that's left are questions with between 30 and
60% passing rate. There is only one possible purpose these items can
serve--to rank people.

This is the basic theory behind the NRTs. If you gather a bunch of
questions that close to half the subjects can answer, you get a near
normal distribution.  In reality, the questions don't have to be on
the same subject or in any way valid--as long as the distribution on
each question is "about right", the distribution of the total score
will come out on a bell curve.

Of course, this becomes silly. On a "standards-based" test or a CRT
you want at least some questions that most if not all subjects can
answer. Thus, we should be able to find at least some items with 80%,
90% passing rates. These items, of course, would have very low
discrimination value--they will be answered consistently by
low-scoring as well as high-scoring subjects. But if you are simply
verifying what students know, this is what you want to test. However,
if all you want to do is rank, then you don't want these items
because they "inflate" the low scores. The MCAS pretends to be a CRT.
However, as Walt Haney pointed out in a recent paper, all the items
that had a trial passing rate above 60% have been eliminated and
never offered on the real test. The people building the test either
don't have a clue about the difference between the CRT and NRT or
don't care.

       VS-)

--------------------------------------------------------------------------
To unsubscribe from the ARN-L list, send command SIGNOFF ARN-L
to LISTSERV@LISTS.CUA.EDU.