Re: Norm-Referenced Tests

["Gerald W. Bracey" <gbracey@EROLS.COM>]

I was trying to be succinct and the use of the word "genes" might have
tripped
some emotional wires.  I suggest that you read Jim Popham's article which
you
can do online via www.ascd.org

Jim merely says that there are differences among people in abilities which
are
genetically determined.  It's true.  Some items on test might tap into these
differences, just one more reason why norm-referenced tests cannot, should
not be used as indices of teacher quality.


----- Original Message -----
From: Judi Hirsch <judih@OUSD.K12.CA.US>
To: <ARN-L@LISTS.CUA.EDU>
Sent: Friday, November 05, 1999 7:27 PM
Subject: Re: Norm-Referenced Tests


> Dear Gerald,
>  I'm responding to this quote of yours:
> "3.  Kids learn stuff out of school that might show up on the test; this
> has> nothing to do with teacher effectiveness.  Relatedly, items that give
> the> dsired variability often tap into the kinds of genetic abilities that
> are
> > not easily modified by school and this, too, has nothing to do with
> teacher> effectivenss."
>     where do you get the "genetic" stuff? I agree that lots--if not most--
> of what kids learn has nothing to do with the teacher at all, let alone
> teacher effectiveness, but what makes you jump to genes as the answer?
>     what can you possibly mean? should we just not provide schools and
> educational programs for the "genetically underendowed" since it would be
a
> waste of time and simply reinstitute slavery? or perhaps we should use
> eugenics too weed them out of the gene pool altogether? what of people
like
> Stephen Hawkings? should he have died in utero?
>     Judi
>
> >
> >
> >
> > ----- Original Message -----
> > From: Sheridan, George
> ----- Original Message -----
> From: Gerald W. Bracey <gbracey@EROLS.COM>
> To: <ARN-L@LISTS.CUA.EDU>
> Sent: Friday, November 05, 1999 9:03 AM
> Subject: Re: Norm-Referenced Tests
>
>
> > I don't know why you think an item that only 10% of the students answer
is
> > useful on an NRT.  Jim makes it clear in the article you cite that 90%
and
> > 10% are equally useless.  He gives the figure of 40%-60%.  My experience
> > with a few specific tests is that the range is more like
> > 30%-70%, but you do get maximal dispersion when 50% of the takers get it
> > right.
> >
> > I think that there is some confusion in your recall of what Jim is
saying
> > which he puts in three points:
> >
> > 1.  Test makers want to sell to a national market, hence the tests can't
> be
> > tailored to specific curricula.  Since curricula do vary except in the
> most
> > general sense there therefore must be some mismatch between test and
> > curriculum.  The teachers will likely be teaching some things that
aren't
> on
> > the test and the test will be testing some things the teacher didn't
> teach,
> > ergo, the test results are not a valid measure of teaching quality.
> >
> > 2.  If the test tested what teachers taught, then too many kids would
get
> > the items right.  The items wouldn't show the needed 60-40 or 70-30
> correct
> > rates.  Therefore, items which actually do cover what teachers think is
> > important are tossed out from the test.  So this part of teaching
> > effectiveness becomes invisible because the items that measure it never
> make
> > it into the final, published test.
> >
> > 3.  Kids learn stuff out of school that might show up on the test; this
> has
> > nothing to do with teacher effectiveness.  Relatedly, items that give
the
> > dsired variability often tap into the kinds of genetic abilities that
are
> > not easily modified by school and this, too, has nothing to do with
> teacher
> > effectivenss.
> >
> >
> >
> > ----- Original Message -----
> > From: Sheridan, George <gsheridan@BOMUSD.EDCOE.K12.CA.US>
> > To: <ARN-L@LISTS.CUA.EDU>
> > Sent: Friday, November 05, 1999 11:25 AM
> > Subject: Norm-Referenced Tests
> >
> >
> > > Gerald Bracey wrote:
> > >
> > > >To George Sheridan:  I would like a specific citation for the Ed
> > > >Leadership article because I don't fully understand the point.
> > >
> > >      Why Standardized Tests Don't Measure Educational
> > >      Quality, W. James Popham, in Educational Leadership, March 1999
> > >
> > > Popham makes several related points,  but the one I intended to relay
> had
> > > to do with core knowledge (as defined by the practice of most teachers
> in
> > > schools throughout the nation). In almost all classes, such material
may
> > be
> > > taught to the point of mastery and in many it may represent the bulk
of
> > > instructional time. If I understood Popham correctly, this core
> knowledge
> > > will be represented by a much smaller proportion of the tested items
> > > because student scores will not vary substantially.
> > >
> > > My two questions were:
> > >
> > > Is it accurate to say that, to makers of norm-referenced tests, items
> only
> > > a few students can answer are useful ?
> > >
> > > Is it accurate to say that when too many students can answer an item,
it
> > is
> > > not useful to makers of norm-referenced tests?
> > >
> > > You replied:
> > >
> > > >neither items that too few students answer correctly nor
> > > >items that too many students get right are useful to the makers of
> NRT's.
> > > <SNIP>
> > > >...let's ask questions that we expect 90% of people to
> > > >know; questions that we expect only 10% of people to know: and
> questions
> > > >that 50% of the people know (the typical value on an NRT).
> > >
> > > My understanding of your response is that less than 10% would be "too
> > few,"
> > > but if only 10% of students can answer a question correctly,
NRT-makers
> > > might consider that a useful question. Your answer to the second
> question
> > > is that if more than 90% of students can answer a question correctly,
it
> > is
> > > not useful to the makers of norm-referenced tests.
> > >
> > > ORIGINAL MESSAGE
> > > ****************
> > >
> > > >>Is it accurate to say that, to makers of norm-referenced tests,
items
> > > only a few students can answer are useful ?
> > >
> > > >>Is it accurate to say that when too many students can answer an
item,
> it
> > > is not useful to makers of norm-referenced tests?
> > >
> > > >>One of the criticisms of these tests in a recent article in Ed
> > Leadership
> > > >>was that the tests underemphasize items representing the knowledge
> > > >>teachers teach most successfully. If nearly all teachers
successfully
> > > >>impart certain "core" knowledge, that knowledge would be represented
> by
> > a
> > > >>very small part of the items on a norm-referenced test, since
> otherwise
> > > >>test-makers could not "spread out" student scores.
> > >
> > > I apologize for having expressed myself with insufficient clarity,
> forcing
> > > you to guess at what I meant. My original post was meant to ascertain
a
> > > factual basis for Judi Hirsch's assertion about test items being
> discarded
> > > if too many students can answer them correctly.
> > >
> > >
> > >
> > >
> > >
> > > George Sheridan
> > > Black Oak Mine Unified School District
> > > Northside School
> > > P.0. Box 217
> > > Cool, CA 95614
> > >
> > > (530) 333 4506
> > >
> > > Hope is ... not the conviction that something will turn out well, but
> the
> > > certainty that something makes sense, regardless of how it turns out.
> > >
> > > Vaclav Havel
> > >
> >
>
> --------------------------------------------------------------------------
> > > To unsubscribe from the ARN-L list, send command SIGNOFF ARN-L
> > > to LISTSERV@LISTS.CUA.EDU.
> > >
> >
>
> --------------------------------------------------------------------------
> > To unsubscribe from the ARN-L list, send command SIGNOFF ARN-L
> > to LISTSERV@LISTS.CUA.EDU.
>
> --------------------------------------------------------------------------
> To unsubscribe from the ARN-L list, send command SIGNOFF ARN-L
> to LISTSERV@LISTS.CUA.EDU.

--------------------------------------------------------------------------
To unsubscribe from the ARN-L list, send command SIGNOFF ARN-L
to LISTSERV@LISTS.CUA.EDU.