Re: Definition of Standardized Tests

[Nancy Patterson <patter@VOYAGER.NET>]

But education isn't a car dealership or the auto industry.  Students aren't
cars.  In Michigan, it doesn't really matter whether the people "in charge"
of the state standardized tests know the limitations of their tests.  The
politicians have deemed that the tests are the best indicator that education
is taking place.  There's something sick about a system that places this
validation power in the hands of politicians.

Plus, your argument sounds a lot like the one the NRA uses--it isn't guns
that kill people, it's people.  It's not the tests that are bad, but the way
people use the tests.  Hmmmm.

Here is a quote from Elliot Eisner in a 1992 English Leadership Quarterly
article.

"...the tacit assumption that the most important outcomes of schooling are
measurable and that a common test or array of assessment tasks will...make
meaningful measured comparisons...To describe a human being in numbers alone
is to say some important things about that person's features.  It is also to
neglect those features that do not lend themselves to quantitative
description, and the features neglected may be precisely those condisered
more important for particular purposes...What is even more troublesome is
that almost all of the nationa proclamations for school reform, including
those demanding higher standards and tougher courses, neglect the deeper
mission of schooling: The stimulation of curiosity, the cultivation of
intellect, the refinement of sensibilities, the growth of imagination, and
the desire to use these unique and special human potentialities."

Please tell me you don't think a standardized peice of assessment can
validly measure those qualities.

Nancy

At 01:06 PM 10/4/98 EDT, you wrote:
>In a message dated 10/3/98 11:24:11 PM Eastern Daylight Time,
>patter@VOYAGER.NET writes:
>
>>
>>  I have a real problem with the notion that test makers validate their
>> tests.
>
>As originally noted in my reply, I agree with your concerns about test
>publishers being the only entity to validate tests.  But as with most products
>that are sold, the adage has been "buyer beware" so I am sure that you would
>trust the auto dealer any more or less than the test publisher.  I can only
>hope that states have appropriately qualified individuals on staff to review
>the technical aspects of the commercial tests that are submitted by test
>publishers too ensure that they do in fact achieve the intended purpose.  My
>experience to date is that while teachers are often on these committees there
>are few that possess the technical expertise to know whether a test is valid
>or not.  With custom tests, I have found that states do look more closely at
>the proposals submitted which are reviewed by those with technical expertise.
>>
>>  First of all, I hate standardized tests.  They are more about how well an
>>  individual knows how to take a test than they are about measuring
>>  achievement.
>
>We have yet to define what everyone means by the term "standardized tests."
>
>Part of the development and validation of a test intended to measure
>achievement  is to ensure that the correlation between the test score and
>achievement is higher than the correlation of the test score to any other
>measure.  The correlations to these other measures can still be high but the
>correlation to achievement must be higher.  We are not in disagreement here.
>If the correlations of the test score to some other measure  (i.e.
>socioeconomic level, speed, etc.) is higher, than the test is not valid.
>
>Second, the best predictor of academic success is still
>>  socio-economic level.  If you're white, middle class, and grew up speaking
>>  standard English, your chances of doing well on the SAT are better than if
>>  you were black, working class, and spoke something other than the dialect
>of
>>  power.
>
>Gee, I always thought those that learned Latin or Greek did better on the
>verbal sections.  The issues you and others are raising relate to what might
>be termed "face validity" -- the extent to which those who take the test
>and/or use the results believe the tests is measuring something of value.  My
>previous posts focused on "content validity" but I agree face validity is an
>important but often difficulty concept to agree upon.
>
> There's something wrong with this.  The fact that our culture puts
>>  such store in test scores is more about the haves wanting to keep their
>>  goodeies than a concern about education.
>>
>>  Nancy
>
>This point addresses a more recent concept used by lawyers, namely the
>consequences of tests.  Test publishers are beginning to be concerned about
>"consequential validity" which I believe was something we thought about but
>really considered it to be the user's responsibility.  The pendulum swings
>back and forth but I agree that high stakes testing does carry with it a
>number of consequences.
>
>My inclination based on my experience is that negative consequences occur more
>from test misuse than poor development.  But who is it that should be
>responsibility for dealing with misuse?  Should the publisher be accountable
>or should the user be accountable.  It seems that most publishers would argue
>that it is the user that is accountable for using the tests properly.  Curious
>to hear what other think as the APA will probably be releasing another draft
>of the APA Guidelines in March and perhaps we can work on improving this
>section.
>
>===================================================
>Deanna M. De'Liberto, President/Director of Assessment
>D Squared Assessments, Inc.
>(Specialists in Test Development/Validation and Test Administration)
>9 Bedle Road, Suite 250
>Hazlet, NJ 07730-1209
>Phone:  (732) 888-9339
>Email:    Ddeliberto@aol.com
>Web:     http://www.quikpage.com/D/dsquared
>
>Member of the Association of Test Publishers
>===================================================
>
>>  At 03:38 PM 10/3/98 EDT, you wrote:
>>  >Nancy's question is a good one.
>>  >
>>  >To date, I would say the test publishers are the ones that validate the
>> tests
>>  >and certify to the public that their tests are valid.  Of course, the
>staff
>> of
>>  >test publishers include content specialist and measurement specialists
>with
>>  >backgrounds in test development and psychometrics.  This is certainly the
>> case
>>  >for shelf products.
>>  >
>>  >With custom tests, I think the states do play an active role in ensuring
>> that
>>  >test publishers do not pull the wool over their eyes.  Some states have
>> issued
>>  >separate contracts--one for the development, one for psychometric analysis
>> and
>>  >another for scoring.  This process is rather good for ensuring quality
>> control
>>  >in all critical stages for the process.  I have often recommended this
>>  >approach.
>>  >
>>  >If all else fails, there are the standards documents for those out there
>> who
>>  >wish to probe into technical reports and determine whether the test
>> publisher
>>  >and states are doing this right.
>>  >
>>  >Nancy, your point is a valid one.  This is a real technical area and it
>> does
>>  >often come down to trusting the test publisher.  But that is not that much
>>  >different from trusting say your auto mechanic, doctor, lawyer, etc.--is
>it?
>>
>>  >Seriously, I am open to a better idea/approach.
>>  >
>>  >Deanna
>>  >===================================================
>>  >Deanna M. De'Liberto, President/Director of Assessment
>>  >D Squared Assessments, Inc.
>>  >(Specialists in Test Development/Validation and Test Administration)
>>  >9 Bedle Road, Suite 250
>>  >Hazlet, NJ 07730-1209
>>  >Phone:  (732) 888-9339
>>  >Email:    Ddeliberto@aol.com
>>  >Web:     http://www.quikpage.com/D/dsquared
>>  >
>>  >Member of the Association of Test Publishers
>>  >===================================================
>>  >
>>  >In a message dated 10/3/98 3:16:15 PM Eastern Daylight Time,
>>  >patter@VOYAGER.NET writes:
>>  >
>>  >>
>>  >>  So, who decides what's valid and invalid?
>>  >>
>>  >>  Nancy
>>  >>
>>  >>  At 09:29 AM 10/3/98 EDT, you wrote:
>>  >>  >All of this discussion about standardized tests, cheating and user
>>  >>  >qualifications has made it necessary for me to think about the
>> definition
>>  >> of
>>  >>  >"standardized tests"  The definition we always used was "a test that
>> was
>>  >>  >administered under uniform conditions"  Such tests could be multiple-
>>  >choice
>>  >> or
>>  >>  >constructed response, norm referenced or criterion referenced but for
>> some
>>  >>  >reason most only consider the multiple-choice norm reference type to
>be
>>  >>  >standardized.  So, I must ask if our definitions are in sync.
>>  >>  >
>>  >>  >I do not think I agree with Rich Gibson's statement as it seems to
>>  >> rationalize
>>  >>  >the reason for cheating on tests.  If tests are developed properly,
>>  >>  >interpreted by those who have are qualified and appropriately trained,
>> and
>>  >>  >then used ONLY for the purpose intended there is NO excuse to tolerate
>>  >>  >cheating.  I agree that INVALID tests do cheat students and those that
>> use
>>  >>  >tests for purposes they were not intended to measure are cheating the
>>  >> public.
>>  >>  >
>>  >>  >Deanna
>>  >>  >===================================================
>>  >>  >Deanna M. De'Liberto, President/Director of Assessment
>>  >>  >D Squared Assessments, Inc.
>>  >>  >(Specialists in Test Development/Validation and Test Administration)
>>  >>  >9 Bedle Road, Suite 250
>>  >>  >Hazlet, NJ 07730-1209
>>  >>  >Phone:  (732) 888-9339
>>  >>  >Email:    Ddeliberto@aol.com
>>  >>  >Web:     http://www.quikpage.com/D/dsquared
>>  >>  >
>>  >>  >Member of the Association of Test Publishers
>>  >>  >===================================================
>>  >>  >
>>  >>  >
>>  >>  >In a message dated 10/3/98 1:37:16 AM Eastern Daylight Time,
>>  >>  >rgibson@PIPELINE.COM writes:
>>  >>  >
>>  >>  >> Standarized testing is and foments cheating...best r
>>  >>  >>
>>  >>  >>
>>  >>  >>  Rich Gibson
>>  >>  >>  Program Coordinator of Social Studies
>>  >>  >>  Wayne State University
>>  >>  >>  College of Education
>>  >>  >>  Detroit MI 48202
>>  >>  >>
>>  >>  >
>>  >>
>>------------------------------------------------------------------------
>> --
>>  >>  >To unsubscribe from the ARN-L list, send command SIGNOFF ARN-L
>>  >>  >to LISTSERV@LISTS.CUA.EDU.
>>  >>  >
>>  >>  >
>>  >>  Nancy G. Patterson
>>  >>  patter@voyager.net
>>  >>  <http://www.msu.edu/user/patter90/opening.htm>
>>  >>  |  \  /  |
>>  >>  |   \/   |__  _   _
>>  >>  |       / __/| | | |
>>  >>  |  |\/| \___\| |_| |
>>  >>  |__|  |_/___/|_____|
>>  >>
>>  >>
>-------------------------------------------------------------------------
>> -
>>  >>  To unsubscribe from the ARN-L list, send command SIGNOFF ARN-L
>>  >>  to LISTSERV@LISTS.CUA.EDU.
>>
>
>--------------------------------------------------------------------------
>To unsubscribe from the ARN-L list, send command SIGNOFF ARN-L
>to LISTSERV@LISTS.CUA.EDU.
>
>
Nancy G. Patterson
patter@voyager.net
<http://www.msu.edu/user/patter90/opening.htm>
|  \  /  |
|   \/   |__  _   _
|       / __/| | | |
|  |\/| \___\| |_| |
|__|  |_/___/|_____|

--------------------------------------------------------------------------
To unsubscribe from the ARN-L list, send command SIGNOFF ARN-L
to LISTSERV@LISTS.CUA.EDU.