Re: meeting with State Chair of Bd of Educ

[Art Burke <aburke@VANSD.ORG>]

Rogosa's work on the California API somewhat contradicts Kane and Staiger.  Popham is past it.  Art

>>> Allen.Flanigan@USPTO.GOV 03/26 5:30 PM >>>
Here's a pdf file Kane and Staiger's paper on Test Score Volatility.
Basically standardized test scores (particularly year to year comparisons of
tests given in the same grade) are worthless measures of school quality,
because the variation in test scores is caused more by factors beyond the
school's control (50 percent or more).  This confirms Popham's frequent
assertion that standardized tests (particularly NRTs, but he has included
most state tests which are put together just like NRTs) are useless as
measures of school quality.

http://www.ascd.org/readingroom/edlead/9903/extpopham.html 

We confirmed this volatility locally as well through looking at numbers for
various schools in the "High flying minority schools" report that Ed Trust
put together (along with all sorts of small print caveats about their
unwillingness to confirm or vouch for the data or implications their website
presents).  Schools touted as having "high flying" performance often simply
had a high score for one grade in one subject, then fell off the radar
screen.  Other schools had such small numbers of students tested (26, for
example) that wild swings in scores from one year to the next would be
expected.  Steve Krashen also noticed similar anomalies in scores reported
for California.

http://brookings.org/gs/brown/kanestaigerbrookrevision.pdf 

It's hard to say how best to address this in a short period of time.

Thumbnail version:

These tests have no validity for high stakes uses (judging school quality or
student learning).  Peter Sacks mentioned in his Standardized Minds book, in
fact, that in the lawsuit in one NC county, a test expert forget who)
pointed out that it was ridiculous to use a test designed to measure school
performance to hold students accountable, and the county decided to drop
their policy of (I believe) retaining students based on failing this test).
I doubt that even if the test NC is using is "domain referenced" (test item
content matches state standards) that they have assembled any evidence
showing predictive validity (students who do well on test tend to succeed in
college or in life) or measuring consequential validity (determining if, as
Robert Linn points out, the negative consequences of the testing program,
narrowing of curriculum and so on, are outweighing the alleged positive
consequences).

Students and parents deserve tests that are properly used.  Proper
psychometric practice requires the use of multiple measures (different types
of assessments, not retesting opportunities) in making important decisions
about schools and students.  Attaching high stakes to these instruments
destroys their usefulness as measures (Dan Koretz, Robert Linn).

States that do the most testing (and attach the most consequences) tend to
do the worst on the National Report Card.

High stakes testing distorts learning.  Ask Kirk if he saw John Merrow's
program this Thursday night (if not, offer to send him a transcript or
copy).  The narrowing of curriculum phenomenon is over 150 years old (it was
first noticed in England, when a money-for-results policy was instituted,
and schools were given tests, money was disbursed based on how they
performed on the test.  Unsurprisingly, schools in England spent a large
amount of time prepping their students to take the test, and England
abandoned the test and the policy as degrading to the quality of education).
Over a century later, we are still repeating England's mistake.

The tests are biased.  Briefly, since they are put together like
norm-referenced tests, and are designed to sort students, questions that are
likely to be answered correctly by minority students tend to get discarded,
because they have poor sorting ability while questions likely to be answered
correctly by majority students (middle class) tend to be included.

-----Original Message-----
From: jtglenn@CAROLINA.NET [mailto:jtglenn@CAROLINA.NET] 
Sent: Tuesday, March 26, 2002 7:43 PM
To: ARN-L@listsrva.CUA.EDU 
Subject: meeting with State Chair of Bd of Educ


Okay, I am meeting with Phil Kirk, chairman of the NC State Board of
Education, on May 10 (I think)-- the same day we are taking our EOG tests
(ironic).  Specifically, I have about 20 minutes to talk to him about high
stakes testing.

Pretend you are me.

What do you say?  What facts do you cite?  What research would you read to
be sure you're well informed?  What arguments should you prepare for?

I'm VERY excited about this opportunity-- not that I think it'll change
*his* mind, but he'll be with an entourage that includes some state
legislators and so on, and there'll be press coverage, so... it's a chance
to reach a couple different groups.

Anything you all can offer would be much appreciated. :)

Teresa G.
who needs to find some panty hose for this meeting... ;)

--------------------------------------------------------------------------
To unsubscribe from the ARN-L list, send command SIGNOFF ARN-L
to LISTSERV@LISTS.CUA.EDU.

--------------------------------------------------------------------------
To unsubscribe from the ARN-L list, send command SIGNOFF ARN-L
to LISTSERV@LISTS.CUA.EDU.

--------------------------------------------------------------------------
To unsubscribe from the ARN-L list, send command SIGNOFF ARN-L
to LISTSERV@LISTS.CUA.EDU.