Fwd: [arn2-strategy] Untested Theores Behind NCLB

[Susan Harman <susanharman@igc.org>]

Begin forwarded message:

> From: Bob Schaeffer <bobschaeffer@earthlink.net>
> Date: Tue Jan 16, 2007  5:54:58 AM US/Pacific
> To: ARN Main List <arn-l@interversity.org>,  arn2-strategy  
> <arn2-strategy@yahoogroups.com>
> Subject: [arn2-strategy] Untested Theores Behind NCLB
> Reply-To: arn2-strategy@yahoogroups.com
>
> THE UNTESTED THEORIES BEHIND NO CHILD LEFT BEHIND
>
> Nieman Foundation for Journalism Watchdog -- January 15, 2007
> by George Madaus and Michael Russell
>
> The 2002 passage of the No Child Left Behind Act (NCLB) was designed to
> reduce the gap in achievement between specific groups of students and  
> to
> ensure that all students develop basic skills in reading and
> mathematics. NCLB is the culmination of testing’s extraordinary growth
> that began in the 1950s. Now, five years later, NCLB is up for
> reauthorization.
>
> To measure progress toward reducing the achievement gap and improving
> school quality, states must annually administer tests in language arts
> and mathematics to all students in grades 3 to 8 and in high school.
> NCLB requires each state to establish its own testing programs and
> criteria for student performance. These requirements result in testing
> of more than 30 million public school children each year. Additionally,
> all students for whom English is a second language (ESL) must also be
> assessed in listening, speaking, reading, and writing. Under these  
> state
> and federal requirements, students entering kindergarten today must  
> take
> a minimum of sixteen state tests before graduating. The cost to  
> develop,
> administer, score and report all of these tests ranges from $3 billion
> to $7 billion a year.
>
> NCLB requires states to classify all students into one of the following
> categories: Advanced, Proficient, Basic, and, by default, Failure. The
> federal government uses these classifications to hold schools
> accountable for improving student performance each year for each of the
> following categories of students: ESL; those with disabilities;  
> American
> Indian/Alaska Native; Asian, African American; Hispanic; and White.
> Schools that fail to meet these improvement goals for one or more
> sub-groups face closure or state-takeover. It is consequences such as
> these that make these high-stakes tests.
>
> It is important to recognize that the NCLB test-based approach to
> accountability is not a full-scale reform plan designed to transform  
> our
> system of public education. Instead, test-based accountability is a
> fallible tactic that produces paradoxical outcomes. For example, it is
> relatively easy to increase tests scores without improving what  
> students
> know and can do. For this reason alone, it is misleading to equate
> school quality with student test performance.
>
>  From its inception NCLB has been criticized for inadequate funding and
> for too much testing, among other concerns. But even if adequate funds
> were provided and the amount of testing was reduced, several important
> questions about test-based accountability remain. As the President,
> Congress, and the nation debate the reauthorization of NCLB this year,
> the following questions need serious consideration.
>
> Q. What are the unintended, predictable, negative consequences of the
> test-driven accountability provisions of NCLB?
>
> The term iatrogenic refers to doctor induced illness; a negative,
> unanticipated effect on a patient of a well-intended treatment. The
> paradox of high-stakes testing can be called peiragenics – test induced
> illness -- the unintended negative consequences of well-intentioned  
> test
> policies. These include: narrowing the curriculum; ignoring non-tested
> subjects; test preparation and tutoring that may increase test scores
> without actually improving students’ knowledge and skills; cheating;
> giving extra attention to students close to the cut score at the  
> expense
> of those seen to have little chance of moving to the next performance
> level; retaining students in grade; dropping out; and decreasing
> motivation to learn. Paradoxically and most importantly, these
> unintended consequences corrupt the truthfulness of the inferences and
> decisions about student achievement and school quality based on changes
> in test scores. Further, these negative consequences are chronic and
> predictable; they have occurred over centuries and across continents.
>
> There are three predictable reasons why a high-stakes test produces
> negative consequences.
>
> First, within a subject field (e.g., English), teachers give greater
> attention to topics most likely to appear on the test (e.g., grammar  
> and
> persuasive writing) and decrease coverage of non-tested topics (e.g.,
> poetry and creative writing). Students then adjust their focus
> accordingly. This combined effect narrows the content and skills taught
> and learned within a subject.
>
> Second, a high-stakes test preempts time and coverage from subject not
> tested. Art, physical education, science, foreign languages, and social
> studies are short-changed in favor of the tested subjects, math and
> language arts. This narrows the curriculum across subject fields.
>
> Third is a “trickle down” effect on lower grades not directly subject  
> to
> a high-stakes test. The content and skills covered on the high-stakes
> tests diminishes the content and skills in the non-tested lower grades
> thus altering the curriculum across grades.
>
> Q. Why aren’t testing companies and state departments of education
> subject to independent monitoring?
>
> The nation has long needed, but never had, an independent means to
> monitor high-stakes testing. What other institution would contemplate a
> nearly universal treatment for children without moving slowly and
> systematically, with an independent mechanism to monitor the
> consequences? There is independent monitoring in a variety of fields
> including medicine, the stock market, the work of tradesmen,
> transportation, food, even pet food. There is no comparable independent
> group that evaluates a testing program before adoption or monitors test
> use and impact after implementation.
>
> There are three reasons for monitoring high-stakes testing programs.
> First, such a body is long overdue. Since the end of the 19th century,
> there have been repeated calls for such oversight. The benefits and
> risks to institutions and individuals that result from high-stakes
> testing policies are real and serious. Currently, examinees, educators,
> parents and the public only have the assurances of those that build the
> test or control testing programs that the tests, procedures, uses, and
> classification of students are fair and valid.
>
> Second, policy makers rely on test results to insure students are
> learning and that taxpayers receive value for expenditures on  
> education.
> As George W Bush proclaimed, “We've got to hold people accountable…. It
> is so important to have an accountability system become the cornerstone
> of [educational] reform in America.” However, the testing programs and
> tests used to hold students, teachers, and schools accountable are
> themselves not subject to independent, transparent accountability to
> ensure test quality, validity, and proper use. How do we really know  
> any
> of these tests measure what they’re intended to measure?
>
> Third, testing is a useful but fallible technology. All technology is
> subject to errors, misuse and unintended consequences. This does not
> mean you abandon a useful technology but instead work to make it better
> by minimizing its shortcomings. Because of testing’s importance and
> imperfections, oversight and monitoring is needed to assure the public
> that test scores are as accurate as possible, and that the benefits of
> the test far outweigh any harm. We do not have such assurances now, and
> can only acquire them through independent monitoring.
>
> Q. How timely and useful is the information teachers receive from the
> tests mandated by NCLB?
>
> Not very. Most high-stakes testing occurs in the spring and results are
> not available until after school begins the following fall. By then
> students have moved on to the next grade, often to a different school.
> When testing occurs in the fall, the test focuses on content covered  
> the
> previous year. When the fall test results are received later that year,
> they have little relevance to the content and skills being taught in  
> the
> current grade. Proponents of NCLB testing claim that the tests provide
> accurate diagnostic information that can help teachers improve and
> individualize their instruction. The tests can tell teachers that a
> student is not doing well in mathematics – something they already
> know—but not why. Put simply, high-stakes tests are not built to  
> provide
> diagnostic information in a timely manner.
>
> Q. What other assessment technologies can be used to help teachers
> tailor their instruction?
>
> Today’s high-stakes tests generally contain forty to sixty items
> sampling knowledge and skills students are expected to develop over the
> year. The majority of items are multiple-choice, with perhaps a few
> short-answer questions and one essay item. This test format is nearly
> identical to that of eighty years ago. Since then several advances in
> testing’s technology have occurred, but current testing policies  
> inhibit
> their use.
>
> Consider three examples. First, rather than have all examinees take the
> same fixed set of test items, computer-adaptive tests tailor the test  
> so
> that the examinee does not take items that are too easy or too
> difficult. By tailoring, fewer items are needed to obtain a reliable
> test score. For a high-stakes test that covers an entire year, the
> increased efficiency enables test developers to more broadly sample the
> year’s work. Initially, federal policy required all students within a
> state to take the same fixed test. After considerable controversy, this
> restriction was only recently reversed.
>
> Second, the costs of NCLB high-stakes tests led to the disappearance of
> two advantageous methods of assessing student achievement: performance
> assessments and student portfolios. Performance assessments consist of
> complex problems that require the use of materials and equipment to
> solve. Portfolios contain samples of student work collected over the
> year. Both assessment techniques increase the cost of testing. Given  
> the
> need and costs to develop multiple tests for grades 3-8 and high school
> to satisfy NCLB, states stopped investing in these alternate forms of
> assessment.
>
> Finally, an alternative to the NCLB single test administered once a  
> year
> is to take smaller, more specific samples at multiple points in time.
> Short computer-based tests designed to measure a sub-set of content and
> skills collectively result in a broader sampling of attainment over the
> course of the year. By measuring a smaller set of skills and knowledge,
> each test provides teachers with more precise information about each
> student’s learning in a timely manner. Finally, being computer-based,
> the tests could be adaptive, allowing the test to probe misconceptions
> and misunderstandings held by low performing students. This broader  
> body
> of information, timely return of results, and ability to identify
> reasons for low performance would greatly improve the instructional
> value of the tests while increasing the information used to hold  
> schools
> accountable for student learning.
>
> Q How realistic is the goal of having all students proficient by the
> year 2014?
>
> The 2014 goal was viewed by many in the testing community as  
> unrealistic
> and unattainable. These warnings were ignored. And even if the goal was
> met what would it mean? Each state uses different tests keyed to their
> state curriculum; each state sets its own proficiency scores for each
> performance level; there is no way to equate “proficient” performance
> across 50 states. How valid, then, is the label “proficient”? It is
> accurate, if and only if, “proficient” is whatever each states decides
> it means – a classic circular argument.
>
> Q Is NCLB leaving some students behind?
>
> One indicator of whether no child is left behind is school completion
> rates. One might think that if scores were improving, more students
> should be graduating high school. But the opposite pattern has occurred
> in several states. Texas saw improved test scores, but a noticeable
> decrease in its graduation rates, particularly for minority students.
> When Massachusetts introduced a test-based graduation requirement, a
> similar pattern occurred. This pattern occurs in several other states
> and large urban areas as well. These consequences are especially severe
> for many minority students.
>
> The use of test results in many schools also leads to students being
> held back, either because they did poorly on the state test or so that
> they do not take the state test required in the next year. Holding
> students back due to low performance might appear to be educationally
> sound, but research shows that such practices leads to negative
> consequences. There is a persuasive relationship between grade
> retention, being overage for grade, disengagement, and dropout rates.
> Being overage for grade predicts dropping out better than do
> below-average test scores.
>
>
> http://niemanwatchdog.org/ 
> index.cfm?fuseaction=ask_this.view&askthisid=00256
>
>
>
> Yahoo! Groups Links
>
> <*> To visit your group on the web, go to:
>     http://groups.yahoo.com/group/arn2-strategy/
>
> <*> Your email settings:
>     Individual Email | Traditional
>
> <*> To change settings online go to:
>     http://groups.yahoo.com/group/arn2-strategy/join
>     (Yahoo! ID required)
>
> <*> To change settings via email:
>     mailto:arn2-strategy-digest@yahoogroups.com
>     mailto:arn2-strategy-fullfeatured@yahoogroups.com
>
> <*> To unsubscribe from this group, send an email to:
>     arn2-strategy-unsubscribe@yahoogroups.com
>
> <*> Your use of Yahoo! Groups is subject to:
>     http://docs.yahoo.com/info/terms/