New Study: NCLB Ignores What We Know about School Change

[PRISCILLA GUTIERREZ <pgutpgut@msn.com>]

**NEWS RELEASE**
 
FOR IMMEDIATE RELEASE CONTACT: Heinrich Mintrop 415-250-0156
April 22, 2009 Gail L. Sunderman
571-217-7004
-Los Angeles, CA-
 
New Study by UCLA’s Civil Rights Project:
NCLB Ignores What We Know about School Change and
Is Motivated by Politics
 
 
A new report from the Civil Rights Project at UCLA, a non-partisan research
center which has been systematically studying the implementation of the
federal No Child Left Behind Act (NCLB) since its inception, finds that some of the
basic assumptions of the law are not working and may well be mistaken. In
this study, Why High Stakes Accountability Sounds Good but Doesn’t Work-- And Why
We Keep on Doing It Anyway, commissioned by the Civil Rights Project,
Researchers Gail Sunderman and Heinrich Mintrop evaluate whether the
accountability system endorsed by NCLB is likely to succeed or fail, and
whether it is compatible with what researchers across the country have
learned about the conditions needed for lasting school reforms.
 
The report finds that NCLB is failing on three fronts. First, there is
little evidence that high stakes accountability under NCLB works. It has not
improved student achievement and the sanctions have had limited effects in producing
real improvement. The law also results in high numbers of schools being
mislabeled as “failing” and far outstrips the ability of states to
intervene effectively in the schools it sanctions. Third, the law has
failed to connect in a meaningful way to the educators who must implement it --
they do not see the accountability goals as realistic and consider the sanctions
to be misguided and counterproductive for improving schools.
 
The most important finding is the damage the NCLB is doing to our
educational system. Under NCLB, the system “works” when education systems operate
within only a basic skills framework and with low test rigor. The cost to
our nation is revealed in an educational system stuck in low-level intellectual
work.
 
Civil Rights Project Co-Director, Gary Orfield, concludes, “The new
administration has a unique opportunity to address the serious structural
problems of NCLB and to forge a more constructive and effective federal
role. To persist in sound-bite educational politics that sound tough but have
failed for a generation would be a tragic mistake. To claim that it would further
the civil rights of children increasingly segregated in schools that have been
officially branded and sanctioned as failures -- but not provided help that
makes a real difference -- would be a blunder.”
 
Even though the law is failing in some critical respects, the authors argue
that we may maintain NCLB anyway because many derive secondary benefits from
the system, specifically those who are politically and ideologically
committed to NCLB and those deriving economic or political benefits from the law.
 
The authors contend that after fifteen years of state and federal
sanctions-driven accountability that has yielded relatively little, it is
time to try a new approach. A system based on mandates and legal administrative
enforcement should be replaced with one that emphasizes respect for the
professionalism of educators and active involvement of communities in
developing the capacity to implement lasting changes.
 
The full report can be found at http://www.civilrightsproject.ucla.edu/
 
A copy of the Executive Summary and Foreword can be found at the end of this
advisory. Copies of CRP’s previously released NCLB reports may also be
found on our web site above. Funding for this research was generously provided by
a grant from The Charles Stewart Mott Foundation.
 
 
About the Authors:
 
Heinrich Mintrop, Ph.D. taught middle school and high school for over a
decade in both the United States and Germany. He received a Ph.D. in education from
Stanford University in 1996. He is currently an associate professor of
education at the University of California, Berkeley. As a researcher, he
explores issues of school improvement and accountability in both their
academic and civic dimensions. He has recently published the book Schools on
Probation. How Accountability Works (and Doesn’t Work) at Teachers College Press. At UC
Berkeley, he is involved in programs that prepare strong leaders for
high-need urban schools.
 
Gail Sunderman, Ph.D. is a Senior Research Scientist at the George
Washington University Center on Equity and Excellence in Education where she directs
the Mid Atlantic Equity Center (MAEC). Prior to that, she directed a five-year
study examining the implementation of the No Child Left Behind Act of 2001
for the CRP. She is co-author of the book, NCLB Meets School Realities: Lessons
from the Field (with James S. Kim and Gary Orfield, 2005) and editor of
Holding NCLB Accountable: Achieving Accountability, Equity, and School Reform,
published in 2008. She is a former Fulbright Scholar to Afghanistan and
received her Ph.D. in political science from the University of Chicago.
 
 
Executive Summary
 
The federal accountability system, made universal though the No Child Left
Behind Act of 2002, is at its heart a quota and sanctions system. This
system stipulates the progression of underperforming schools through a set of
increasingly severe sanctions based on meeting performance quotas for
specific demographic groups. While it includes standards, assessments, and
performance targets, sanctions are the means by which the higher levels of the system
put pressure on lower-levels of the system to take accountability seriously.
Even though the law formulates the sanctions in the language of improvement,
support, and radical renewal, the punitive core for districts and schools is
apparent: when improvement efforts fail, loss of control and threat of
organizational survival is at stake.
 
But whether this system is up to the job of achieving its goal of improving
the performance of persistently underperforming schools is an open question.
Using findings from the best available research, this report examines whether an
accountability system based on the imposition of sanctions is likely to
succeed or fail and, if it does persist, what the consequences may be for sustaining
an educationally rigorous system. The report asks three questions: (1) does
the system work, that is, does it produce the intended results, (2) is it
practical, that is, can it be implemented, and (3) is it legitimate, or is
it valued among those who must implement it. We conclude with a discussion of
the costs of maintaining the current sanctions system.
 
Does the System Work? There are two aspects to this question: does the
system as a whole produce the expected outcomes; and do the actual sanctions result
in school improvement.
 
Does the system produce the expected outcomes? There is little evidence
that high stakes accountability under NCLB improves student achievement.
Although state accountability systems appear to be a success since test scores
continue to rise in most systems, the picture looks far less positive when one looks
at the National Assessment of Educational Progress (NAEP). When NAEP scores are
used, gains appear to be much lower, there is substantial variation among
states, and few states have narrowed the achievement gap among racial and
socioeconomic subgroups while improving overall performance at the same
time. Given the large discrepancies between NAEP and state assessments results, it
is not quite clear what the state tests measure. By all indication, state
accountability systems with their own pressures and sanctions are successful
at focusing schools’ and districts’ attention on state assessments.
 
Do the sanctions work? There is also a lack of evidence that the sanctions
themselves have been successful as an effective and universal treatment for
low-performing schools. Neither the transfer option nor the supplemental
educational service provisions have been widely embraced by parents or
districts. Whether or not the transfer option produces improvements in
school performance is a moot point since the percentage of students taking
advantage of this option (about 1% of eligible students) is so low. The response to
supplemental educational services has also been low (14% of eligible
students); and third party evaluations of these services are finding small, if any
statistically significant effects of the program on improving student
achievement. The corrective action and restructuring options, such as
reconstitution, charter school conversion or take-over by education
management organizations (EMOs), may work in some limited situations but are not
effective across the board. Among the variety of corrective action and restructuring
strategies that have been tried, none stick out as universally effective or
robust enough to overcome the power of local context.
 
Is the Sanctions System Practical? If the NCLB system was practical, it
would identify schools in need of improvement and restructuring with high
accuracy; appropriately direct schools to pay attention to students most in need of
help; produce an intervention burden for states and districts commensurate with
capacities to provide new impetus, ideas, resources, and personnel; and
lastly, through the imposition of sanctions, create momentum for deliberate and a
well articulated improvement processes for schools and districts stuck in low
performance. NCLB fails those practical criteria.
 
In state systems with at least moderately high performance demands, NCLB has
led to high numbers of failing schools that by far outstrip district and
state capacity to intervene. But it is not even clear if the bulk of these
schools are in fact correctly classified. Most notably, the system has no practical
answers to address the full spectrum of student performance and learning
needs, particularly for students far-below proficient, special needs students, and
marginally performing students; moreover, it does not speak to the
predicament of low-capacity schools and districts. While it may appear that the
sanctions system has succeeded in fermenting a climate of reform, such ferment, in
many instances, is more likely to result in unproductive turbulence than
sustained school improvement.
 
Is the Sanctions System Legitimate? Despite an almost twenty year period in
some states, accountability systems, and particularly NCLB, continue to
encounter serious legitimacy and acceptability problems among the groups
that they are designed to target—teachers, principals, and administrators in low
performing schools and districts. In general, while standards, assessments
of performance, and consequences for low performance are widely accepted ideas
in general, research suggests that attitudes about high stakes accountability
systems are more negative. This is because accountability systems designed
around sanctions violate core professional norms of educators and produce
widespread frustration and de-moralization among those charged with carrying
out school improvement efforts. Accountability goals are often not seen as
realistic, and the sanctions are considered to be misguided and not very
useful for improving schools. In efforts to improve test scores, teachers widely
report that they must compromise standards of good teaching in order to meet
accountability goals.
 
What are the Costs of Maintaining a Sanctions System? The combination of
uncertain effects, loose connections to the broader educational values and
norms of educators, and the difficulties or impossibilities of carrying out
the system day-to-day makes the sanctions system a prime candidate for declaring
it a failing system. But there is a way to maintain the system, although this
way produces high educational costs. As long as states maintain low-rigor
systems that concentrate on basic skills, and the more lenient options for school
improvement or restructuring are chosen, the system can persist with
relative ease. NCLB “works” when systems place low demands on the cognitive
complexity of learning tasks and, subsequently, on teacher capacity
building. State accountability systems that operate within a basic skills framework
and with low test rigor tend to produce lower numbers of failing schools.
Because such systems tackle school improvement goals that are fairly light,
affordable, and manageable, they are more practical within the NCLB framework. Systems
that are more ambitious produce an intervention burden that makes them
unworkable.
 
Improvement strategies that may be sufficient to reach lower level goals are
not sufficient to reach higher order goals. And once educators have operated
within the confines of a system oriented around test-driven basic skills
remediation, strategies for teaching and school improvement cannot simply be
switched to higher level skills, with the result that whole state systems
get stuck in low level intellectual work.
 
A redesign of the federal accountability system should start from four
principles:
 
• The system should reflect the complexity of the task by allowing
multiple measures, more flexibility and local options.
• Ambitious goals require ambitious capacity building within schools
and districts; and in all likelihood beyond.
• More comprehensive investments in student welfare that link
education with health, job development, and community building as well as
redistributive investments to attract and keep top-flight professionals in poorly
performing schools are needed.
• Overreliance on sanctions can be reduced when policies aim to
develop a partnership between government, the teaching profession, and empowered
low-income parents and to motivate changes by linking to the professional
values and standards of educators.
 
The report contends that after fifteen years of state and federal sanctions
driven accountability that has yielded relatively little, it is time to try
a new approach that replaces a system based on mandates and legal
administrative enforcement with one that emphasizes the professionalism of educators and
the active involvement of communities.
 
FOREWORD
 
We have bet the future of federal education policy on a theory of
accountability that does not work. It has been the dominant educational
reform theory for decades and the No Child Left Behind Act (NCLB) is its extreme
expression. It turns out, after studying research results from across the
country, it does not make much sense either as a managerial or an
educational strategy. It has very good intentions but often sanctions those institutions
where progress is most difficult and most urgently needed rather than offer
the kind of help that could really make a difference. This report, by
researchers Heinrich Mintrop and Gail Sunderman, dissects the logic of high-stakes
accountability policies, explores why they have failed, and concludes that
the failure was not one of implementation (though that made things worse) but of
the basic structure of the policy.
 
The Civil Rights Project has been studying the results of NCLB in six states
since it was passed and has previously issued 12 reports, as well as two
books and a number of articles, on its implementation and the results. Gail
Sunderman has led this research. Professor Mintrop is a leading expert on the impacts
of sanctions-based policies at the state as well as national level. Years ago,
we showed that the standards were inconsistent and sometimes meaningless, the
goals were incorrectly set and unfairly punished integrated schools and
those serving English language learners and other minority groups. Our work
showed that the law’s assumptions about teachers and sanctions were wrong and that
the sanction process was undermining the good goal of keeping experienced
teachers where they were most needed. We showed early on that neither the
transfer option nor the supplemental educational services provisions were
working. Work we commissioned demonstrated that the dropout provisions had
been gutted and that the requirements placed on the states went far beyond the
limited capacities of state agencies to fulfill. We have recognized all
along that the goals of more equal outcomes, good statistics on outcomes by
subgroups, and a number of other provisions in the Act could be part of a
good policy. The Civil Rights Project has joined many other researchers in
recommending the replacement of the very narrow and arbitrary goals of test
scores in two subjects with a much richer accountability scheme. When we
originally raised a number of these issues we were attacked by the law’s
defenders, but, increasingly, the issues we raised have become part of much
more broadly shared views of the NCLB experience. We believe that applying
the lessons of the past eight years could produce a much more effective federal
policy.
 
Now, as the country thinks about what to do next, it is important to focus
on some fundamental design problems with the NCLB that undermine its very
important goal of increasing the equity and success of American schools. The
first is that it was not designed around real educational experience, nor
does it utilize what research has shown about the sources of educational
inequality or the possibilities and conditions necessary for reform to work. Instead,
NCLB is based on the dual assumptions that children are falling behind very
largely because educators don’t care enough and that deadlines and strong sanctions
imposed by the federal government can cure the problem so that all subgroups
of children will become proficient by 2014. The second problem is that it often
punishes schools that are making a positive difference for students,
discouraging the staff and undermining future prospects for the school. The
third is that it has a very narrow definition of education that not only
diverts attention from other vital goals but also produces a strong focus on
tactics that create a semblance rather than reality of success in those
limited areas. The fourth is that all schools are being required to attain goals
that are impossible to attain on any broad level given what we know about both
the impact of schools relative to other forces in children’s lives and the
distribution of talent and achievement that appears in all human
populations. Finally, while the law obviously hopes schools will experience deep reform,
the deadlines and yearly goals do not connect with what is actually known about
the time and capacity-building required to actually turn around a school. I
believe that there are good ways to correct each of these problems.
 
This study, commissioned by the Civil Rights Project, finds that some
fundamental assumptions of the law are in error and, if continuously
pursued, are very likely to do more harm than good. Since state and local educational
institutions have the primary responsibility for public education—paying
nine-tenths of the bills and setting most of the rules—the first requirement
for federal policy should be that it does no additional harm to the public
school systems. This report shows that, in that respect, NCLB falls
short—not only in operation but even in its design and basic assumptions. A reasonable
standard would be that a policy not weaken key institutions, not undermine
support for public schools, and not try to impose impossible requirements.
Historically the role of the federal government has been to encourage new
initiatives, to commission research, to disseminate information and
statistics, and to provide resources to the schools. The sudden decision by NCLB to
define the most important subject matter, to mandate the grades that are tested, to
control teacher requirements, to set detailed requirements for yearly
educational gains, and to order what can be very drastic sanctions represent
truly radical interventions on state and local authority. A reasonable
standard for such interventions is that: 1) they are based on a solid understanding
of schools and school reform; 2) they make sense to and win cooperation from
the teachers, local administrators and state officials who must try to make them
work; and 3) they provide the resources needed to meet the additional
demands. As it now stands, NCLB does not meet any of those standards.
 
As a political scientist who has closely followed this law, it is obvious to
me that the logic of NCLB is much more political than educational. Educators
and those who spend their careers studying school reform were almost wholly
excluded from the framing of NCLB and were often the target of attacks by
some of its advocates. NCLB is about the politics of looking tough on educational
reform. Since the l983 Reagan Administration’s A Nation at Risk report, the
dominant style in education policy has been to look strong by demanding
accountability, putting more focus on tests, criticizing teacher
organizations, and either implicitly or explicitly blaming teachers, schools and school
districts serving large numbers of poor children for inequality in
educational results. One theme has been that some other means of running schools, such
as charter schools, would be better because it was less public. At the same
time, the law raises the pressure for schools, by themselves, to produce equal
outcomes while other social policies bearing on the lives of poor children
have been cut back. The dominant rhetoric has ignored the reality—reflected in
countless studies over the past four decades—that poverty, low parent
education, poor health, and inferior segregated schools all contribute
powerfully to unequal outcomes, and that those conditions can only partially
be addressed inside the schools. The vast majority of a child’s life is spent
outside of school; students come to kindergarten with hugely unequal
preparation. Ignoring the rest of children’s lives and communities and
expecting the schools to produce perfect equality is to expect something
that is impossible and has not been achieved in any nation under any educational
system.
 
The NCLB’s strategy makes sense in political terms. Policy makers look
strong by being critical, demanding and tough. They blame teachers and school
districts for the striking inequalities that exist between middle class
suburban, largely white schools and those serving poor and minority
children. There is a tendency toward escalating demands and criticisms as the latter
schools fall behind. Blaming schools and their teachers takes the pressure
off political leaders (and privileged communities) to play a serious role in
solving the problems of children in a society that tolerates a level of
child poverty higher than any other nation of similar stature.
 
After the massive failure of policies adopted by the first President Bush
and then President Clinton and reflected in Goals 2000 (which were supposed to
produce equal schooling outcomes in the decade leading up to 2000), the
second Bush Administration and Congress adopted a much more demanding set of
requirements, backed with even stronger sanctions in No Child Left Behind in
2001. It was particularly striking that this was done without the slightest
evidence that it was likely to work. As was widely predicted at the time by
researchers who had seriously studied school reform, that policy is a
failure, substantiated by the ever-growing numbers of schools and communities
officially branded as failures and sanctioned. There isn’t the slightest probability
that the law’s goals will be met, in spite of intense pressure and
widespread sanctions directed against many of the nation’s most troubled schools and
communities. Instead of inspiring hope and drawing our strong educators and
administrators to the schools that need them the most, it gives them an
incentive to leave faster so that they are not branded as part of a failed
institution.
 
This study shows that unfounded and unattainable requirements backed by
strict sanctions produce counterproductive reactions, produce massive failure, and
leave states with problems they cannot solve. Needless to say, this
undermines the attractiveness of teaching in the sanctioned schools and the interest of
student and teachers in what often becomes a narrow drill-based curriculum.
 
Often the alternatives are posed as a choice between accountability
NCLB-style and no accountability. This is like saying that there is no alternative
between bleeding a patient and letting him die. There are, of course, better
alternatives. Medicine would not dream of accepting directives from
Congress that it should only pay attention to two indicators of health and that
Congress could prescribe the rate of mandatory patient recovery without any reference
to what the best medical research indicated was possible. This is what has
happened in education. The real choice is between narrow accountability with
arbitrary standards and broader accountability linked to research on what
are actually attainable goals within a prescribed period of time. Many civil
rights groups and experts have signed statements advocating multiple measures which
we strongly support.
 
Our survey of teachers in two highly impoverished school districts actually
showed that teachers supported accountability but they wanted reasonable
standards, materials that effectively addressed the learning challenges of
students, time to work together to back the school’s educational operations,
and a good professional leader in their building. Federal law should foster
these goals. What we need are richer forms of accountability measuring:
school completion; acquisition of key skills needed for college; encouragement of a
full education rather than endless drills on just two subjects; promotion of
the kind of imagination and activities that attract students to learning;
preparation of our students to become capable citizens in a democracy, and
rewards for teachers and principals who make a clear difference in the level
of success for their students.
 
As we go into the next round of debate it would be very valuable to go back
to first principles and consider how the federal investment in education could
be refocused in a more positive way. It is very clear that there were
tremendous inequalities in our schools before NCLB that still exist and still need to
be addressed. I think that a strong and positive federal role is an important
part of the solution, but that it must be developed in collaboration with those
who truly understand schools and work in them, that it must provide resources to
build capacity, that it should support building more knowledge in a field
where there are too few proven remedies, and that it should provide recognition
and support for those schools, administrators, and teachers whose work makes a
real difference for children. This study by two leading scholars in the field is
a solid contribution to that effort.
 
Gary Orfield




Priscilla Gutierrez

...change is inevitable, growth is optional...
words of advice from Michael Fullan