Re: The New Republic praises KIPP

["Monty Neill" <monty@fairtest.org>]

I think the story below is was not sent to EDDRA, but is relevant to the 
KIPP discussion.

The story focuses on selection bias. Evidence presented on the lists has in 
part covered the departure/removal of many students, probably leaving higher 
scoring students, thus making KIPP look better by its retention process. 
That strikes me as an even more serious problem than selection bias - if 
true.

Those doing this investigation might want to send a letter to New Republic - 
but I do suggest keeping the letter brief (within whatever TNR's guidelines 
are) - a long letter that is not published does no one any good.

Monty
----- Original Message ----- 
From: "George Sheridan" <learn@jps.net>
To: <ARN-l@interversity.org>
Sent: Thursday, December 07, 2006 8:25 PM
Subject: [arn-l] The New Republic praises KIPP


Can successful schools serve average students?
The Paradox of Choice
by Conor Clarke
Only at TNR Online | Post date 12.07.06
http://www.tnr.com/doc.mhtml?i=w061204&s=clarke120706
Talkback at http://www.tnr.com/doc_posts.mhtml?i=w061204&s=clarke120706


For teachers, policymakers, and academics, the question of how to close the
achievement gap--the disparity in education success between the average
middle-class and impoverished student--has a kind of Holy Grail quality:
difficult to answer, but impossible not to pursue. Would more money do the 
trick?
Or fewer students? Better teachers? These questions are not going anywhere
anytime soon. But, every once in a while, at least, a ripple comes along and
suggests that maybe, just maybe, the debate is moving. This happened last 
week
with Paul Tough's hefty New York Times Magazine essay--8,393 words, but 
worth
every syllable--about a new breed of intensive charter schools, meant to 
serve
lower-income students. The most influential of these is the Knowledge is 
Power
Program (KIPP), a collection of 52 schools spattered across the country. 
And,
unlike almost every other educational experiment out there, there just might 
be
something approaching consensus that these schools actually work.

When programs like KIPP are pushed into the spotlight as saviors of American
education, critics are skeptical for two reasons. First, they wonder if 
these
rigorous charter schools can be reproduced on a mass scale, since the hours 
tend
to run late and the pay tends to fall short. (Jonathan Chait tackled this 
earlier
in the week: To successfully replicate the programs, you'd need to either 
find an
army of masochistic saints or start paying real teachers real money.) The 
second
charge is that these schools don't tell you a whole lot about "average"
achievement, because they don't contain "average" students--a problem 
academics
lovingly refer to as "selection bias." The notion is controversial, but it 
makes
sense: After all, students don't fall out of the sky and into charter
schools--parents must decide to place them there. The critics that cry bias 
claim
that the kind of parent willing to make this choice--willing to research 
schools,
fill out forms, or sign a "commitment to excellence contract"--is the kind 
of
parent who would push a child to succeed no matter the school. And, for many 
(if
not most) lower-income kids, that's atypical.

When you're telling a larger story about education, the first complaint gets 
all
the attention: Educators are desperate for strategies that can be applied to 
the
public education system as a whole. But, for the new breed of charter 
schools
themselves--as they grow in number and in reputation--the second problem is 
just
as important, at least if the schools genuinely want to find and serve the
neediest kids on the block. And, while it's obvious to anyone who visits a 
KIPP
school that the organization is genuine in this pursuit, finding and keeping 
the
average low-income students--students that lack educational advantages 
coming
in--is no mean feat. Indeed, the battle these schools face isn't just about
finding the right educational strategy. It's about finding the right 
students.

If educators could force students into charter schools, selection bias 
wouldn't
be a problem, and we could know whether or not a school succeeds in helping
average students. But they can't, and selection bias is a problem. Few 
rigorous
studies have been performed in the realm of charter schools--in part because
there is a frustratingly limited amount of data to study--but the problem of
selection bias has been confirmed repeatedly in the context of school 
vouchers.
Just this summer, it happened in Washington, D.C.: A July working paper 
examining
the D.C. School Choice Incentives Act--a 2004 voucher program offered to 
students
whose families are at or below 185 percent of the poverty line--found that
students who accepted the offer had higher test scores than those that did 
not, a
fact that probably reflects familial differences. "Some see low income 
families
as an undifferentiated mass without looking at differences between families, 
but
some low-income and blue-collar families are very committed to education," 
says
Bruce Fuller, a professor of education and political science at the 
University of
California, Berkeley. "The charters are probably taking on the families that 
are
most committed to education."

For some critics, the problem with KIPP is that its students have
higher-than-average entering test scores, too--a shot fired by the Economic
Policy Institute's (EPI) 2005 study, "The Charter School Dust-Up," which 
found
that while 42 percent of students entering KIPP-Bronx schools could pass a
fourth-grade reading test, only 28 percent of fourth graders from 
neighborhood
schools could claim the same. As a result, EPI charged, KIPP institutions 
were
"unrepresentative of the neighborhood schools from which they draw."

But KIPP knows the danger of selection bias exists, and it does take steps 
to
reduce it--especially when the program opens a new school--through what
Washington, D.C. Director Susan Schaeffler calls "community recruiting." 
KIPP
teachers go door-to-door. They hunt parents in front of grocery stores and
carpet-bomb impoverished neighborhoods with flyers. In Washington, D.C., 
they
advertised on pizza boxes; in Gary, Indiana, it was with refrigerator 
magnets.
And, most importantly, KIPP asks public school teachers to refer struggling
students to the program. "We don't sit around and wait for the kids to come 
to
us," laughs Schaeffler. "We beg." Some of this reflects a simple need to 
fill a
class, especially when a school is first founded, but KIPP makes a bigger 
effort
to target the neediest students. "We want the students in underserved
communities," says Public Affairs Director Steve Mancini.

These steps are all about manipulating the flow of information: Since KIPP 
can't
force average low-income parents to sign up, it wants to at least offer the
choice to the right parents. But what happens when the information becomes 
harder
to manipulate? When a national magazine cover story is written about the
organization--or, more importantly, when it's featured on "Oprah" or 
"Anderson
Cooper 360"--KIPP loses control over the spread of information. For the 
purposes
of fundraising, that's obviously a good thing. But, for the purposes of 
retaining
a carefully tailored student body, it isn't. Could educational results 
actually
make selection bias a bigger problem? "I think that's likely," says Fuller. 
"Your
applicant pool simply goes up, and it becomes a richer mix of applicants." 
In
other words, when a school like KIPP is proclaimed a success, it becomes 
more
attractive to the atypical families that EPI worries about and KIPP wants to
avoid. "The demographic makeup of some KIPP schools is changing," says 
Harvard's
Richard Elmore, "because there's a sense that KIPP schools are very 
effective."

There is a very simple reason why this is the case: The longer a KIPP school 
is
around, the larger its footprint grows and the more parents try to place 
their
kids in it. "It is clearly in evidence that the longer a KIPP school is 
around,
the larger a waiting list becomes," says Mancini. Some of this is just the
natural progress of spillover from one year to the next. But much of it has 
to do
with media and word-of-mouth exposure. Now, Mancini says, "the original two 
KIPP
schools have waiting lists of well over 400." And if the applicant pool is
flooded with high-achieving students, there's nothing the organization can 
do: By
law, it must use a lottery system.

No one can accuse them of lacking good intentions. KIPP's AIM Academy, 
located
in Southeast Washington, D.C., has all the trappings of a needy school 
trying to
serve a needy populace. Its physical plant--a church annex on semi-permanent 
loan
from Methodists--is distinctively unglamorous, and the teachers have the
characteristic combination of idealism and exhaustion that seems to come 
with the
job description. ("It seemed like the only way I could get a vacation," 
quips one
teacher of her pregnancy and imminent maternity leave.) And the student body
seems to fit the criteria, too: All of AIM's 166 students are black, and the
percentage of students on free and reduced lunch is equal to or greater than 
that
in the surrounding schools.

That's unlikely to satisfy KIPP's critics at EPI or in the academy. But the
organization's challenge, over the coming years and months, will be to prove 
the
naysayers wrong. It's a challenge KIPP wants to meet: Last month, it started
drafting a request for research proposals to study not just how it serves, 
but
who it serves--looking at questions like "how do students at KIPP schools 
compare
to students of similar characteristics at other traditional public schools?"
There are reasons to believe that KIPP students might be different--that 
they
might leave for school each morning from homes with better resources or more
supportive parents. But it's in everyone's interest to hope that's not the 
case.

Conor Clarke is a writer in Washington, D.C.

George Sheridan