[Fwd: Re: Does It Work?]

[Rick Parkany <rparkany@BORG.COM>]

EMERGENCY, fellow ARNers!

Please excuse the length of this fwd post, Folks. But, there is this fellow, Dr. HAKE, a
specialist on *effect-size* and meta-analysis who  regularly posts to the EVALTALK
(AmEvaluationAssoc) discussion list. Art & Ken are felow correspondents, as well as a host
of others of us, this list, unnamed...

He cites the the Campbell Collaboration. .  .
<http://www.campbellcollaboration.org/Franews.html>. . . .,
a fledgling international research group based at the University of Pennsylvania in
Philadelphia,
and the American Institutes for Research, or
AIR <http://www.air-dc.org/>,
a Washington-based think tank. . . . .

as the clearing-house(s) for LNCB protocols deciding on the validity of *scientific*
research concerning *what works* in K-12 public educxation.

As HAKE goes on to say (quote):

        *Although Mr. Whitehurst expects the clearinghouse to eventually settle
        on criteria emphasizing experimental . . . (read "randomized field trials")...,
        he said the final standards would be determined by an outside panel
        of 10 research experts.* (HAKE, EVALTALK, 18Nov02)

Please check out the links on this reference and see if we can get a picture on the nature
of these *experts* who will, allegedly, winnow the good from the bad from the ungly in
K-12 according to *control-group* based research!

Anyone familiar w/these folks and their ilk?

I'm already suspicious, because one of the cohorts operating in collusion w/the Campbell
Collaboration is the Cochrane Group, whom I have criticized talready his list a few years
ago for advocating industrial efficiency in the schools a la organizational and corporate
management theories and practice.

I'd like to send along a synopsis of our action research concerning these purveyors of
*quasi-experimental designs* (I call them *quasi-designs, experimental*) to the EVALTALK,
ASAP.

Your help will be appreciated AND cited in my reply...  ;-} rap.

-------- Original Message --------
Subject: Re: Does It Work?
Date: Mon, 18 Nov 2002 10:01:52 -0800
From: Richard Hake <rrhake@EARTHLINK.NET>
Reply-To: American Evaluation Association Discussion List
<EVALTALK@BAMA.UA.EDU>
To: EVALTALK@BAMA.UA.EDU

Please excuse the:

a. length of this post - please DON'T hit the reply button and
thereby inflict it yet again on all subscribers!! and

b. cross-posting (in the interests of interdisciplinary synergy) to
discussion lists with archives at:

ASSESS <http://lsv.uky.edu/archives/assess.html>,
Biopi-L <http://listserv.ksu.edu/archives/biopi-l.html>,
Chemed-L <http://mailer.uwf.edu/archives/chemed-l.html>,
EVALTALK <http://bama.ua.edu/archives/evaltalk.html>,
Math-learn <http://groups.yahoo.com/group/math-learn/>,
Math-teach <http://mathforum.org/epigone/math-teach>,
PhysLrnR <http://listserv.boisestate.edu/archives/physlrnr.html>,
POD <http://listserv.nd.edu/archives/pod.html>.

In his Math-learn post of Nov 17, 2002  8:40 am titled Does It Work?"
Jerry Becker called Math-Learn'ers attention to the James Traub's
(2002) New York Times article "Does it Work."

Traub wrote:

"Last month, Congress passed legislation replacing the Office of
Education, Research and Improvement, which has been widely criticized
for being too easily sold on fads, with a more independent institute
intended to foster a new culture of rigorous research. Dr.
Whitehurst, a psychologist who has researched the effectiveness of
the Head Start program, will lead the new "Institute of Education
Services." He is also in the process of setting up the "What Works
Clearinghouse". . . (see, e.g., Viadero 2002). . . a body that will
establish standards for research and then determine which of
thousands of studies on class-size reduction, peer tutoring, reading
instruction and so on meet those standards. There is giddy talk in
the research world of some day establishing the equivalent of the
"Food and Drug Administration," declaring educational doctrines safe
and effective, or not. . . . ."

I have previously commented (Hake 2002) on the Education Department's
initiative to make education research more "scientific." The "What
Works Clearinghouse" appears to be its latest effort in that
direction.

Debra Viadero (2002) writes:

VIADERO-VIADERO-VIADERO-VIADERO-VIADERO-VIADERO-VIADERO

As part of its campaign to make education an evidence-based endeavor,
the Department of Education has awarded an $18.5 million contract to
a group of researchers and education organizations to build a
national clearinghouse on
"what works" in schools.

When it's up and running over the next year or two, federal officials
say, the "What Works" computer database will give educators and the
public the lowdown on the scientific research undergirding a wide
range of programs, tests, practices, and policies. . . .

The launch of the clearinghouse comes as educational administrators
are gearing up to comply with the new requirements in the "No Child
Left Behind" Act of 2001, an overhaul of the Elementary and Secondary
Education Act. The law requires states and districts to use only
those programs and practices that can be backed up by 'scientifically
based' research - a tough task for educators who have neither the
time nor the expertise to pore over the research literature on all
the programs they encounter.

To develop and manage the clearinghouse, the department last month
chose two groups that have already tried their hand at synthesizing
social science studies: the Campbell Collaboration. .  .
<http://www.campbellcollaboration.org/Franews.html>. . . ., a
fledgling international research group based at the University of
Pennsylvania in Philadelphia, and the American Institutes for
Research, or AIR <http://www.air-dc.org/>, a Washington-based think
tank. . . . .

Although Mr. Whitehurst expects the clearinghouse to eventually
settle on criteria emphasizing experimental . . . (read "randomized
field trials"). .  . , he said the final standards would be
determined by an outside panel of 10 research experts.

That issue is critical in education because good evaluations are rare
and pure scientific experiments are even rarer. When AIR reviewed
reform programs three years ago, it turned up only three with strong
research bases.

Nonetheless, Rebecca S. Herman, who led the earlier study and is
heading AIR's part in the new clearinghouse, said she expects the
pickings to be better this time around.

"The last couple of years, there's been a lot of interest in what is
high-quality research," she said, "and I think the field has moved
forward a little bit" . . . .

Establishing the clearinghouse is also a politically delicate venture
for the department because the federal agency is barred by law from
recommending specific curricula. Some previous attempts to highlight
promising educational programs, in fact, have run into heated
controversy.

By relying on hard science, however, Mr. Whitehurst says the
clearinghouse can avoid such pitfalls. "It's not in the end a
judgmental process, though, of course, humans will be involved," he
said.

The proof will be in the clearinghouse's products, the first of which
are expected to go online within a year.

VIADERO-VIADERO-VIADERO-VIADERO-VIADERO-VIADERO-VIADERO

It has occurred to me that some subscribers might know of
"scientific" research studies that should be considered for inclusion
in the "What Works Clearing House." If so, the first step might be to
contact the American Institutes for Research (AIR) at
<http://www.air-dc.org/contact_us/contact_us-set.htm>.


Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367
<rrhake@earthlink.net>
<http://www.physics.indiana.edu/~hake>
<http://www.physics.indiana.edu/~sdi>


REFERENCE
Hake, R.R. 2002. "Scientifically Based Research,"
PhysLrnR/POD/AERA-D/Chemed-L/EvalTalk post of 13 Mar 2002
13:43:21-0800; online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0203&L=pod&P=R6333>.

Shavelson, R.J. & L. Towne, editors. 2001. "Scientific Inquiry in
Education," National Academy Press, prepublication copy online at
<http://www.nap.edu/catalog/10236.html>.

Traub, J. "Does It Work," New York Times, Sunday, November 10, 2002;
online at <http://www.nytimes.com/2002/11/10/edlife/10CHILD.html>.

Viadero, D. 2002. "Ed. Department Picks Groups To Develop Database of
Effective Practices," Education Week, September 4, 2002; online at
<http://www.edweek.org/ew/ewstory.cfm?slug=01whatworks.h22&keywords=viadero>.

------------------------------------------------------------------
EVALTALK - American Evaluation Association (AEA) Discussion List. See also
   the website:  http://www.eval.org
To unsubscribe from EVALTALK, send e-mail to listserv@bama.ua.edu
   with only the following in the body: UNSUBSCRIBE EVALTALK
To get a summary of commands, send e-mail to listserv@bama.ua.edu
   with only the following in the body: INFO REFCARD
To use the archives, go to this web site: http://bama.ua.edu/archives/evaltalk.html
For other problems, contact the list owner at kbolland@sw.ua.edu


--
"Dein Wachstum sei feste und lache vor Lust!
Deines Herzens Trefflichkeit
Hat dir selbst das Feld bereit',
Auf dem du bluehen musst." JS Bach: Bauern Kantata
Richard A. Parkany: SUNY@Albany
Prometheus Educational Services
http://www.borg.com/~rparkany/
Upper Hudson & Mohawk Valleys; New York State, USA

--------------------------------------------------------------------------
To unsubscribe from the ARN-L list, send command SIGNOFF ARN-L
to LISTSERV@LISTS.CUA.EDU.