Scientifically Based Research
- Subject: Scientifically Based Research
- From: George Sheridan <learn@JPS.NET>
- Date: Sun, 3 Mar 2002 21:42:55 -0800
- Reply-to: Assessment Reform Network Mailing List <ARN-L@LISTS.CUA.EDU>
- Sender: Assessment Reform Network Mailing List <ARN-L@LISTS.CUA.EDU>
Nancy Creech wrote: You might enjoy doing a little discourse analysis at
Click here: Scientifically Based Research -- U.S. Department of Education
Susan Neuman, Assistant Secretary for Elementary and Secondary Education
hosted this gathering. She began her welcoming statement with these words:
"One of our goals today?we have a very practical goal actually. We're
no longer debating whether scientifically based research and scientifically
based evidence is important, we know it now is important and we know it is
critical. As many of you know, we have counted one hundred and eleven times
that the phrase "scientifically based research" is in our new law."
Two things struck me on first reading:
First: The presentation by Valerie Reyna, Deputy for the Office of
Educational Research and Improvement was at a very elementary level. Who in
that room needed this sort of basic introduction to science?
Second, the scientists they called on to define scientifically-based
research defined it much less narrowly than Reid Lyon, et.al. have done.
And they stressed that something is "scientific" not because the government
says it is, but because it withstands critical scrutiny.
But the scientists were only window-dressing. The core purpose of the
program was revealed in the afternoon sessions. Look at the presentation on
reading < http://www.ed.gov/offices/OESE/esea/research/greer.html
won't be surprised to see phonemic awareness and phonics listed as number
one and number two of the five elements research has proven essential to
reading instruction. In fact, the presenter (Dr. Eunice Greer, identified
as a reading consultant from Illinois) stated, "Most of my comments today
are drawn from the National Reading Panel Report that was delivered late in
the year 2000." Whenever someone mentions the National Reading Panel,
remember that the National Research Council of the National Academy of
Sciences did a report on Preventing Reading Difficulties in Young Children,
which did not endorse Phonics-uber-alles. While it is possible to disagree
with specific emphases in the NRC report, that report was explicitly
designed to reflect the scientific consensus on reading instruction.Reid
Lyon put together his panel because he didn't like what science is really
Still, it may be possible to draw on what these scientists said to oppose
the highly restrictive definitions of "scientific" likely to be embodied in
proposed U.S. Department. of Education regulations.
Speaking on the basic principles of scientifically based research, Lisa
Towne of the Center for Education of the National Research Council said:
"Replicating" is a very core notion in science. It has to do with
the fact that since in any particular study you're only relying on a
limited set of observations, to what extent does what you're looking at
here and now generalize to other times, places and contexts. In education,
as you know, this is a critical question. Teachers and researchers alike
have been knowing for years that something that works in a particular
classroom may not work in the classroom next door and may not work in the
same classroom a year later. So attention to sort of what's going on in the
classroom at that time can help you understand the conditions under which
things tend to work and therefore how to think about how findings can
generalize from one time to another.
"I'll go on to the last principle here, which has to do with the
transparency of the scientific enterprise. Valerie alluded to this as well.
This just has to do with the role of the scientific community actually
working together to try and make sense of all of the findings and all of
the conclusions that come from individual studies. Educators often bemoan
what there perceive as bickering among the research community and we'll
grant you that there is some bickering. But there is actually something
important to say about that and that is that researchers are actually
trained and employed and paid money to be skeptical observers and to ask
critical questions. That's their job. So, this critical kind of work,
critiquing other peoples findings and trying to make sense of them is
actually an indication of the health of the scientific enterprise, not its
<SNIP> "at one level there is a difference between the so-called
hard and soft sciences. And, that has to do with differences that emanate
from studying inanimate objects and studying people, which are complex and
do crazy things that we often can't understand or predict very well.
"So, there are some things that are different. Broadly, research
or control is one of them. Think of it this way, a petri dish of heart
cells is a heck of a lot better behaved than a classroom of third graders.
Anyone whose tried to study education research and has done cell biology,
as one of my committee members did, can attest to this.
"There's other things that are different. I'll just touch on this
last one on the slide which has to do with certainty. Valerie said, and the
committee completely agrees, that science is by definition an uncertain
enterprise. The key is understanding the degree of uncertainty that is
associated with what we know. In general terms, in the physical sciences we
because of this ability to control the environment tend to have more
certainty associated with them than sciences that have to do with people,
like education research."
Stephen Raudenbush of the University of Michigan said, "A scientist is
expected to search for disconfirming evidence, and that's a crucial feature."
He continued, "The role of the scientific community is key. It's a
healthy scientific community who can?and this relates to democracy, being
able to freely evaluate alternative points of view, not feel that there's
going to be some censorship.
"The people who are committed to the principles I just mentioned
who evaluate this, the process of objectivity really involves this group of
people engaging in this ongoing debate. Scientists, as was mentioned, are
trained to be skeptical and that process can really work. What's really in
the final analysis scientific is what the community of scientists says is
Raudenbush went on to say, " Does qualitative research play a
role? I would say, yes, without doubt. Because we need to not just test the
impact of things out in the field, we need to do a lot more of that. We
haven't done enough. But we have to have good things to take into the
field. We have to have good ideas about how to teach math, how to teach
reading. Those ideas come from up close, careful study of expert
practitioners in real settings and how kids learn. So, we need that up
close kind of research but see we've got to do a better job of connecting
that research with field trials of what works, and that's what's really
"My tenth and final question is: Is there any danger here that we
are going to be overselling the role of science in education? I think there is.
"I've got a quote here from E. L. Thorndike who wrote the lead
article in the founding edition of the Journal of Educational Psychology in
1910. I won't read the entire quote except to say that Thorndike felt that
a scientific psychology was about to produce decisive evidence on virtually
every practical question that arises in education. We know in retrospect
that he was wrong. Unfortunately, by overselling what science can do, it
led to a crisis of, you might say, rising expectations that couldn't be
met. For a long time thereafter science in education fell into disarray.
"The same thing happened in the '60s with scientific problem
solving, the idea that we would have kind of a social engineering model.
We'd try programs, we'd evaluate them, we'd get feedback, the programs
would get better and the great society was going to be born out of this
sort of scientific and engineering model. That was an overselling. We
couldn't really pull that off.
"So, let's make sure that we have a balanced view this time. I am
so excited that we have an opportunity to do it, to do it right without
overselling it this time."
Post a Message to arn-l: