Doing "scientifically valid" experiments in the field of psycholo gy

["Allen Flanigan." <Allen.Flanigan@USPTO.GOV>]

One of the cornerstones of the scientific method is repeatability; another
is controlled experimentation with a single variable being changed (all else
being the same).  This is impossible when it comes to psychological testing
of humans.  You cannot, for example, compare two curricula by teaching
little Johnny using textbook A, testing him, then jumping in a time machine
to go back and do it all over again with the same teacher in the same
classroom with little Johnny having the same thoughts in his brain and so
on, except this time you use textbook B.  Further, controlling all variables
save one implies that we know the "master equation", the function which
accurately and completely describes the phenomenon we are hypothesizing
about (in the case of macroscopic objects, newton's laws of motion, which
predict how objects will react under any circumstance; in the case of
psychology, it would require equations describing the activities of the
human brain in whose accuracy we are confident).  Not posessing such
equations describing brain function and activity fully, we cannot be sure
that we are aware of all the variables that may contribute to the
experimental results we seek to measure.

So what they do of course is infer that if we do this using a large number
of children, controlling for as many variables as we can that are suspected
to potentially affect the outcome (eaten any lead paint lately?) we might
reasonably draw some conclusions.  This is not, strictly speaking, science;
it is more properly data analysis combined with conjecture.  The classic
scientific method eliminates all uncertainties making the conclusion
inescapable (provided you have designed your experiment properly). Such
tight control is impossible in any kind of psychological testing or
experimentation.

There are of course ways to confirm that you have not overlooked anything.
Aside from encouraging others to repeat your results, you can try to prove
competing hypotheses in order to disprove your own hypothesis.  Hence, as
the saying goes, "the first sin of scientific research is to find what you
are looking for."  Better to set out to prove your hypothesis is false, and
if you cannot, it must be correct.

http://www.problemsolving.net/i_8.htm

-----Original Message-----
From: pfarr@UCLINK4.BERKELEY.EDU [mailto:pfarr@UCLINK4.BERKELEY.EDU]
Sent: Thursday, September 06, 2001 11:20 PM
To: ARN-L@listsrva.CUA.EDU
Subject: Re: an interesting article from The Chronicle of Higher
Education


Thanks, Victor!  I wonder how many randomized, controlled experiments Dr
Cook has actually done in public schools, and, if so, how carefully he
checked for confounding variables.  In large semi-democratic institutions
such as US public schools, it's not as easy as doing aspirin/placebo
experiments with thousands of willing subjects like medical doctors eager
to advance scientific knowledge.

--------------------------------------------------------------------------
To unsubscribe from the ARN-L list, send command SIGNOFF ARN-L
to LISTSERV@LISTS.CUA.EDU.