|
 Exposing
The Mystery of Standardized Test-Question Writers
by Laura Pappano, Boston Globe, March 14, 2004
For more articles like this
visit
http://www.bridges4kids.org.
For those
stressing about upcoming MCAS tests, here's a tip from some
inside sources: The first questions will be easy, and if an
answer seems right, it probably is.
That's because test makers aren't evil trolls plotting student
failure, but normal people who know how students think.
"We are not wringing our hands trying to be tricky," said Judith
Rubenstein Gerstenblatt, publishing manager for Measured
Progress, the Dover, N.H., testing company just hired to create
future MCAS tests, starting in 2005. The company, which will be
paid $118 million over five years, also will administer and
report results of the Massachusetts Comprehensive Assessment
System tests.
The same company already designs the MCAS alternative assessment
and has 22 other state testing contracts. A visit to the
high-security warehouse and offices tucked into a renovated
brick factory building offered a peak into the standardized
test-making world.
The test writers choose reading passages from a variety of
sources, even pamphlets plucked from car repair shops or bus
terminals. They also edit questions and fix those that might
confuse the student, like a math query about rows of roses.
And they are busier than ever because the demand for testing
keeps rising. The latest culprit? The federal No Child Left
Behind law, which requires students in grades 3-8 to be tested
yearly in English and math, starting next year.
Over the years, the way states use tests has changed, leading to
a need for even more test versions, said Stuart Kahl, the
cofounder and CEO of Measured Progress.
When the company started 20 years ago, states tested samples of
children, but didn't give feedback about individual schools'
performance. Testing soon expanded to all students, but Kahl
said schools reused test booklets, setting off cheating scandals
in the 1980s.
Now states want tailor-made tests, and new versions each year.
The more tests, the more questions, and the busier the question
writers get.
Coming up with test questions involves creativity. "Every step
along the way, there is human creativity, intelligence,
subjectivity," said Gerstenblatt. Questions may reflect a
writer's passion, which is why the math question writer who is
also a musician often sets his math puzzlers in a musical
context.
In the renovated mill building, question writers are grouped by
subject area and sit close enough to bounce ideas off each
other. The science question writers have journals, books, and
manuals nearby, while the English specialists keep Shakespeare
handy but also may wander the stacks of the local public library
in search of a reading passage or inspiration. Proofreaders make
sure there's nothing in a question that could confuse or
distract a kid from answering. Others make sure a test doesn't
have questions with all male or all female names, or names of a
single ethnicity. The goal: test, don't trick.
It's the writers' job, with help from state education officials
and teachers, to get at how well kids have learned what the
state has decided they should learn. Unlike other types of
standardized tests that rank students nationally, state tests
are linked to classroom lessons.
That means kids shouldn't need test prep to do well. Ron Dietel,
spokesperson for UCLA's Center for Research on Evaluation,
Standards and Student Testing, said studies indicate old
test-taking tricks -- like picking choice "C" when unsure of the
answer -- doesn't help. "The only real thing to do is know the
content," he said.
But it helps to understand that test makers don't try to be
tricky, but do include answer choices reflecting common errors.
For example, math problems requiring kids to multiply will often
include a wrong answer choice students would get by adding
instead. Other wrong answers reflect the failure to carry in an
addition or multiplication problem.
Science question developer Ann Adjutant, a former Maine high
school physics teacher, culls ideas from science magazines and
includes common misconceptions among her wrong answer choices.
Sometimes test makers get permission to use a current author's
work. Sometimes they write passages themselves. Gerstenblatt
wrote "Drinking Milk Is Good for Birds." Kahl authored a sappy
verse to test student understanding of rhyme schemes, requiring
them to pick the last line with the correct rhyme:
"Sometimes you may be happy and other times you're blue/And if
you want to change your mood, there is something you can do/You
can take a ray of sunshine (But handle it with care)/And give it
to a friend in need?"
The correct finish? "Whose smile you soon will share."
Test makers stick closely to state standards, and committees
check for bias, but they strive to design creative questions
that reflect a region's flavor. That's why you'll find more
Native American allusions in tests created for schools in the
Southwest -- and no Greyhound bus schedules in Wyoming.
So the Patriots' Super Bowl win will probably appear in a future
MCAS question, and you might find Red Sox players' batting
averages in a math problem.
back to the top ~
back to Breaking News
~ back to
What's New
|
