Campus & Community

Assessing the assessments

5 min read

Daniel Koretz takes a critical look at the pros and cons of mandatory testing

A recent trend in higher education may mean fewer sleepless nights for prospective college students.

A number of mostly small, liberal arts colleges have dropped the standardized test requirements that have long been a part of their regular application processes. Last month, Smith College in Northampton, Mass., and Wake Forest University in North Carolina, joined the list.

According to Smith’s Office of College Relations, the reason for the change included “the recognition that SATs may not be the best predictor of academic potential.”

Educational testing is a fundamental part of the educational system in the United States, but many argue that far too much emphasis is placed on it.

One influential voice in the lively, often contentious, testing debate belongs to Daniel Koretz, professor of education at the Harvard Graduate School of Education (HGSE), whose research focuses on educational assessment as it relates to educational policy, with an emphasis on the effects of high-stakes testing. His new book, titled “Measuring Up: What Educational Testing Really Tells Us” (Harvard University Press, 2008), is a detailed exploration of the pros and cons and complexities of testing.

Koretz has been interested in educational testing since the early days of the Reagan administration when he worked in the Congressional Budget Office on policy analyses.

At the time, he recalled, “there was a tremendous amount of fuss and bother about trends in the performance of American kids, and what was really puzzling to me was that people cited the same sources … and came to diametrically opposed positions.”

Over the next several years, he said, he devoted most of his time to a project that was “trying to sort out what really had happened to the performance of American kids on tests and why.” And, he said, “I got hooked.”

The idea for his book grew out of a class Koretz developed at HGSE, one for master’s degree students who had little or no background in the basics of educational testing. He had looked around for such a class and, when he couldn’t find one, set about designing his own. He wanted something that could both offer instruction on the core principles of measurement and explore some of the debate surrounding the field.

Soon, Koretz said, students who took the class were returning to complain to him that many claims about testing made by policymakers were “simply wrong.” They encouraged him to write a book as a corrective, and, eventually, he did.

“The goal of this book,” Koretz writes in his introduction, “is to help readers understand the complexities inherent in testing, avoid the common mistakes, and be able to interpret test scores reasonably and use tests productively.”

One of the main problems with using tests as an accountability tool, argues Koretz, is that it tends to produce serious exaggerations in scores. Teachers who simply “teach to the test” produce scores that are misleading. The negative consequences of failing to meet performance standards, he said, encourage teachers to “game the system” in such ways.

Nowhere is the debate starker than when it comes to the controversial “No Child Left Behind” law, passed in 2001 and currently up for reauthorization. The law mandates the use of standardized tests, and holds primary and secondary schools accountable for student performance.

No Child Left Behind, said Koretz — while in part well-intentioned in its efforts to narrow the achievement gap between rich and poor and minority and majority students — “is very, very, very poorly designed” and in need of a major overhaul.

The problem, he contended, has a lot to do with the establishment of unreasonable performance standards used to judge school success.

“If you ask a dedicated teacher to make modest improvements in what they’re doing — and offer them some help in doing it — most of them will try to do that,” he says. “But if you just set very, very high targets and say, ‘By whatever magical means you can come up with, you have to [reach these targets],’ that’s asking people to cut corners — and they do. They focus on the test itself rather than on instruction, and that produces inflated test scores.”

But when it comes to eliminating the SAT, said Koretz, the problem is complex. The standard tests, he said, while not, by themselves, adequate measures of a student’s academic ability, can help an admissions officer attempting to compare students by providing a more complete academic picture.

Evidence suggests, he said, that schools from low-income areas have more lenient grading standards. So trying to compare a student from such a school district with another from a school with more rigorous standards can be tricky.

“What test scores do is provide something that is independent of grade inflation or the school’s standards. It means the same thing no matter where the kid comes from — that’s something grades simply just don’t do.”

Currently, Koretz is on sabbatical, but his work in the area of testing continues, this time with an international bent. He is chairing the International Project for the Study of Accountability Systems, an international network of educators who are, as its Web site states, “working on the design of test-based monitoring and accountability systems.”

Ultimately, said Koretz, the main problem, when it comes to educational testing, is that people have simply come to expect too much.

“We’ve put so much emphasis on one indicator that we’ve gotten people to misbehave, and if we just kind of calmed down about it and said, ‘All right, it’s a really valuable indicator of what schools are producing, but it’s only one,’ I think we would make much better progress.”