Measuring effective teaching

Tom Kane’s 6-year-old son recently offered the Harvard Graduate School of Education (HGSE) professor insight into the tricky nuances of assessing complex educational data. The boy’s teaching tool: popular superheroes.

He showed me that “if you are going to have a team of superheroes, you don’t want three supermen, you want three people each with different superpowers.”

The lesson, Kane said, directly applies to his work aimed at measuring effective teaching. “You shouldn’t be looking for the most super of the supermen, you should be looking for different measures that have different strengths.”

The superman metaphor applies to classroom observations, student surveys of teachers, and student achievement on state tests, three measures that are core parts of the “Measures of Effective Teaching” project, an ongoing study aimed at finding out exactly what good teaching looks like. While each measure has its own strengths, none individually can fully predict teacher effectiveness. But combining them, said Kane, balances out their weaknesses, while capitalizing on their strengths.

Kane was at HGSE  for an Askwith Forum on Jan. 26 to discuss the most recent findings from the initiative. Launched in 2009, the project, backed by the Bill & Melinda Gates Foundation and led by Kane, is the largest effort of its kind to collect video observations, student perceptions, and assessments of student achievement and teacher knowledge.

The recent findings focus largely on classroom observations.

Classroom observations have the “highest potential for diagnostic insights,” said Kane, HGSE professor of education and economics. For such observations to be successful, raters need to be given a clear set of standards of effective teaching and carefully trained to identify such standards. Teachers also need to be observed multiple times, Kane said, to help account for discrepancies in scoring that may be attributable to a certain rater’s judgment, a particular lesson, or even a certain group of students.

One important approach, said Kane, “is to get multiple observations” and average them.

The recent findings also reveal that the combination of the researchers’ three key measures did better than “master’s degrees and years of experience in predicting which teachers would have large gains with another group of students.”

The project includes 3,000 teachers from seven urban school districts: Charlotte, N.C.; Dallas; Denver; Memphis, Tenn.; New York City; Pittsburgh; and Hillsborough County, Fla.

Kane conducted his work with the help of Harvard’s Center for Education Policy Research. Ronald Ferguson, a senior lecturer in education and public policy at HGSE and the Harvard Kennedy School, helped to develop the student questionnaires for the research.