Teaching educators to be data wise

Last week, in public schools across Massachusetts, students were racking their brains to show what they know on the Massachusetts Comprehensive Assessment System (MCAS) test. The test results, which will be released in the fall, will provide data that show students’ proficiency in English language arts, mathematics, and science and technology.

So where does it go from there?

Harvard Graduate School of Education (HGSE) students are working to turn these test scores into a viable teaching tool. Their class, called “Using Student Assessment Data to Improve Instruction: A Workshop,” pairs principals and other education leaders from eight Boston public schools with eight HGSE students. The work begins by having each school take a hard look at its MCAS data.

“We always start the year looking at standardized test scores because we feel it’s a credible, logical starting point … everyone knows you’re supposed to care about what these scores are,” said Kathryn Parker Boudett, adjunct lecturer on education and instructor of the course. “Our big punch line is that looking at MCAS scores is going to raise more questions than it’s going to answer.”

MCAS will tell you where a school’s problems are, she continued, but it won’t tell you what’s going wrong with the instruction that leads to the problems. “You can use MCAS to get a feel for where your specific problems are, like reading, math, or writing, but then quickly you need to start looking at other types of data.”

And by other types of data, Boudett means any of the ways that student work is evaluated: tests, quizzes, papers, class participation, journals, etc. And it doesn’t only include student assessment: Boudett encourages the school leaders to go into their school’s classrooms and observe how the teaching is occurring. “Because that’s data, too,” she said.

After looking at the mix of test scores and other types of data, Boudett has each team identify a “learner-centered problem.” Then, the problem must be restated, said Boudett, as a “problem of practice,” or as a problem of teaching. “[Teachers] are the ones who are going to have to learn to do something differently so that the learner-centered problem is addressed,” said Boudett. “You’re recasting your test results as something you can do something about instructionally.”

After the instruction in the classroom has been adjusted, then teachers and school leaders can return to the MCAS and check their progress toward goals they set when they first developed their action plans, goals such as “We hope within three years our writing scores will improve,” she added.

Ludwig van Beethoven Elementary

In the real world, school leaders are reaping the benefits of the process. As Eileen Nash, principal of the Ludwig van Beethoven Elementary School in West Roxbury, put it, “Before I took the course, we were analyzing our data, but we weren’t doing it in an efficient manner. This course has allowed us to become much more focused and refined in not just how we look at data, but in identifying the next steps once the data is analyzed.” She added, “The course is having a much bigger impact on my school than I anticipated.”

Julian Underwood, the HGSE student who works with Nash, said, “Schools are going through a process where they’re expected to use data more and more to improve student learning, but the amount of data can be very intimidating. If you’re a school like the Beethoven with 250-300 students, you have what seems like an insurmountable amount.”

“The biggest challenge to the productive use of data is time,” said Nash. There’s lots of data available, but a finite amount of time to decipher it. The key, as taught in the course, is to identify information that points to a particular problem and then separate that from the rest so the school can focus on what’s wrong.

After reviewing recent MCAS results, the Beethoven team decided to focus on substandard student writing. To gather additional information about how writing is being taught and learned, the teachers at Beethoven revisited written assignments to analyze the evaluation process. “We then did a schoolwide writing prompt [written assignment] that was scored by everyone on the staff. This gave cross-grade levels a chance to see what skills the students are either mastering or what skills the students still need to work on,” said Nash.

The process was illuminating for the teachers at Beethoven. “The teachers are all very well intentioned,” said Nash, “But it’s hard to look at your own practice and be really reflective about how your instructional strategies are getting across to the kids.”

Having other teachers observe and identify what they saw as successful or unsuccessful teaching strategies informed them about what were best teaching practices, Nash explained. And reframing the problem from a learner-centered problem (“The kids don’t write well”) to a practice-centered problem (“We aren’t teaching the elements of good writing”) gave more ownership to the teachers.

As a result of the course and this analysis, Nash does see teaching changing. Beginning next year, all teachers will be using a different model – the six-trait model – for teaching writing. The model is a way of assessing and teaching writing that focuses on six qualities seen in outstanding written works.

Mixing the pros with the students

The HGSE students and the school leaders appreciated what each other brought into the classroom. “A lot of the courses we get are very theoretical,” said Underwood. “This is a course that bridges the gap between theory and practice by including practitioners in every class.”

For the purposes of the class, Boudett said that it’s “powerful” to have the principals of schools enrolled. “If a principal is sitting there working on the project, it really increases the chance that something will actually happen with the assignment.”

Also, Boudett said that one of the goals of the class is to allow the Harvard students an opportunity to see the process as it plays out in action. People can get defensive, especially when the data show that things aren’t going well. “We think it’s important for Harvard students to see that taking care of how you have the conversations around data is more important than creating a cool-looking graph. … If you can bring everyone into the conversation in a safe way to talk about the data without fear of being blamed, then you can make some progress.”

The major goal is to actually improve the teaching at Boston public schools. “We can make our curriculum just right and we can build in a course design that allows students to get into the schools and see what happens, but making sure that something changes at the schools is the hardest part,” said Boudett. “You can imagine that if schools were able to easily change what they do, then they would’ve done it already.”

Not all of the participating schools make as much progress as did Beethoven toward improving instruction.

“But what we do hear,” said Boudett, “is that every school has made some progress, every one of them has moved the ball somewhat.”