What if AI could help students learn, not just do assignments for them?

Educators’ concerns are running high when it comes to AI and how it may undermine teaching and learning. But what if teachers could find ways to use AI to measurably help students learn, rather than simply do their work for them?

Across campus, professors are experimenting with AI “tutor bots” to help students succeed in courses with complicated material when individualized attention is not always feasible. According to Professors Greg Kestin and Kelly Miller, the bots represent one piece in a larger learning strategy that helps students home in on what they do get, and don’t, and do so at their own pace.

As part of an AI Innovations at Harvard presentation during Boston’s AI week, Kestin and Miller shared findings from a study of the custom tutor bot used in their introductory course “Physical Sciences 2” in fall 2023. Based on their findings, Kestin said they’re optimistic about the technology.

“What we have found is that when AI tutors are used with tried-and-true research principles kept in mind, they can improve learning as well as enhance time spent with peers and instructors in the classroom,” he said.

In their study, Kestin and Miller compared a typical classroom setting in which students initially get lessons from a human instructor, to an AI-supported “flipped classroom” where students learn core lessons from the tailored bot on their own and bring questions to class.

The bot was built by Kestin using class materials and specific directions on how to frame answers to students.

According to the research, students in the AI-supported classroom reported significantly better engagement with the course and more motivation to learn.

A few factors may be contributing to this outcome, Miller said in the presentation. A main one is the ability to tailor the AI experience to each student.

“Our problem with a traditional classroom [is that] there can be low student engagement and students are typically not getting individualized feedback,” Miller said. “They’re mostly just passively sitting and listening to somebody speak, and so they’re not able to really test their understanding or get feedback on that understanding as they go.”

But with an AI tutor, she added, students can go at their own pace and ask as many questions as they like, any time they like — without fear of being judged.

And that’s the most important aspect of the tutor bot, the researchers said — to make sure it prompts students to think and ask questions, rather than do the thinking for them.

“We have learned that there are some places where it’s not effective to put it in the classroom,” Kestin said. “If you give ChatGPT to a group of students and you say, use it for homework, study, whatever … it turns out the students often do worse on the test because they’re using the AI to think for them. They basically circumvent the critical thinking.”

Instead, Kestin added, AI should help students by giving hints or visual representations of concepts. It can help with data analyses or generate practice problems.

“And then in exams, let AI be like their calculator. Basically, how they would do their work in the real world anyway,” Kestin said.

While their experiment was one of the first to hit Harvard classrooms following the debut of large language models like ChatGPT, many have embraced AI help for students.

Over the summer Harvard Information Technology introduced its own tutor bots and AI assistants, HUbot and PingPong. At Harvard Business School, students taking “Financial Reporting and Control” have access to their own custom tutor bot.

Last year, a section of Math 21A was offered with the AI tutor. Instructor Eva Politou is currently gathering data to assess the success in that course.

According to Kestin, more ongoing work in the field of AI tutors involves “investigating the qualities and types of interactions between students and chatbots that prove most valuable to the learning experience.”

That way, he said, the AI tutors can continue to improve and potentially find their way into even more classrooms. Kestin added that follow-up research is in progress focused on understanding the long-term impacts of tutor bots, including retention.