Good medicine requires high-quality care and top-notch research. But it also requires a willingness to adapt, said Atul Gawande, a surgeon at Harvard-affiliated Brigham and Women’s Hospital and the Dana-Farber Cancer Institute who is also a staff writer for The New Yorker and a best-selling author.

He spoke to a packed house at Sanders Theatre on Wednesday about “The Battle for the Soul of Medicine.” Sponsored by the Harvard Institute for Learning in Retirement (HILR), which is part of the Harvard Division of Continuing Education, Gawande delivered the Robert C. Cobb Memorial Lecture of 2011.

Gawande, whose latest book is “The Checklist Manifesto,” suggested that the challenge of making health care both effective and affordable may be traced back to the discovery of a miracle drug: penicillin. “We were fooled by penicillin,” Gawande said. “It was so simple: just an injection. We came to expect that sort of miracle from medicine.”

In modern medicine, Gawande said, there are 13,600 diagnoses, or ways in which the human body can fail, and no patient comes in with just one diagnosis at a time. Now more than 6,000 drugs can be prescribed, and 4,000 medical and surgical procedures can be performed. Because of that, medicine can make people’s lives significantly better. Yet many Americans still struggle to pay for health care.

“Health care costs are destroying American prosperity,” said Gawande, who is also an associate professor in the Department of Health Policy and Management in the Harvard School of Public Health and associate professor of surgery at Harvard Medical School. “But the field of medicine is incredibly complex. How do we, in the field, deal with that complexity? It’s man’s most ambitious endeavor. Is it any surprise that we’re finding it hard?”

Indeed, Gawande said, the traditional solo physician “simply can’t do it alone anymore. We can’t remember it all. Modern medicine requires teams of people. The modern health care system needs to develop new skills.”

Among those skills, Gawande said, is the ability to recognize success and failure, the capacity to devise solutions for the challenges discovered, and the means to implement solutions. One example of devising solutions is the focus of Gawande’s book “The Checklist Manifesto.”

As medicine has improved, Gawande explained, the need for surgical care “has exploded. The World Health Organization asked us to find a way to reduce surgery deaths worldwide. We found that other groups had training and technology — which medicine certainly has — but in addition, these other fields had checklists.”

Gawande pointed out that in other fields, such as air travel, checklists are built into the daily routine of the staff, from the top down. “The checklists start at the top of the totem pole. It’s seen as integral to the team’s success.” After developing a two-minute, 19-item checklist for the medical field and testing it in eight cities, including London and Seattle, Gawande’s team found that “complications from surgeries fell in every location by an average of 47 percent.”

Despite this success, Gawande found that the checklist is not always popular in the medical field. Therefore, it’s the challenge of implementing the solution — not discovering the solution itself — that can prove the most challenging in some areas.

“There are deeper issues,” he said. “There is resistance. When you could hold it all in your head, you needed autonomy and independence.” When working with today’s teams of specialists in the operating room, however — what Gawande called “pit crews for patients” — the skill set shifts dramatically.

“With teams as large as 15 people for one procedure, you need humility. You need discipline. You need teamwork. In that environment, everyone, no matter how low on the totem pole, can help you achieve your goals.”

“This is the battle for our generation, for the soul of American medicine, and even for the American dream,” Gawande said.

Still, when asked by a student what advice he had for people entering today’s medical field and facing this enormous challenge, Gawande said he felt lucky.

“Right now, everything is open to interpretation and innovation. There is some glimmer of the enormous amount of invention going on in America. There are entrepreneurial minds trying to help us focus on results. For those who like the possibility of being able to lead, to make care better in your part of the world, the field is wide open.”

The sponsoring HILR is a membership organization, now celebrating its 35th anniversary year. “HILR offers retired leaders of government, industry, academia, and the professions a curriculum that reflects the liberal arts tradition of the University,” said Leonie Gordon, assistant dean and director of HILR.