It starts with the best of intentions. When policymakers push a new strategy or program, they usually do so believing their efforts will solve a problem or improve some vital aspect of life for the community. But when the rubber meets the road, things don’t always go as planned, especially in the developing world where administrative capacity, along with access to information and training, is weak.
Evidence for Policy Design (EPoD) launched six years ago at Harvard Kennedy School (HKS) with the belief that using economic insights to inform the design, implementation, and recalibration of policy solutions and working in close collaboration with local policy actors is the key to building governments that benefit all citizens. EPoD now runs research and policy partnerships in 13 countries, including India, Pakistan, Indonesia, and Saudi Arabia, around important issues such as public health, education, labor, and finance, among others.
“The policymakers that we work with are largely constrained not by a lack of will, but by a lack of resources. They lack training in using and assessing evidence, and they don’t have access to good information,” said Rohini Pande, co-director of EPoD and the Mohammed Kamal Professor of Public Policy at HKS. “But they are the solution to the challenges faced by their countries. Our goal is to give them the tools to make a difference.”
Using a multistep, soup-to-nuts approach they call “smart policy design,” EPoD researchers begin by sitting down with a broad swath of stakeholders, from politicians and bureaucrats to civic organizations, nonprofits, and even private entities, in order to accurately and jointly identify the problems at hand.
Typically, outside researchers are asked to come in after the fact to evaluate the success or failure of policy reforms and rollouts. “And that’s too late,” said Asim Khwaja, EPoD’s other co-director and the Sumitomo-Foundation for Advanced Studies on International Development Professor of International Finance and Development at HKS. “I can’t help you improve your program, because it’s already done or it’s committed to being done. So our approach said, ‘What if the very genesis of what we’re doing is done together?’”
“If you want to take policy design seriously, you really need to spend time upfront doing diagnostic work with the policy partners,” said Pande, whose work includes helping state and federal government agencies in India determine how best to implement policies, ranging from workfare programs to environmental regulation, to ensure that citizens actually benefit. “And you need to recognize that the distinction between design and implementation is a very artificial one.”
After problems are identified, researchers and their policy counterparts dig in to uncover the underlying causal failures, a dynamic process that draws upon insights from people at all levels in an organization, not just top administrators. This yields greater insight about the root problem and helps create broader institutional buy-in so reform efforts aren’t tied to a single politician-champion.
Rema Hanna, a professor of public policy at HKS, runs several projects to assist the Indonesian government in delivering assistance to the poor.
“In developing countries, this is actually really challenging because the very people you want to reach are not part of the formal economy, and so there is no paper trail,” she said. “It’s a problem that a lot of countries face in terms of trying to provide social services: How do you measure and verify who’s actually poor?”
Strategies are custom-designed. “Best practice has no value if it doesn’t solve someone’s problems,” Khwaja said of the tendency in developing countries to blindly copy the reforms and practices used in the developed world, largely because that affords political safe harbor if an initiative doesn’t work out.
To make sure policy reform delivers in real life and not just on paper requires informed calibration. Researchers build a diagnostic “black box” into the policy to gather data on how well things are running and identify where breakdowns occur.
“Things will change. Even if the policy was working brilliantly in the first six months or the year, it may change,” said Khwaja, who works on education in Pakistan, among other subjects. “The only way you can be smart about it is to know and monitor when that change is happening so you can readjust.”
While the “black box” can be an effective tool, it can be a hard sell for politicians. “You never want to say, ‘I know my reform isn’t perfect’ … you’re announcing failure in advance,” he said. “The political economy of this is pretty tricky.”
So far, EPoD’s cyclical template has proven successful in countries where researchers have deep, existing networks. “We’ve been experimenting with training policymakers in India and Pakistan,” said Dan Levy, senior lecturer in public policy and faculty chair of the HKS SLATE Initiative. “Through a combination of online and in-person work, they gain the skills to become better consumers of evidence.” Faculty now teach smart policy design in the degree programs and in HKS’ executive education program as a means to further refine and enhance its portability.
EPoD is now expanding its approach into places where it lacks a prior network. One promising example of this is a five-year engagement with the ministry of labor in Saudi Arabia to conduct research in a host of areas, including unemployment insurance, women in the labor force, job search platforms and job placement. The smart policy design template is being used to draw in economics researchers and policy practitioners from outside HKS.
EPoD Director Deanna Ford believes the template could eventually expand to new contexts where it could benefit Harvard researchers in other fields who also work closely with practitioners.
“As we’ve created this research-meets-practice world in the Kennedy School, why not the Business School, why not the Law School, why not the School of Public Health?” she said.
For young researchers, EPoD’s fellows program offers opportunities to collaborate with faculty and supports extensive fieldwork, and also provides access to high-level policymakers and rare data sets while tackling authentic issues in real time.
Given the prolonged nature of policymaking and research, Hanna said directly engaging with field partners provides an immediate sense of accomplishment that waiting years for a paper to be published does not. “One of the things that keeps me motivated is knowing that I helped a particular group solve a particular problem. It feels very good.”
Recently, EPoD’s purview has expanded to include the training of bureaucrats and civil servants so they can better implement the new reforms. With access to greater resources, they work to increase the number of projects funded and offer more support to governments so they can better understand their own policy challenges and take fuller advantage of the data available to make smart decisions.
In the end, truly measuring the effectiveness of EPoD’s approach will be a matter of longevity, say Khwaja and Pande. “The true test for people like us will be in 10 or 15 years. Can we look back and say: Did this change fundamentally a policy discourse or a practice discourse?”
“We build sustainability into our process from the outset,” said Ford. “So the real marker of our success is that moment when we’re no longer needed, when local policymakers and citizens have all the tools and information they need to make their institutions work for the benefit of all.”