Are top-ranked colleges really the best? Depends. At what?

Traditional college rankings gauge little more than an institution’s selectivity on grades and standardized test scores. They say nothing about which schools are good at leveling the playing field, particularly for first-generation students and those from under-represented groups.

“Essentially, the rankings don’t account for anything that happens after students walk through the college gates,” said Christina Ciocca Eller, assistant professor of sociology and social studies. 

In a paper published this month in the Journal of American Sociology, Ciocca Eller calls for new government and media ratings that measure how U.S. institutions of higher education shape the outcomes of those they enroll. That means relying on metrics that look beyond pre-college achievement to compare actual outcomes across diverse populations. 

“I argue that one of the very important things we don’t rank is the equalizing effect of colleges,” Ciocca Eller said. 

Her paper even models one approach. Ciocca Eller worked with an anonymized data set representing every student over roughly 20 years in a public college system that enrolls nearly 250,000 students annually. 

She then compared graduation rates across various schools, majors, and demographics. There may be other ways to gauge a college’s equalizing effect, she said in an interview. Perhaps another researcher would opt to study long-term earnings. 

“But I think it’s really valuable to look at degree completion,” she said, citing a large body of research establishing the positive educational and job-market impacts of holding a bachelor’s. “With this credential in hand,” the sociologist added, “many different kinds of opportunities open.”

“Students’ experiences and outcomes in U.S. higher education are highly unequal based on their racial and socioeconomic backgrounds,” Ciocca Eller writes in a research brief posted to her personal website. Black, Latinx, Native, and lower-income students are, for example, less likely than their comparatively privileged counterparts to attend elite colleges, which produce high graduation rates across demographics.

What’s more, students from traditionally underrepresented groups who attend non-selective colleges are less likely, on average, to complete their degrees.

Ciocca Eller emphasized that colleges like those in her sample should not necessarily be penalized for low graduation rates. After all, their student bodies face disproportionate obstacles to success, compared with those at more elite schools — from insufficient academic preparation to economic stressors and extra family responsibilities. Understanding each school’s equalizing effect requires a “really close measurement” of its influence on these students and their peers, she said. 

Ciocca Eller’s data analysis showed that nine of the 11 colleges in her study produced strong equalizing effects. That means the schools narrowed degree-completion gaps between students from traditionally underrepresented and well-represented groups. In addition, three of the colleges elevated degree completion rates for underrepresented groups “above what they would be if each college were to have no impact or a negative impact on students’ BA outcomes,” she wrote. 

Additionally, Ciocca Eller identified a few particulars that drive or diminish the equalizing effect. Some less surprising factors associated with positive results included high levels of full-time student enrollment and strong first-year retention. A big surprise associated with negative outcomes concerned schools with high numbers of STEM majors.

“The takeaway is not to lessen the proportion of students in STEM majors,” she argued. “It’s to work harder in figuring out how to enable traditionally disadvantaged students to both get into and stay in STEM majors.”

“The takeaway is not to lessen the proportion of students in STEM majors. It’s to work harder in figuring out how to enable traditionally disadvantaged students to both get into and stay in STEM majors.”

Christina Ciocca Eller

“Another unexpected finding concerned the amount of spending on academic support services,” Ciocca Eller said. “The intuitive thing would be that spending more money on academic support would increase graduation outcomes. But in this system, I’m not actually finding that. And that is in part because colleges are spending more on academic support when they have the most at-need populations.”

Ciocca Eller’s interest in college equity was sparked more than a decade ago when she worked as a speechwriter for Georgetown University President John J. DeGioia. “I got really engaged and thoughtful about what colleges actually do in people’s lives, especially for first-generation college students,” she recalled. “Many of these students were interns in our office, and they told me about their experiences.”

That experience inspired her path to Columbia University, where she earned a Ph.D. in sociology in 2019 while completing her dissertation on the role colleges play in driving disparities in degree completion. After landing her Harvard faculty role, Ciocca Eller was appointed assistant director for evidence and policy in the Biden Administration’s Office of Science and Technology Policy, where she served for 18 months. 

In her research summary, she advocates for reversing the current ban on collecting student-record data, as currently proposed in the U.S. House of Representatives under the College Transparency Act. She also calls for bolstering the federal government’s Integrated Postsecondary Education Data System (IPEDS for short) to include disaggregated information on completion rates. Both changes would help facilitate rankings that account for the equalizing effect at every U.S. postsecondary institution.

For now, Ciocca Eller’s study is the first to rigorously interrogate the equalizing effects of non-elite public colleges, responsible for educating a full 85 percent of all U.S. college entrants and far more accessible to the average applicant who lacks a dazzling standardized test score. 

“This is a bit of an intervention,” she said. “What I’m saying is — Hey! Can we please focus on this larger population of people going to college?”