{"id":233076,"date":"2017-11-09T14:00:46","date_gmt":"2017-11-09T19:00:46","guid":{"rendered":"https:\/\/news.harvard.edu\/gazette\/?p=233076"},"modified":"2023-11-08T20:56:39","modified_gmt":"2023-11-09T01:56:39","slug":"small-media-can-have-broad-impact-on-the-national-conversation","status":"publish","type":"post","link":"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/","title":{"rendered":"Small media, big payback"},"content":{"rendered":"\n\t\n\t
\n\t\t\t\n\t\t\tScience & Tech\t\t<\/a>\n\t\t\n\t\t

\n\t\tSmall media, big payback\t<\/h1>\n\n\t\n\t\n\t
\n\t\t
\n\t\t\t
\n\t\t\t\t\t

\n\t\tPeter Reuell\t<\/p>\n\t\t\t

\n\t\t\tHarvard Staff Writer\t\t<\/p>\n\t\t\t\t\t<\/address>\n\t\t<\/div>\n\n\t\t

\n\t\t\tStudy shows that such outlets can have broad impact on the national conversation\t\t<\/h2>\n\t\t\n<\/header>\n\n\n\n
\n\n\n\t\t

With a readership in the millions, The New York Times routinely influences public debate on a host of issues through its news coverage.<\/p>\n

Can a small news outlet of perhaps 50,000 circulation do the same thing?<\/p>\n

The answer, says Albert J. Weatherhead III University Professor Gary King, is that in an age that relies on internet publication and social-media dispersal, even small- to medium-size media outlets can have a dramatic impact on the content and partisan balance of the national conversation about major public-policy issues.<\/p>\n

In the first large-scale, randomized media experiment<\/a> of its kind, King and former students Benjamin Schneer, now an assistant professor at Florida State University, and Ariel White, now an assistant professor at Massachusetts Institute of Technology, found that if just three outlets wrote about a major national policy topic \u2014 such as jobs, the environment, or immigration \u2014 discussion of that topic across social media rose by more than 62 percent, and the balance of opinion in the national conversation could be swayed by several percentage points.<\/p>\n

\u201cFor several hundred years, scholars have tried to measure the influence of the media. Most people think it is influential, but measuring this influence rigorously with randomized experiments has until now been impossible,\u201d King said. \u201cOur findings suggest that the effect of the media is surprisingly large. Our study\u2019s implications suggest every journalist wields a major power, and so has an important responsibility.\u201d<\/p>\n

Those findings, King said, are the result of more than five years of work, much of it spent convincing 48 news outlets to agree to take part in the study. About half of these outlets were represented by the Media Consortium, a network of independent news outlets that was eager to find a way to measure impact and was willing to help.<\/p>\n

\u201cMuch of the work leading up to this study involved finding a way to bridge the cultural divide between journalism and science,\u201d King said. \u201cThrough years of conversations, much trial and error, and a partnership with Media Consortium Executive Director Jo Ellen Green Kaiser, we learned to understand journalistic standards and practices, and the journalists learned to understand our scientific requirements. What ultimately made it all work was a novel research design we developed that satisfied both camps.\u201d<\/p>\n\r\n\t\n\n\t

\"\"
Credit: Carla Schaffer\/AAAS\t\t\t<\/figcaption><\/figure>\n\t\n\t\r\n\n

Though similar efforts have been tried in the past, they invariably collapsed as journalists chafed at the idea of being told what to report and when to report it. To address the problem, previous researchers fell back on clever tricks, such as studying areas that fell outside the broadcast area of a particular outlet but, because no one knew whether the areas were truly random, were hard to evaluate. Such studies faced many problems, particularly their inability to control for a host of factors such as race, education, or income.<\/p>\n

\u201cFrom a scientific point of view, we have to be able to tell the journalists what to publish, and preferably at random times,\u201d King said. \u201cFrom a journalistic point of view, these scientific requirements seem crazy, and journalists reasonably insist on retaining absolute control over what they publish. The two sets of requirements seem fundamentally incompatible, but we found a way to create a single research design that accomplished the goals of both groups.\u201d<\/p>\n

It wasn\u2019t only the participation of news outlets that made the study noteworthy, though.<\/p>\n

\u201cIf you\u2019re doing something like a medical experiment, you may randomly assign individual people to one of two groups, and then each person is your unit of analysis,\u201d he said. \u201cBut when a media outlet publishes something \u2014 no matter how small \u2014 the potential audience it could impact includes basically everybody in the country. That means our unit of analysis can\u2019t be a person. It has to be the entire country, which greatly increases the cost of the study.\u201d That means that the equivalent of an entire experiment in many other studies constitutes only one observation in this study.<\/p>\n

Because collecting each observation was so expensive and logistically challenging, King and colleagues used, and further developed, novel statistical techniques to enable them to collect only as much data as needed. After each massive national experiment they could then examine whether they had amassed enough data to draw reliable conclusions.<\/p>\n

\u201cThat allowed us to keep going until we got to the point where we had exactly as much data as we needed, and no more,\u201d King said. \u201cAs it turns out, we ran 35 national experiments that produced 70 observations.\u201d<\/p>\n

To achieve the randomization needed for the study, King\u2019s team, the Media Consortium staff, and journalists at the 48 outlets identified 11 broad policy areas. They then simulated the tendency of journalists to influence each other and publish stories on similar topics, sometimes called \u201cpack journalism,\u201d by choosing three or four outlets from their participating group of 48 to develop stories together that fell into the same broad policy area.<\/p>\n

\u201cFor example, if the policy area was jobs, one story might be about Uber drivers in the Philadelphia area,\u201d King said. \u201cWe would then identify a two-week period where we predicted there wouldn\u2019t be any surprises related to that topic area. So if the president was planning to give a speech about immigration in one of those two weeks, we would not run an experiment on immigration during that time.\u201d<\/p>\n

Randomization came from researchers flipping a coin to determine which of those weeks would be the publication week, and which the control week.<\/p>\n

\u201cAt first, our outlets didn\u2019t really understand what randomization meant,\u201d said Kaiser. \u201cOur project manager, Manolia Charlotin, and the researchers worked very closely with all the outlets to ensure they followed the researchers\u2019 rules. This was a resource-intensive project for us, but the unexpected benefit was that outlets found they also gained many qualitative benefits from collaborating.\u201d<\/p>\n

In both treatment and control weeks, King, Schneer, and White used tools and data from the global analytics company Crimson Hexagon to monitor the national conversation in social media posts. (King is a co-founder of Crimson Hexagon; with a previous generation of graduate and postdoctoral researchers, he developed the automated text analysis technology that Harvard licensed to the company.) He explained that this methodology \u201cis used to evaluate meaning in social media posts. So if you have a set of categories you care about, we identify example posts in these categories, which is what humans are good at,\u201d King said. \u201cThen our algorithm can amplify that human intelligence and, without classifying individual posts, can accurately estimate the percent of posts in each category each day.\u201d<\/p>\n

What they found, King said, was that the effect was larger than anticipated.<\/p>\n

\u201cThe actual effect is really big,\u201d King said. \u201cIf three outlets (with an average circulation of about 50,000) get together and write stories, the size of the national conversation in that policy area increases a lot. It\u2019s a 62 percent increase on the first day\u2019s volume distributed over the week, just from these three little outlets.<\/p>\n

\u201cThese national conversations about major policy areas are essential to democracy,\u201d he added. \u201cToday this conversation takes place, in part, in some of the 750,000,000 publicly available social media posts written by people every day \u2014 and all available for research. At one time, the national conversation was whatever was said in the public square, where people would get up on a soapbox, or when they expressed themselves in newspaper editorials or water-cooler debates. This is a lot of what democracy is about. The fact that the media has such a large influence on the content of this national conversation is crucial for everything from the ideological balance of the nation\u2019s media outlets, to the rise of fake news, to the ongoing responsibility of professional journalists.\u201d<\/p>\n

This research was supported with funding from Voqal and Harvard\u2019s Institute for Quantitative Social Science.<\/em><\/p>\n\n\n<\/div>\n\n\t\t","protected":false},"excerpt":{"rendered":"

Researchers found that if just three outlets write about a particular major national policy topic, discussion of that topic across social media rises by more than 62 percent.<\/p>\n","protected":false},"author":108352576,"featured_media":233086,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"gz_ga_pageviews":22,"gz_ga_lastupdated":"2021-12-16 16:22","document_color_palette":null,"author":"Peter Reuell","affiliation":"Harvard Staff Writer","_category_override":"","_yoast_wpseo_primary_category":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1387],"tags":[12941,13050,14078,15359,15970,20845,39375,39376,39374,25585,27327,39377,28373,39378,29235],"gazette-formats":[],"series":[],"class_list":["post-233076","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-science-technology","tag-faculty-of-arts-and-sciences","tag-fas","tag-gary-king","tag-harvard","tag-news","tag-king","tag-national-conversation","tag-national-policy","tag-news-coverage","tag-newspapers","tag-peter-reuell","tag-policy-area","tag-public-opinion","tag-randomized-experiment","tag-reuell"],"yoast_head":"\nSmall media can have broad impact on the national conversation — Harvard Gazette<\/title>\n<meta name=\"description\" content=\"Researchers found that if just three outlets write about a particular major national policy topic, discussion of that topic across social media rises by more than 62 percent.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Small media can have broad impact on the national conversation\" \/>\n<meta property=\"og:description\" content=\"Researchers found that if just three outlets write about a particular major national policy topic, discussion of that topic across social media rises by more than 62 percent.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/\" \/>\n<meta property=\"og:site_name\" content=\"Harvard Gazette\" \/>\n<meta property=\"article:published_time\" content=\"2017-11-09T19:00:46+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-11-09T01:56:39+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/news.harvard.edu\/gazette\/wp-content\/uploads\/2017\/11\/091411_dataverse_046_605.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"605\" \/>\n\t<meta property=\"og:image:height\" content=\"403\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"gazettejohnbaglione\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:title\" content=\"Small media can have broad impact on the national conversation\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/\"},\"author\":{\"name\":\"gazettejohnbaglione\",\"@id\":\"https:\/\/news.harvard.edu\/gazette\/#\/schema\/person\/99782494e562769a740295b11ce6dafe\"},\"headline\":\"Small media, big payback\",\"datePublished\":\"2017-11-09T19:00:46+00:00\",\"dateModified\":\"2023-11-09T01:56:39+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/\"},\"wordCount\":1301,\"publisher\":{\"@id\":\"https:\/\/news.harvard.edu\/gazette\/#organization\"},\"image\":{\"@id\":\"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/news.harvard.edu\/wp-content\/uploads\/2017\/11\/091411_dataverse_046_605.jpg\",\"keywords\":[\"Faculty of Arts and Sciences\",\"FAS\",\"Gary King\",\"Harvard\",\"Harvard News\",\"King\",\"national conversation\",\"national policy\",\"news coverage\",\"Newspapers\",\"Peter Reuell\",\"policy area\",\"Public Opinion\",\"randomized experiment\",\"Reuell\"],\"articleSection\":[\"Science & Tech\"],\"inLanguage\":\"en-US\",\"copyrightYear\":\"2017\",\"copyrightHolder\":{\"@id\":\"https:\/\/news.harvard.edu\/gazette\/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/\",\"url\":\"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/\",\"name\":\"Small media can have broad impact on the national conversation — Harvard Gazette\",\"isPartOf\":{\"@id\":\"https:\/\/news.harvard.edu\/gazette\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/news.harvard.edu\/wp-content\/uploads\/2017\/11\/091411_dataverse_046_605.jpg\",\"datePublished\":\"2017-11-09T19:00:46+00:00\",\"dateModified\":\"2023-11-09T01:56:39+00:00\",\"description\":\"Researchers found that if just three outlets write about a particular major national policy topic, discussion of that topic across social media rises by more than 62 percent.\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/#primaryimage\",\"url\":\"https:\/\/news.harvard.edu\/wp-content\/uploads\/2017\/11\/091411_dataverse_046_605.jpg\",\"contentUrl\":\"https:\/\/news.harvard.edu\/wp-content\/uploads\/2017\/11\/091411_dataverse_046_605.jpg\",\"width\":605,\"height\":403,\"caption\":\"Gary King, Albert J. Weatherhead III University Professor works in his CGIS Knafel office and with the building's computer server at Harvard University. Kris Snibbe\/Harvard Staff Photographer\"},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/news.harvard.edu\/gazette\/#website\",\"url\":\"https:\/\/news.harvard.edu\/gazette\/\",\"name\":\"Harvard Gazette\",\"description\":\"Official news from Harvard University covering innovation in teaching, learning, and research\",\"publisher\":{\"@id\":\"https:\/\/news.harvard.edu\/gazette\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/news.harvard.edu\/gazette\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/news.harvard.edu\/gazette\/#organization\",\"name\":\"The Harvard Gazette\",\"url\":\"https:\/\/news.harvard.edu\/gazette\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/news.harvard.edu\/gazette\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/news.harvard.edu\/wp-content\/uploads\/2023\/12\/Harvard_Gazette_logo.svg\",\"contentUrl\":\"https:\/\/news.harvard.edu\/wp-content\/uploads\/2023\/12\/Harvard_Gazette_logo.svg\",\"width\":164,\"height\":64,\"caption\":\"The Harvard Gazette\"},\"image\":{\"@id\":\"https:\/\/news.harvard.edu\/gazette\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/news.harvard.edu\/gazette\/#\/schema\/person\/99782494e562769a740295b11ce6dafe\",\"name\":\"gazettejohnbaglione\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Small media can have broad impact on the national conversation — Harvard Gazette","description":"Researchers found that if just three outlets write about a particular major national policy topic, discussion of that topic across social media rises by more than 62 percent.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/","og_locale":"en_US","og_type":"article","og_title":"Small media can have broad impact on the national conversation","og_description":"Researchers found that if just three outlets write about a particular major national policy topic, discussion of that topic across social media rises by more than 62 percent.","og_url":"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/","og_site_name":"Harvard Gazette","article_published_time":"2017-11-09T19:00:46+00:00","article_modified_time":"2023-11-09T01:56:39+00:00","og_image":[{"width":605,"height":403,"url":"https:\/\/news.harvard.edu\/gazette\/wp-content\/uploads\/2017\/11\/091411_dataverse_046_605.jpg","type":"image\/jpeg"}],"author":"gazettejohnbaglione","twitter_card":"summary_large_image","twitter_title":"Small media can have broad impact on the national conversation","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/#article","isPartOf":{"@id":"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/"},"author":{"name":"gazettejohnbaglione","@id":"https:\/\/news.harvard.edu\/gazette\/#\/schema\/person\/99782494e562769a740295b11ce6dafe"},"headline":"Small media, big payback","datePublished":"2017-11-09T19:00:46+00:00","dateModified":"2023-11-09T01:56:39+00:00","mainEntityOfPage":{"@id":"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/"},"wordCount":1301,"publisher":{"@id":"https:\/\/news.harvard.edu\/gazette\/#organization"},"image":{"@id":"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/#primaryimage"},"thumbnailUrl":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2017\/11\/091411_dataverse_046_605.jpg","keywords":["Faculty of Arts and Sciences","FAS","Gary King","Harvard","Harvard News","King","national conversation","national policy","news coverage","Newspapers","Peter Reuell","policy area","Public Opinion","randomized experiment","Reuell"],"articleSection":["Science & Tech"],"inLanguage":"en-US","copyrightYear":"2017","copyrightHolder":{"@id":"https:\/\/news.harvard.edu\/gazette\/#organization"}},{"@type":"WebPage","@id":"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/","url":"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/","name":"Small media can have broad impact on the national conversation — Harvard Gazette","isPartOf":{"@id":"https:\/\/news.harvard.edu\/gazette\/#website"},"primaryImageOfPage":{"@id":"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/#primaryimage"},"image":{"@id":"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/#primaryimage"},"thumbnailUrl":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2017\/11\/091411_dataverse_046_605.jpg","datePublished":"2017-11-09T19:00:46+00:00","dateModified":"2023-11-09T01:56:39+00:00","description":"Researchers found that if just three outlets write about a particular major national policy topic, discussion of that topic across social media rises by more than 62 percent.","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/#primaryimage","url":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2017\/11\/091411_dataverse_046_605.jpg","contentUrl":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2017\/11\/091411_dataverse_046_605.jpg","width":605,"height":403,"caption":"Gary King, Albert J. Weatherhead III University Professor works in his CGIS Knafel office and with the building's computer server at Harvard University. Kris Snibbe\/Harvard Staff Photographer"},{"@type":"WebSite","@id":"https:\/\/news.harvard.edu\/gazette\/#website","url":"https:\/\/news.harvard.edu\/gazette\/","name":"Harvard Gazette","description":"Official news from Harvard University covering innovation in teaching, learning, and research","publisher":{"@id":"https:\/\/news.harvard.edu\/gazette\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/news.harvard.edu\/gazette\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/news.harvard.edu\/gazette\/#organization","name":"The Harvard Gazette","url":"https:\/\/news.harvard.edu\/gazette\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/news.harvard.edu\/gazette\/#\/schema\/logo\/image\/","url":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2023\/12\/Harvard_Gazette_logo.svg","contentUrl":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2023\/12\/Harvard_Gazette_logo.svg","width":164,"height":64,"caption":"The Harvard Gazette"},"image":{"@id":"https:\/\/news.harvard.edu\/gazette\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/news.harvard.edu\/gazette\/#\/schema\/person\/99782494e562769a740295b11ce6dafe","name":"gazettejohnbaglione"}]}},"parsely":{"version":"1.1.0","canonical_url":"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/","smart_links":{"inbound":0,"outbound":0},"traffic_boost_suggestions_count":0,"meta":{"@context":"https:\/\/schema.org","@type":"NewsArticle","headline":"Small media, big payback","url":"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/","mainEntityOfPage":{"@type":"WebPage","@id":"https:\/\/news.harvard.edu\/gazette\/story\/2017\/11\/small-media-can-have-broad-impact-on-the-national-conversation\/"},"thumbnailUrl":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2017\/11\/091411_dataverse_046_605.jpg?w=150","image":{"@type":"ImageObject","url":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2017\/11\/091411_dataverse_046_605.jpg"},"articleSection":"Science & Tech","author":[{"@type":"Person","name":"gazettejohnbaglione"}],"creator":["gazettejohnbaglione"],"publisher":{"@type":"Organization","name":"Harvard Gazette","logo":"https:\/\/news.harvard.edu\/gazette\/wp-content\/uploads\/2023\/12\/Harvard_Gazette_logo.svg"},"keywords":["faculty of arts and sciences","fas","gary king","harvard","harvard news","king","national conversation","national policy","news coverage","newspapers","peter reuell","policy area","public opinion","randomized experiment","reuell"],"dateCreated":"2017-11-09T19:00:46Z","datePublished":"2017-11-09T19:00:46Z","dateModified":"2023-11-09T01:56:39Z"},"rendered":"<script type=\"application\/ld+json\" class=\"wp-parsely-metadata\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@type\":\"NewsArticle\",\"headline\":\"Small media, big payback\",\"url\":\"https:\\\/\\\/news.harvard.edu\\\/gazette\\\/story\\\/2017\\\/11\\\/small-media-can-have-broad-impact-on-the-national-conversation\\\/\",\"mainEntityOfPage\":{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/news.harvard.edu\\\/gazette\\\/story\\\/2017\\\/11\\\/small-media-can-have-broad-impact-on-the-national-conversation\\\/\"},\"thumbnailUrl\":\"https:\\\/\\\/news.harvard.edu\\\/wp-content\\\/uploads\\\/2017\\\/11\\\/091411_dataverse_046_605.jpg?w=150\",\"image\":{\"@type\":\"ImageObject\",\"url\":\"https:\\\/\\\/news.harvard.edu\\\/wp-content\\\/uploads\\\/2017\\\/11\\\/091411_dataverse_046_605.jpg\"},\"articleSection\":\"Science & Tech\",\"author\":[{\"@type\":\"Person\",\"name\":\"gazettejohnbaglione\"}],\"creator\":[\"gazettejohnbaglione\"],\"publisher\":{\"@type\":\"Organization\",\"name\":\"Harvard Gazette\",\"logo\":\"https:\\\/\\\/news.harvard.edu\\\/gazette\\\/wp-content\\\/uploads\\\/2023\\\/12\\\/Harvard_Gazette_logo.svg\"},\"keywords\":[\"faculty of arts and sciences\",\"fas\",\"gary king\",\"harvard\",\"harvard news\",\"king\",\"national conversation\",\"national policy\",\"news coverage\",\"newspapers\",\"peter reuell\",\"policy area\",\"public opinion\",\"randomized experiment\",\"reuell\"],\"dateCreated\":\"2017-11-09T19:00:46Z\",\"datePublished\":\"2017-11-09T19:00:46Z\",\"dateModified\":\"2023-11-09T01:56:39Z\"}<\/script>","tracker_url":"https:\/\/cdn.parsely.com\/keys\/news.harvard.edu\/p.js"},"jetpack_featured_media_url":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2017\/11\/091411_dataverse_046_605.jpg","has_blocks":true,"block_data":{"0":{"blockName":"harvard-gazette\/article-header","attrs":{"blockColorPalette":"","coloredHeading":"","creditText":"","displayDetails":"","displayTitle":"","categoryId":1387,"mediaAlt":"","mediaCaption":"","mediaId":"","mediaSize":"","mediaType":"","mediaUrl":"","poster":"","title":"Small media, big payback","subheading":"Study shows that such outlets can have broad impact on the national conversation","className":"is-style-square","backgroundFixed":false,"backgroundTone":"light","centeredImage":false,"coloredBackground":false,"displayOverlay":true,"fadeInText":false,"isAmbient":false,"mediaHeight":0,"mediaLength":"","mediaPosition":"","mediaWidth":0,"posterText":"","titleAbove":false,"useUncroppedImage":false,"lock":[],"metadata":[]},"innerBlocks":[],"innerHTML":"","innerContent":[],"rendered":"<header\n\tclass=\"wp-block-harvard-gazette-article-header alignfull article-header is-style-square has-light-background has-colored-heading\"\n\tstyle=\" \"\n>\n\t\n\t<div class=\"article-header__content\">\n\t\t\t<a\n\t\t\tclass=\"article-header__category\"\n\t\t\thref=\"https:\/\/news.harvard.edu\/gazette\/section\/science-technology\/\"\n\t\t>\n\t\t\tScience & Tech\t\t<\/a>\n\t\t\n\t\t<h1 class=\"article-header__title wp-block-heading has-large-text\">\n\t\tSmall media, big payback\t<\/h1>\n\n\t\n\t\n\t<div class=\"article-header__meta\">\n\t\t<div class=\"wp-block-post-author\">\n\t\t\t<address class=\"wp-block-post-author__content\">\n\t\t\t\t\t<p class=\"author wp-block-post-author__name\">\n\t\tPeter Reuell\t<\/p>\n\t\t\t<p class=\"wp-block-post-author__byline\">\n\t\t\tHarvard Staff Writer\t\t<\/p>\n\t\t\t\t\t<\/address>\n\t\t<\/div>\n\n\t\t<time class=\"article-header__date\" datetime=\"2017-11-09\">\n\t\t\tNovember 9, 2017\t\t<\/time>\n\n\t\t<span class=\"article-header__reading-time\">\n\t\t\t7 min read\t\t<\/span>\n\t<\/div>\n\n\t\t\t<\/div>\n\t\t\n\t\t\t<h2 class=\"article-header__subheading wp-block-heading\">\n\t\t\tStudy shows that such outlets can have broad impact on the national conversation\t\t<\/h2>\n\t\t\n<\/header>\n"},"2":{"blockName":"core\/group","attrs":{"templateLock":false,"metadata":{"name":"Article content"},"align":"wide","layout":{"type":"constrained","justifyContent":"center"},"tagName":"div","lock":[],"className":"","style":[],"backgroundColor":"","textColor":"","gradient":"","fontSize":"","fontFamily":"","borderColor":"","ariaLabel":"","anchor":""},"innerBlocks":[{"blockName":"core\/freeform","attrs":{"content":"","lock":[],"metadata":[]},"innerBlocks":[],"innerHTML":"\n\t\t<p>With a readership in the millions, The New York Times routinely influences public debate on a host of issues through its news coverage.<\/p>\n<p>Can a small news outlet of perhaps 50,000 circulation do the same thing?<\/p>\n<p>The answer, says Albert J. Weatherhead III University Professor Gary King, is that in an age that relies on internet publication and social-media dispersal, even small- to medium-size media outlets can have a dramatic impact on the content and partisan balance of the national conversation about major public-policy issues.<\/p>\n<p>In the first large-scale, randomized media <a href=\"http:\/\/science.sciencemag.org\/content\/358\/6364\/776\">experiment<\/a> of its kind, King and former students Benjamin Schneer, now an assistant professor at Florida State University, and Ariel White, now an assistant professor at Massachusetts Institute of Technology, found that if just three outlets wrote about a major national policy topic \u2014 such as jobs, the environment, or immigration \u2014 discussion of that topic across social media rose by more than 62 percent, and the balance of opinion in the national conversation could be swayed by several percentage points.<\/p>\n<p>\u201cFor several hundred years, scholars have tried to measure the influence of the media. Most people think it is influential, but measuring this influence rigorously with randomized experiments has until now been impossible,\u201d King said. \u201cOur findings suggest that the effect of the media is surprisingly large. Our study\u2019s implications suggest every journalist wields a major power, and so has an important responsibility.\u201d<\/p>\n<p>Those findings, King said, are the result of more than five years of work, much of it spent convincing 48 news outlets to agree to take part in the study. About half of these outlets were represented by the Media Consortium, a network of independent news outlets that was eager to find a way to measure impact and was willing to help.<\/p>\n<p>\u201cMuch of the work leading up to this study involved finding a way to bridge the cultural divide between journalism and science,\u201d King said. \u201cThrough years of conversations, much trial and error, and a partnership with Media Consortium Executive Director Jo Ellen Green Kaiser, we learned to understand journalistic standards and practices, and the journalists learned to understand our scientific requirements. What ultimately made it all work was a novel research design we developed that satisfied both camps.\u201d<\/p>\n","innerContent":["\n\t\t<p>With a readership in the millions, The New York Times routinely influences public debate on a host of issues through its news coverage.<\/p>\n<p>Can a small news outlet of perhaps 50,000 circulation do the same thing?<\/p>\n<p>The answer, says Albert J. Weatherhead III University Professor Gary King, is that in an age that relies on internet publication and social-media dispersal, even small- to medium-size media outlets can have a dramatic impact on the content and partisan balance of the national conversation about major public-policy issues.<\/p>\n<p>In the first large-scale, randomized media <a href=\"http:\/\/science.sciencemag.org\/content\/358\/6364\/776\">experiment<\/a> of its kind, King and former students Benjamin Schneer, now an assistant professor at Florida State University, and Ariel White, now an assistant professor at Massachusetts Institute of Technology, found that if just three outlets wrote about a major national policy topic \u2014 such as jobs, the environment, or immigration \u2014 discussion of that topic across social media rose by more than 62 percent, and the balance of opinion in the national conversation could be swayed by several percentage points.<\/p>\n<p>\u201cFor several hundred years, scholars have tried to measure the influence of the media. Most people think it is influential, but measuring this influence rigorously with randomized experiments has until now been impossible,\u201d King said. \u201cOur findings suggest that the effect of the media is surprisingly large. Our study\u2019s implications suggest every journalist wields a major power, and so has an important responsibility.\u201d<\/p>\n<p>Those findings, King said, are the result of more than five years of work, much of it spent convincing 48 news outlets to agree to take part in the study. About half of these outlets were represented by the Media Consortium, a network of independent news outlets that was eager to find a way to measure impact and was willing to help.<\/p>\n<p>\u201cMuch of the work leading up to this study involved finding a way to bridge the cultural divide between journalism and science,\u201d King said. \u201cThrough years of conversations, much trial and error, and a partnership with Media Consortium Executive Director Jo Ellen Green Kaiser, we learned to understand journalistic standards and practices, and the journalists learned to understand our scientific requirements. What ultimately made it all work was a novel research design we developed that satisfied both camps.\u201d<\/p>\n"],"rendered":"\n\t\t<p>With a readership in the millions, The New York Times routinely influences public debate on a host of issues through its news coverage.<\/p>\n<p>Can a small news outlet of perhaps 50,000 circulation do the same thing?<\/p>\n<p>The answer, says Albert J. Weatherhead III University Professor Gary King, is that in an age that relies on internet publication and social-media dispersal, even small- to medium-size media outlets can have a dramatic impact on the content and partisan balance of the national conversation about major public-policy issues.<\/p>\n<p>In the first large-scale, randomized media <a href=\"http:\/\/science.sciencemag.org\/content\/358\/6364\/776\">experiment<\/a> of its kind, King and former students Benjamin Schneer, now an assistant professor at Florida State University, and Ariel White, now an assistant professor at Massachusetts Institute of Technology, found that if just three outlets wrote about a major national policy topic \u2014 such as jobs, the environment, or immigration \u2014 discussion of that topic across social media rose by more than 62 percent, and the balance of opinion in the national conversation could be swayed by several percentage points.<\/p>\n<p>\u201cFor several hundred years, scholars have tried to measure the influence of the media. Most people think it is influential, but measuring this influence rigorously with randomized experiments has until now been impossible,\u201d King said. \u201cOur findings suggest that the effect of the media is surprisingly large. Our study\u2019s implications suggest every journalist wields a major power, and so has an important responsibility.\u201d<\/p>\n<p>Those findings, King said, are the result of more than five years of work, much of it spent convincing 48 news outlets to agree to take part in the study. About half of these outlets were represented by the Media Consortium, a network of independent news outlets that was eager to find a way to measure impact and was willing to help.<\/p>\n<p>\u201cMuch of the work leading up to this study involved finding a way to bridge the cultural divide between journalism and science,\u201d King said. \u201cThrough years of conversations, much trial and error, and a partnership with Media Consortium Executive Director Jo Ellen Green Kaiser, we learned to understand journalistic standards and practices, and the journalists learned to understand our scientific requirements. What ultimately made it all work was a novel research design we developed that satisfied both camps.\u201d<\/p>\n"},{"blockName":"core\/image","attrs":{"sizeSlug":"full","align":"none","id":233078,"caption":"Credit: Carla Schaffer\/AAAS","blob":"","url":"https:\/\/news.harvard.edu\/gazette\/wp-content\/uploads\/2017\/11\/mediagrap.jpg","alt":"","lightbox":[],"title":"","href":"","rel":"","linkClass":"","width":"","height":"","aspectRatio":"","scale":"","linkDestination":"","linkTarget":"","lock":[],"metadata":[],"className":"","style":[],"borderColor":"","anchor":""},"innerBlocks":[],"innerHTML":"\n\n\t<figure class=\"wp-block-image alignnone size-full is-resized\"><img src=\"https:\/\/news.harvard.edu\/gazette\/wp-content\/uploads\/2017\/11\/mediagrap.jpg\" alt=\"\" class=\"wp-image-233078\"><figcaption class=\"wp-element-caption\">Credit: Carla Schaffer\/AAAS\t\t\t<\/figcaption><\/figure>\n\t","innerContent":["\n\n\t<figure class=\"wp-block-image alignnone size-full is-resized\"><img src=\"https:\/\/news.harvard.edu\/gazette\/wp-content\/uploads\/2017\/11\/mediagrap.jpg\" alt=\"\" class=\"wp-image-233078\"><figcaption class=\"wp-element-caption\">Credit: Carla Schaffer\/AAAS\t\t\t<\/figcaption><\/figure>\n\t"],"rendered":"\n\n\t<figure class=\"wp-block-image alignnone size-full is-resized\"><img src=\"https:\/\/news.harvard.edu\/gazette\/wp-content\/uploads\/2017\/11\/mediagrap.jpg\" alt=\"\" class=\"wp-image-233078\"><figcaption class=\"wp-element-caption\">Credit: Carla Schaffer\/AAAS\t\t\t<\/figcaption><\/figure>\n\t"},{"blockName":"core\/freeform","attrs":{"content":"","lock":[],"metadata":[]},"innerBlocks":[],"innerHTML":"\n<p>Though similar efforts have been tried in the past, they invariably collapsed as journalists chafed at the idea of being told what to report and when to report it. To address the problem, previous researchers fell back on clever tricks, such as studying areas that fell outside the broadcast area of a particular outlet but, because no one knew whether the areas were truly random, were hard to evaluate. Such studies faced many problems, particularly their inability to control for a host of factors such as race, education, or income.<\/p>\n<p>\u201cFrom a scientific point of view, we have to be able to tell the journalists what to publish, and preferably at random times,\u201d King said. \u201cFrom a journalistic point of view, these scientific requirements seem crazy, and journalists reasonably insist on retaining absolute control over what they publish. The two sets of requirements seem fundamentally incompatible, but we found a way to create a single research design that accomplished the goals of both groups.\u201d<\/p>\n<p>It wasn\u2019t only the participation of news outlets that made the study noteworthy, though.<\/p>\n<p>\u201cIf you\u2019re doing something like a medical experiment, you may randomly assign individual people to one of two groups, and then each person is your unit of analysis,\u201d he said. \u201cBut when a media outlet publishes something \u2014 no matter how small \u2014 the potential audience it could impact includes basically everybody in the country. That means our unit of analysis can\u2019t be a person. It has to be the entire country, which greatly increases the cost of the study.\u201d That means that the equivalent of an entire experiment in many other studies constitutes only one observation in this study.<\/p>\n<p>Because collecting each observation was so expensive and logistically challenging, King and colleagues used, and further developed, novel statistical techniques to enable them to collect only as much data as needed. After each massive national experiment they could then examine whether they had amassed enough data to draw reliable conclusions.<\/p>\n<p>\u201cThat allowed us to keep going until we got to the point where we had exactly as much data as we needed, and no more,\u201d King said. \u201cAs it turns out, we ran 35 national experiments that produced 70 observations.\u201d<\/p>\n<p>To achieve the randomization needed for the study, King\u2019s team, the Media Consortium staff, and journalists at the 48 outlets identified 11 broad policy areas. They then simulated the tendency of journalists to influence each other and publish stories on similar topics, sometimes called \u201cpack journalism,\u201d by choosing three or four outlets from their participating group of 48 to develop stories together that fell into the same broad policy area.<\/p>\n<p>\u201cFor example, if the policy area was jobs, one story might be about Uber drivers in the Philadelphia area,\u201d King said. \u201cWe would then identify a two-week period where we predicted there wouldn\u2019t be any surprises related to that topic area. So if the president was planning to give a speech about immigration in one of those two weeks, we would not run an experiment on immigration during that time.\u201d<\/p>\n<p>Randomization came from researchers flipping a coin to determine which of those weeks would be the publication week, and which the control week.<\/p>\n<p>\u201cAt first, our outlets didn\u2019t really understand what randomization meant,\u201d said Kaiser. \u201cOur project manager, Manolia Charlotin, and the researchers worked very closely with all the outlets to ensure they followed the researchers\u2019 rules. This was a resource-intensive project for us, but the unexpected benefit was that outlets found they also gained many qualitative benefits from collaborating.\u201d<\/p>\n<p>In both treatment and control weeks, King, Schneer, and White used tools and data from the global analytics company Crimson Hexagon to monitor the national conversation in social media posts. (King is a co-founder of Crimson Hexagon; with a previous generation of graduate and postdoctoral researchers, he developed the automated text analysis technology that Harvard licensed to the company.) He explained that this methodology \u201cis used to evaluate meaning in social media posts. So if you have a set of categories you care about, we identify example posts in these categories, which is what humans are good at,\u201d King said. \u201cThen our algorithm can amplify that human intelligence and, without classifying individual posts, can accurately estimate the percent of posts in each category each day.\u201d<\/p>\n<p>What they found, King said, was that the effect was larger than anticipated.<\/p>\n<p>\u201cThe actual effect is really big,\u201d King said. \u201cIf three outlets (with an average circulation of about 50,000) get together and write stories, the size of the national conversation in that policy area increases a lot. It\u2019s a 62 percent increase on the first day\u2019s volume distributed over the week, just from these three little outlets.<\/p>\n<p>\u201cThese national conversations about major policy areas are essential to democracy,\u201d he added. \u201cToday this conversation takes place, in part, in some of the 750,000,000 publicly available social media posts written by people every day \u2014 and all available for research. At one time, the national conversation was whatever was said in the public square, where people would get up on a soapbox, or when they expressed themselves in newspaper editorials or water-cooler debates. This is a lot of what democracy is about. The fact that the media has such a large influence on the content of this national conversation is crucial for everything from the ideological balance of the nation\u2019s media outlets, to the rise of fake news, to the ongoing responsibility of professional journalists.\u201d<\/p>\n<p><em>This research was supported with funding from Voqal and Harvard\u2019s Institute for Quantitative Social Science.<\/em><\/p>\n","innerContent":["\n<p>Though similar efforts have been tried in the past, they invariably collapsed as journalists chafed at the idea of being told what to report and when to report it. To address the problem, previous researchers fell back on clever tricks, such as studying areas that fell outside the broadcast area of a particular outlet but, because no one knew whether the areas were truly random, were hard to evaluate. Such studies faced many problems, particularly their inability to control for a host of factors such as race, education, or income.<\/p>\n<p>\u201cFrom a scientific point of view, we have to be able to tell the journalists what to publish, and preferably at random times,\u201d King said. \u201cFrom a journalistic point of view, these scientific requirements seem crazy, and journalists reasonably insist on retaining absolute control over what they publish. The two sets of requirements seem fundamentally incompatible, but we found a way to create a single research design that accomplished the goals of both groups.\u201d<\/p>\n<p>It wasn\u2019t only the participation of news outlets that made the study noteworthy, though.<\/p>\n<p>\u201cIf you\u2019re doing something like a medical experiment, you may randomly assign individual people to one of two groups, and then each person is your unit of analysis,\u201d he said. \u201cBut when a media outlet publishes something \u2014 no matter how small \u2014 the potential audience it could impact includes basically everybody in the country. That means our unit of analysis can\u2019t be a person. It has to be the entire country, which greatly increases the cost of the study.\u201d That means that the equivalent of an entire experiment in many other studies constitutes only one observation in this study.<\/p>\n<p>Because collecting each observation was so expensive and logistically challenging, King and colleagues used, and further developed, novel statistical techniques to enable them to collect only as much data as needed. After each massive national experiment they could then examine whether they had amassed enough data to draw reliable conclusions.<\/p>\n<p>\u201cThat allowed us to keep going until we got to the point where we had exactly as much data as we needed, and no more,\u201d King said. \u201cAs it turns out, we ran 35 national experiments that produced 70 observations.\u201d<\/p>\n<p>To achieve the randomization needed for the study, King\u2019s team, the Media Consortium staff, and journalists at the 48 outlets identified 11 broad policy areas. They then simulated the tendency of journalists to influence each other and publish stories on similar topics, sometimes called \u201cpack journalism,\u201d by choosing three or four outlets from their participating group of 48 to develop stories together that fell into the same broad policy area.<\/p>\n<p>\u201cFor example, if the policy area was jobs, one story might be about Uber drivers in the Philadelphia area,\u201d King said. \u201cWe would then identify a two-week period where we predicted there wouldn\u2019t be any surprises related to that topic area. So if the president was planning to give a speech about immigration in one of those two weeks, we would not run an experiment on immigration during that time.\u201d<\/p>\n<p>Randomization came from researchers flipping a coin to determine which of those weeks would be the publication week, and which the control week.<\/p>\n<p>\u201cAt first, our outlets didn\u2019t really understand what randomization meant,\u201d said Kaiser. \u201cOur project manager, Manolia Charlotin, and the researchers worked very closely with all the outlets to ensure they followed the researchers\u2019 rules. This was a resource-intensive project for us, but the unexpected benefit was that outlets found they also gained many qualitative benefits from collaborating.\u201d<\/p>\n<p>In both treatment and control weeks, King, Schneer, and White used tools and data from the global analytics company Crimson Hexagon to monitor the national conversation in social media posts. (King is a co-founder of Crimson Hexagon; with a previous generation of graduate and postdoctoral researchers, he developed the automated text analysis technology that Harvard licensed to the company.) He explained that this methodology \u201cis used to evaluate meaning in social media posts. So if you have a set of categories you care about, we identify example posts in these categories, which is what humans are good at,\u201d King said. \u201cThen our algorithm can amplify that human intelligence and, without classifying individual posts, can accurately estimate the percent of posts in each category each day.\u201d<\/p>\n<p>What they found, King said, was that the effect was larger than anticipated.<\/p>\n<p>\u201cThe actual effect is really big,\u201d King said. \u201cIf three outlets (with an average circulation of about 50,000) get together and write stories, the size of the national conversation in that policy area increases a lot. It\u2019s a 62 percent increase on the first day\u2019s volume distributed over the week, just from these three little outlets.<\/p>\n<p>\u201cThese national conversations about major policy areas are essential to democracy,\u201d he added. \u201cToday this conversation takes place, in part, in some of the 750,000,000 publicly available social media posts written by people every day \u2014 and all available for research. At one time, the national conversation was whatever was said in the public square, where people would get up on a soapbox, or when they expressed themselves in newspaper editorials or water-cooler debates. This is a lot of what democracy is about. The fact that the media has such a large influence on the content of this national conversation is crucial for everything from the ideological balance of the nation\u2019s media outlets, to the rise of fake news, to the ongoing responsibility of professional journalists.\u201d<\/p>\n<p><em>This research was supported with funding from Voqal and Harvard\u2019s Institute for Quantitative Social Science.<\/em><\/p>\n"],"rendered":"\n<p>Though similar efforts have been tried in the past, they invariably collapsed as journalists chafed at the idea of being told what to report and when to report it. To address the problem, previous researchers fell back on clever tricks, such as studying areas that fell outside the broadcast area of a particular outlet but, because no one knew whether the areas were truly random, were hard to evaluate. Such studies faced many problems, particularly their inability to control for a host of factors such as race, education, or income.<\/p>\n<p>\u201cFrom a scientific point of view, we have to be able to tell the journalists what to publish, and preferably at random times,\u201d King said. \u201cFrom a journalistic point of view, these scientific requirements seem crazy, and journalists reasonably insist on retaining absolute control over what they publish. The two sets of requirements seem fundamentally incompatible, but we found a way to create a single research design that accomplished the goals of both groups.\u201d<\/p>\n<p>It wasn\u2019t only the participation of news outlets that made the study noteworthy, though.<\/p>\n<p>\u201cIf you\u2019re doing something like a medical experiment, you may randomly assign individual people to one of two groups, and then each person is your unit of analysis,\u201d he said. \u201cBut when a media outlet publishes something \u2014 no matter how small \u2014 the potential audience it could impact includes basically everybody in the country. That means our unit of analysis can\u2019t be a person. It has to be the entire country, which greatly increases the cost of the study.\u201d That means that the equivalent of an entire experiment in many other studies constitutes only one observation in this study.<\/p>\n<p>Because collecting each observation was so expensive and logistically challenging, King and colleagues used, and further developed, novel statistical techniques to enable them to collect only as much data as needed. After each massive national experiment they could then examine whether they had amassed enough data to draw reliable conclusions.<\/p>\n<p>\u201cThat allowed us to keep going until we got to the point where we had exactly as much data as we needed, and no more,\u201d King said. \u201cAs it turns out, we ran 35 national experiments that produced 70 observations.\u201d<\/p>\n<p>To achieve the randomization needed for the study, King\u2019s team, the Media Consortium staff, and journalists at the 48 outlets identified 11 broad policy areas. They then simulated the tendency of journalists to influence each other and publish stories on similar topics, sometimes called \u201cpack journalism,\u201d by choosing three or four outlets from their participating group of 48 to develop stories together that fell into the same broad policy area.<\/p>\n<p>\u201cFor example, if the policy area was jobs, one story might be about Uber drivers in the Philadelphia area,\u201d King said. \u201cWe would then identify a two-week period where we predicted there wouldn\u2019t be any surprises related to that topic area. So if the president was planning to give a speech about immigration in one of those two weeks, we would not run an experiment on immigration during that time.\u201d<\/p>\n<p>Randomization came from researchers flipping a coin to determine which of those weeks would be the publication week, and which the control week.<\/p>\n<p>\u201cAt first, our outlets didn\u2019t really understand what randomization meant,\u201d said Kaiser. \u201cOur project manager, Manolia Charlotin, and the researchers worked very closely with all the outlets to ensure they followed the researchers\u2019 rules. This was a resource-intensive project for us, but the unexpected benefit was that outlets found they also gained many qualitative benefits from collaborating.\u201d<\/p>\n<p>In both treatment and control weeks, King, Schneer, and White used tools and data from the global analytics company Crimson Hexagon to monitor the national conversation in social media posts. (King is a co-founder of Crimson Hexagon; with a previous generation of graduate and postdoctoral researchers, he developed the automated text analysis technology that Harvard licensed to the company.) He explained that this methodology \u201cis used to evaluate meaning in social media posts. So if you have a set of categories you care about, we identify example posts in these categories, which is what humans are good at,\u201d King said. \u201cThen our algorithm can amplify that human intelligence and, without classifying individual posts, can accurately estimate the percent of posts in each category each day.\u201d<\/p>\n<p>What they found, King said, was that the effect was larger than anticipated.<\/p>\n<p>\u201cThe actual effect is really big,\u201d King said. \u201cIf three outlets (with an average circulation of about 50,000) get together and write stories, the size of the national conversation in that policy area increases a lot. It\u2019s a 62 percent increase on the first day\u2019s volume distributed over the week, just from these three little outlets.<\/p>\n<p>\u201cThese national conversations about major policy areas are essential to democracy,\u201d he added. \u201cToday this conversation takes place, in part, in some of the 750,000,000 publicly available social media posts written by people every day \u2014 and all available for research. At one time, the national conversation was whatever was said in the public square, where people would get up on a soapbox, or when they expressed themselves in newspaper editorials or water-cooler debates. This is a lot of what democracy is about. The fact that the media has such a large influence on the content of this national conversation is crucial for everything from the ideological balance of the nation\u2019s media outlets, to the rise of fake news, to the ongoing responsibility of professional journalists.\u201d<\/p>\n<p><em>This research was supported with funding from Voqal and Harvard\u2019s Institute for Quantitative Social Science.<\/em><\/p>\n"}],"innerHTML":"\n<div class=\"wp-block-group alignwide\">\n\n\r\n\t\n\t\r\n\n\n<\/div>\n","innerContent":["\n<div class=\"wp-block-group alignwide\">\n\n","\r\n\t","\n\t\r\n","\n\n<\/div>\n"],"rendered":"\n<div class=\"wp-block-group alignwide has-global-padding is-content-justification-center is-layout-constrained wp-block-group-is-layout-constrained\">\n\n\n\t\t<p>With a readership in the millions, The New York Times routinely influences public debate on a host of issues through its news coverage.<\/p>\n<p>Can a small news outlet of perhaps 50,000 circulation do the same thing?<\/p>\n<p>The answer, says Albert J. Weatherhead III University Professor Gary King, is that in an age that relies on internet publication and social-media dispersal, even small- to medium-size media outlets can have a dramatic impact on the content and partisan balance of the national conversation about major public-policy issues.<\/p>\n<p>In the first large-scale, randomized media <a href=\"http:\/\/science.sciencemag.org\/content\/358\/6364\/776\">experiment<\/a> of its kind, King and former students Benjamin Schneer, now an assistant professor at Florida State University, and Ariel White, now an assistant professor at Massachusetts Institute of Technology, found that if just three outlets wrote about a major national policy topic \u2014 such as jobs, the environment, or immigration \u2014 discussion of that topic across social media rose by more than 62 percent, and the balance of opinion in the national conversation could be swayed by several percentage points.<\/p>\n<p>\u201cFor several hundred years, scholars have tried to measure the influence of the media. Most people think it is influential, but measuring this influence rigorously with randomized experiments has until now been impossible,\u201d King said. \u201cOur findings suggest that the effect of the media is surprisingly large. Our study\u2019s implications suggest every journalist wields a major power, and so has an important responsibility.\u201d<\/p>\n<p>Those findings, King said, are the result of more than five years of work, much of it spent convincing 48 news outlets to agree to take part in the study. About half of these outlets were represented by the Media Consortium, a network of independent news outlets that was eager to find a way to measure impact and was willing to help.<\/p>\n<p>\u201cMuch of the work leading up to this study involved finding a way to bridge the cultural divide between journalism and science,\u201d King said. \u201cThrough years of conversations, much trial and error, and a partnership with Media Consortium Executive Director Jo Ellen Green Kaiser, we learned to understand journalistic standards and practices, and the journalists learned to understand our scientific requirements. What ultimately made it all work was a novel research design we developed that satisfied both camps.\u201d<\/p>\n\r\n\t\n\n\t<figure class=\"wp-block-image alignnone size-full is-resized\"><img src=\"https:\/\/news.harvard.edu\/gazette\/wp-content\/uploads\/2017\/11\/mediagrap.jpg\" alt=\"\" class=\"wp-image-233078\"><figcaption class=\"wp-element-caption\">Credit: Carla Schaffer\/AAAS\t\t\t<\/figcaption><\/figure>\n\t\n\t\r\n\n<p>Though similar efforts have been tried in the past, they invariably collapsed as journalists chafed at the idea of being told what to report and when to report it. To address the problem, previous researchers fell back on clever tricks, such as studying areas that fell outside the broadcast area of a particular outlet but, because no one knew whether the areas were truly random, were hard to evaluate. Such studies faced many problems, particularly their inability to control for a host of factors such as race, education, or income.<\/p>\n<p>\u201cFrom a scientific point of view, we have to be able to tell the journalists what to publish, and preferably at random times,\u201d King said. \u201cFrom a journalistic point of view, these scientific requirements seem crazy, and journalists reasonably insist on retaining absolute control over what they publish. The two sets of requirements seem fundamentally incompatible, but we found a way to create a single research design that accomplished the goals of both groups.\u201d<\/p>\n<p>It wasn\u2019t only the participation of news outlets that made the study noteworthy, though.<\/p>\n<p>\u201cIf you\u2019re doing something like a medical experiment, you may randomly assign individual people to one of two groups, and then each person is your unit of analysis,\u201d he said. \u201cBut when a media outlet publishes something \u2014 no matter how small \u2014 the potential audience it could impact includes basically everybody in the country. That means our unit of analysis can\u2019t be a person. It has to be the entire country, which greatly increases the cost of the study.\u201d That means that the equivalent of an entire experiment in many other studies constitutes only one observation in this study.<\/p>\n<p>Because collecting each observation was so expensive and logistically challenging, King and colleagues used, and further developed, novel statistical techniques to enable them to collect only as much data as needed. After each massive national experiment they could then examine whether they had amassed enough data to draw reliable conclusions.<\/p>\n<p>\u201cThat allowed us to keep going until we got to the point where we had exactly as much data as we needed, and no more,\u201d King said. \u201cAs it turns out, we ran 35 national experiments that produced 70 observations.\u201d<\/p>\n<p>To achieve the randomization needed for the study, King\u2019s team, the Media Consortium staff, and journalists at the 48 outlets identified 11 broad policy areas. They then simulated the tendency of journalists to influence each other and publish stories on similar topics, sometimes called \u201cpack journalism,\u201d by choosing three or four outlets from their participating group of 48 to develop stories together that fell into the same broad policy area.<\/p>\n<p>\u201cFor example, if the policy area was jobs, one story might be about Uber drivers in the Philadelphia area,\u201d King said. \u201cWe would then identify a two-week period where we predicted there wouldn\u2019t be any surprises related to that topic area. So if the president was planning to give a speech about immigration in one of those two weeks, we would not run an experiment on immigration during that time.\u201d<\/p>\n<p>Randomization came from researchers flipping a coin to determine which of those weeks would be the publication week, and which the control week.<\/p>\n<p>\u201cAt first, our outlets didn\u2019t really understand what randomization meant,\u201d said Kaiser. \u201cOur project manager, Manolia Charlotin, and the researchers worked very closely with all the outlets to ensure they followed the researchers\u2019 rules. This was a resource-intensive project for us, but the unexpected benefit was that outlets found they also gained many qualitative benefits from collaborating.\u201d<\/p>\n<p>In both treatment and control weeks, King, Schneer, and White used tools and data from the global analytics company Crimson Hexagon to monitor the national conversation in social media posts. (King is a co-founder of Crimson Hexagon; with a previous generation of graduate and postdoctoral researchers, he developed the automated text analysis technology that Harvard licensed to the company.) He explained that this methodology \u201cis used to evaluate meaning in social media posts. So if you have a set of categories you care about, we identify example posts in these categories, which is what humans are good at,\u201d King said. \u201cThen our algorithm can amplify that human intelligence and, without classifying individual posts, can accurately estimate the percent of posts in each category each day.\u201d<\/p>\n<p>What they found, King said, was that the effect was larger than anticipated.<\/p>\n<p>\u201cThe actual effect is really big,\u201d King said. \u201cIf three outlets (with an average circulation of about 50,000) get together and write stories, the size of the national conversation in that policy area increases a lot. It\u2019s a 62 percent increase on the first day\u2019s volume distributed over the week, just from these three little outlets.<\/p>\n<p>\u201cThese national conversations about major policy areas are essential to democracy,\u201d he added. \u201cToday this conversation takes place, in part, in some of the 750,000,000 publicly available social media posts written by people every day \u2014 and all available for research. At one time, the national conversation was whatever was said in the public square, where people would get up on a soapbox, or when they expressed themselves in newspaper editorials or water-cooler debates. This is a lot of what democracy is about. The fact that the media has such a large influence on the content of this national conversation is crucial for everything from the ideological balance of the nation\u2019s media outlets, to the rise of fake news, to the ongoing responsibility of professional journalists.\u201d<\/p>\n<p><em>This research was supported with funding from Voqal and Harvard\u2019s Institute for Quantitative Social Science.<\/em><\/p>\n\n\n<\/div>\n"}},"jetpack-related-posts":[{"id":169893,"url":"https:\/\/news.harvard.edu\/gazette\/story\/2015\/05\/uncertain-forecast-for-social-security-2\/","url_meta":{"origin":233076,"position":0},"title":"Uncertain forecast for Social Security","author":"harvardgazette","date":"May 8, 2015","format":false,"excerpt":"A new study has found that the financial health of Social Security, the program millions of Americans have relied on for decades as a crucial part of their income, has been dramatically overstated.","rel":"","context":"In "Nation & World"","block_context":{"text":"Nation & World","link":"https:\/\/news.harvard.edu\/gazette\/section\/nation-world\/"},"img":{"alt_text":"","src":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2015\/05\/050515_king_014_605.jpg?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2015\/05\/050515_king_014_605.jpg?resize=350%2C200 1x, https:\/\/news.harvard.edu\/wp-content\/uploads\/2015\/05\/050515_king_014_605.jpg?resize=525%2C300 1.5x"},"classes":[]},{"id":184714,"url":"https:\/\/news.harvard.edu\/gazette\/story\/2016\/06\/behind-chinas-viral-curtain\/","url_meta":{"origin":233076,"position":1},"title":"Behind China\u2019s viral curtain","author":"harvardgazette","date":"June 9, 2016","format":false,"excerpt":"A study by Professor Gary King and two former graduate students points to an effort by the Chinese government to use social media to discourage anti-government action.","rel":"","context":"In "Science & Tech"","block_context":{"text":"Science & Tech","link":"https:\/\/news.harvard.edu\/gazette\/section\/science-technology\/"},"img":{"alt_text":"","src":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2016\/06\/080515_king_gary_116_341459_605.jpg?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2016\/06\/080515_king_gary_116_341459_605.jpg?resize=350%2C200 1x, https:\/\/news.harvard.edu\/wp-content\/uploads\/2016\/06\/080515_king_gary_116_341459_605.jpg?resize=525%2C300 1.5x"},"classes":[]},{"id":124470,"url":"https:\/\/news.harvard.edu\/gazette\/story\/2012\/12\/corporation-member-steps-down\/","url_meta":{"origin":233076,"position":2},"title":"Corporation member steps down","author":"harvardgazette","date":"December 3, 2012","format":false,"excerpt":"Patricia A. King, the Carmack Waterhouse Professor of Law, Medicine, Ethics, and Public Policy at Georgetown Law Center, plans to step down from the Harvard Corporation at the end of December, the University announced today.","rel":"","context":"In "Campus & Community"","block_context":{"text":"Campus & Community","link":"https:\/\/news.harvard.edu\/gazette\/section\/campus-community\/"},"img":{"alt_text":"","src":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2012\/11\/king-pat_605.jpg?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2012\/11\/king-pat_605.jpg?resize=350%2C200 1x, https:\/\/news.harvard.edu\/wp-content\/uploads\/2012\/11\/king-pat_605.jpg?resize=525%2C300 1.5x"},"classes":[]},{"id":148142,"url":"https:\/\/news.harvard.edu\/gazette\/story\/2013\/10\/the-teaching-launch\/","url_meta":{"origin":233076,"position":3},"title":"The teaching launch","author":"harvardgazette","date":"October 15, 2013","format":false,"excerpt":"A new study found that middle school teachers can have a real impact not only on students\u2019 short-term educations, but on whether they attend college and on the size of their future paychecks.","rel":"","context":"In "Science & Tech"","block_context":{"text":"Science & Tech","link":"https:\/\/news.harvard.edu\/gazette\/section\/science-technology\/"},"img":{"alt_text":"","src":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2013\/10\/102703_chamberlain_gary_3_605.jpg?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2013\/10\/102703_chamberlain_gary_3_605.jpg?resize=350%2C200 1x, https:\/\/news.harvard.edu\/wp-content\/uploads\/2013\/10\/102703_chamberlain_gary_3_605.jpg?resize=525%2C300 1.5x"},"classes":[]},{"id":61401,"url":"https:\/\/news.harvard.edu\/gazette\/story\/2009\/04\/mexican-program-successful-at-reducing-crippling-health-care-costs\/","url_meta":{"origin":233076,"position":4},"title":"Mexican program successful at reducing crippling health care costs","author":"harvardgazette","date":"April 7, 2009","format":false,"excerpt":"Seguro Popular, a Mexican health care program instituted in 2003, has already reduced crippling health care costs among poorer households, according to an evaluation conducted by researchers at Harvard University in collaboration with researchers in Mexico.","rel":"","context":"In "Health"","block_context":{"text":"Health","link":"https:\/\/news.harvard.edu\/gazette\/section\/health\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":166268,"url":"https:\/\/news.harvard.edu\/gazette\/story\/2015\/02\/climate-engineering-in-from-the-cold\/","url_meta":{"origin":233076,"position":5},"title":"Climate engineering: In from the cold","author":"harvardgazette","date":"February 20, 2015","format":false,"excerpt":"Harvard Professor David Keith says that two new reports by the National Academy of Sciences are likely to boost a deeper look at possible geoengineering options for climate engineering.","rel":"","context":"In "Science & Tech"","block_context":{"text":"Science & Tech","link":"https:\/\/news.harvard.edu\/gazette\/section\/science-technology\/"},"img":{"alt_text":"","src":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2015\/02\/nas_clouds_courtesy-flickr-user-janne-morem.jpg?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/news.harvard.edu\/wp-content\/uploads\/2015\/02\/nas_clouds_courtesy-flickr-user-janne-morem.jpg?resize=350%2C200 1x, https:\/\/news.harvard.edu\/wp-content\/uploads\/2015\/02\/nas_clouds_courtesy-flickr-user-janne-morem.jpg?resize=525%2C300 1.5x"},"classes":[]}],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/news.harvard.edu\/gazette\/wp-json\/wp\/v2\/posts\/233076","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/news.harvard.edu\/gazette\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/news.harvard.edu\/gazette\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/news.harvard.edu\/gazette\/wp-json\/wp\/v2\/users\/108352576"}],"replies":[{"embeddable":true,"href":"https:\/\/news.harvard.edu\/gazette\/wp-json\/wp\/v2\/comments?post=233076"}],"version-history":[{"count":6,"href":"https:\/\/news.harvard.edu\/gazette\/wp-json\/wp\/v2\/posts\/233076\/revisions"}],"predecessor-version":[{"id":233135,"href":"https:\/\/news.harvard.edu\/gazette\/wp-json\/wp\/v2\/posts\/233076\/revisions\/233135"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/news.harvard.edu\/gazette\/wp-json\/wp\/v2\/media\/233086"}],"wp:attachment":[{"href":"https:\/\/news.harvard.edu\/gazette\/wp-json\/wp\/v2\/media?parent=233076"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/news.harvard.edu\/gazette\/wp-json\/wp\/v2\/categories?post=233076"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/news.harvard.edu\/gazette\/wp-json\/wp\/v2\/tags?post=233076"},{"taxonomy":"format","embeddable":true,"href":"https:\/\/news.harvard.edu\/gazette\/wp-json\/wp\/v2\/gazette-formats?post=233076"},{"taxonomy":"series","embeddable":true,"href":"https:\/\/news.harvard.edu\/gazette\/wp-json\/wp\/v2\/series?post=233076"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}