AI-aided study of local news finds media criticism of police has not risen or become more partisan over last decade

In the last decade, high-profile incidents of police violence against Black citizens in the United States—most notably the 2020 murder of George Floyd—have sparked protests and public debate on misconduct and racial bias in policing. These events received substantial news coverage, becoming highly politicized and leading to a perception that media have become more critical and partisan in their treatment of police.

While there is some support for this perception in national news, we know less about local outlets that cover police departments regularly. In a new study, researchers analyzed a decade of local news coverage, finding that local media criticism of the police has not increased significantly or become more politically polarized over the past 10 years.

The study found that while local media criticism spiked after high-profile police killings, these events did not produce sustained rises in negative police news. Furthermore, there were only small differences in coverage between news outlets in more conservative and in more liberal cities, undermining the idea that local news outlets cater to the politics of their audiences.

The study, by researchers at Carnegie Mellon University and the Rochester Institute of Technology, used AI language models and human readers across the political spectrum to measure criticism of the police in 1.3 million news excerpts referencing the police from 10 politically diverse U.S. cities from 2013 to 2023. The study is published in the Proceedings of the National Academy of Sciences.

“Generally speaking, our study found that local media criticism of the police has remained remarkably steady, which contrasts with common perceptions and trends at the national level,” says Logan Crowl, a Ph.D. student in public policy and machine learning at Carnegie Mellon’s Heinz College, who coauthored the study.

“For many Americans, local news shapes how they see their local police department; when we look at 10 years of that reporting across the country, we see that it is essentially no more critical of the police or politically polarized than it was a decade ago.”

In the study, researchers first surveyed 500 representative U.S. residents to gauge their views about reporting on police. They found that 54% thought local news reporting had become more critical of the police, while only 19% thought local reporting had become less critical.

To test the reality of this perception, researchers constructed a data set of news articles published from 2013 to 2023 by local outlets in 10 U.S. cities; the cities represented an equal number of Republican-leaning (Dallas-Fort Worth, San Diego, Jacksonville, Oklahoma City, and Omaha) and Democratic-leaning (Houston, Denver, Tampa, Nashville, and Pittsburgh) metropolitan areas, and were comparable in population, geographic region, and racial composition.

Using Google News and Newsbank, researchers identified relevant news articles published in each of the cities, finding more than 250,000 articles and stories published or aired by about 200 different local outlets, primarily daily newspapers and TV stations.

To detect shifts in coverage, researchers had Republicans, Independents, and Democrats read a sample of these police news excerpts and assess whether each piece of text was critical or supportive of police effectiveness or racial bias in policing. Then, researchers trained large language models—an artificial intelligence method for interpreting text—to mimic these human judgments to measure criticism for the full police news data set.

“Our study sought to boost understanding of polarization in journalism, including the differences between national and local coverage,” explains Daniel Nagin, professor of public policy and statistics at Carnegie Mellon’s Heinz College, who coauthored the study. “The results suggest that local media coverage of the police has not succumbed to the partisanship that objective reporting aims to avoid.”