Categorizing Political Campaign Messages on Social Media using Supervised Machine Learning - Research approach and algorithm accuracy

Aug 4, 2026

Upload by Skyler

Political communication scholars analyze messages through systematic content analysis, a process that demands large-scale reading and consistent categorization to describe message dimensions, types, and styles and to study their effects. Digital trace data from platforms such as Facebook and Twitter expands research opportunities while creating scale challenges when corpora contain thousands to millions of messages. This article presents a human-supervised machine learning approach to classify political campaign messages, designed to deliver theoretically informed, more accurate and reliable results than unsupervised or off-the-shelf methods.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages39

Reading Time1 h 38 min

AccessFree to download

Introduction

Challenges in scaling political message analysis

Limits of metadata and network-focused methods

Toward automated content characterization

Method overview (human-supervised machine learning)

Theoretical basis for classification

Algorithm design decisions and accuracy

RUNNING HEAD: Categorizing Campaign Messages using Supervised Machine Learning Categorizing Political Campaign Messages on Social Media using Supervised Machine Learning Jennifer Stromer-Galleya* and Patricia Rossinib a School of Information Studies, Syracuse University, Syracuse, USA; bSchool of Social & Political Sciences, University of Glasgow, Glasgow, UK *School of Information Studies, 343 Hinds Hall, Syracuse University, Syracuse, NY 13244, [j](jstromer@syr.edu)[stromer@syr.edu](jstromer@syr.edu) Word Count: 7976 Funding Details: This work was supported by a John S. and James L. Knight grant, and a Fellowship from the Tow Center for Digital Journalism at Columbia University. Disclosure Statement: The authors report there are no competing interests to declare. Biographical Note: Jennifer Stromer-Galley (Ph.D., 2002, University of Pennsylvania) is Professor in the School of Information Studies, Senior Associate Dean for Academic and Faculty Affairs, and Director of Diversity, Equity, Inclusion, and Accessibility Initiatives. She is former president of the Association of Internet Researchers. Her book Presidential Campaigning in the Internet Age received the 2015 Roderick P. Hart Top Book Award in the Political Communication Division of the National Communication Association. Jenny has been studying “social media” since before it was called social media, studying online interaction and strategic communication in a variety of contexts, including political forums and online games. She has published over 70 journal articles, proceedings, and book chapters, and received over $15 million in federal and corporate grants to support her research endeavors. RUNNING HEAD: Categorizing Campaign Messages using Supervised Machine Learning Patrícia Rossini (Ph.D., 2017, Federal University of Minas Gerais) is a Senior Lecturer in Communication, Media & Democracy at the University of Glasgow. Prior to joining UofG, she was an inaugural Derby Fellow in the Department of Communication and Media at the University of Liverpool (2019-22), and a post-doctoral researcher at the School of Information Studies (iSchool) at Syracuse University (2017-19). Patrícia studies the interplay between political communication and technologies, with a focus on digital threats to democracy—specifically, uncivil and intolerant online discourse, misand disinformation, as well as (dark) participation, democratic backsliding, and online campaigns. Her research has been funded by social media companies such as Facebook, Google, Twitter, and WhatsApp; the British Academy, and the Knight Foundation (USA) . Data availability statement: Data is available from the authors upon request. RUNNING HEAD: Categorizing Campaign Messages using Supervised Machine Learning Categorizing Political Campaign Messages on Social Media using Supervised Machine Learning Scholars have access to a rich source of political discourse via social media. Although computational approaches to understand this communication is being used, they tend to be unsupervised and off-the-shelf algorithms to describe a corpus of messages. This article details our approach at using human-supervised machine learning to study political campaign messages. Although some declare this technique too labor-intensive, it provides theoretically-informed classification, making it more accurate and reliable. This article describes the design decisions and accuracy of our algorithms, and the applicability of the approach to classifying messages from Facebook and Twitter across two cultures and to advertisements. Keywords: supervised machine learning, political campaigns, content analysis, computational social science RUNNING HEAD: Categorizing Campaign Messages using Supervised Machine Learning The study of political communication messages is a labor-intensive process. Systematic content analysis and other qualitative approaches to characterize and thematize discourse requires reading l

FAQs

Why is political communication message analysis labor-intensive?

Systematic content analysis requires reading large volumes of text and applying categorization to examine message nature, dimensions, and effects across types and styles.

What challenges arise when using digital trace data for political discourse?

The sheer volume of messages makes random sampling difficult and creates scaling problems when datasets reach thousands or millions of units.

How does the article improve classification of campaign messages on social media?

It uses human-supervised machine learning to produce theoretically informed categories, aiming for greater accuracy and reliability than unsupervised or off-the-shelf approaches.

Aug 4, 2026

Upload by Skyler