Assessing Coverage and Temporal Reliability of Twitter's Academic API

Sep 13, 2026

Upload by Lute

Because Twitter shares data with academia and industry, it has become a central platform for research and consulting. Recent work has criticized Twitter’s APIs and led to partial changes in data streams. This paper examines the Academic Research API, focusing on whether its collected datasets are complete and how much historic data is lost over time due to Tweet and account removal. Results indicate near-complete samples from broad search terms, with the v2 endpoint outperforming v1.1, alongside technical artifacts and study implications.

LanguageEnglish

CategoryGeneral

FormatPDF

Pages10

Reading Time4 min

AccessFree to download

Abstract

Introduction

Data access strategies

Research approach and comparisons

Contributions

Related Work

This Sample seems to be good enough! Assessing Coverage and Temporal Reliability of Twitter's Academic API Jrgen Pfeffer, 1 Angelina Mooseder, 1 Jana Lasser, 2 Luca Hammer, 3 Oliver Stritzel, 2 David Garcia 2 1 School of Social Science and Technology, Technical University of Munich, Germany 2 Faculty of Informatics, Technical University of Vienna, Austria 3 Department of Media Studies, University of Siegen, Germany arXiv :2204 .02290v 3 [ cs . SI] 11 Apr 2023 Abstract Because of its willingness to share data with academia and industry, Twitter has been the primary social media platform for scientiﬁc research as well as for consulting businesses and governments in the last decade. In recent years, a series of publications have studied and criticized Twitter's APIs and Twitter has partially adapted its existing data streams. The newest Twitter API for Academic Research allows to ”access Twitter's real-time and historical public data with additional features and functionality that support collecting more precise, complete, and unbiased datasets.”1 The main new feature of this API is the possibility of accessing the full archive of all historic Tweets. In this article, we will take a closer look at the Academic API and will try to answer two questions. First, are the datasets collected with the Academic API complete? Secondly, since Twitter's Academic API delivers historic Tweets as represented on Twitter at the time of data collection, we need to understand how much data is lost over time due to Tweet and account removal from the platform. Our work shows evidence that Twitter's Academic API can indeed create (almost) complete samples of Twitter data based on a wide variety of search terms. We also provide evidence that Twitter's data endpoint v2 delivers better samples than the previously used endpoint v1.1 . Furthermore, collecting Tweets with the Academic API at the time of studying a phenomenon rather than creating local archives of stored Tweets, allows for a straightforward way of following Twitter's developer agreement. Finally, we will also discuss technical artifacts and implications of the Academic API. We hope that our work can add another layer of understanding of Twitter data collections leading to more reliable studies of human behavior via social media data. Introduction In 2013, David Lazer opened his ICWSM (Emre Kicimanet al. 2013) keynote address with the joke ”Welcome to the 7th International Conference on Twitter Analysis”, referring to the fact that a majority of published papers at this conference were working with Twitter data. While a series of other social media platforms have attracted the attention of academic researchers in recent years, Twitter has remained Copyright © 2023, Association for the Advancement of Artiﬁcial Intelligence ([www.aaai.org](www.aaai.org)). All rights reserved. 1[https://developer.twitter.com/en/products/twitter-api/](https://developer.twitter.com/en/products/twitter-api/)[ ](https://developer.twitter.com/en/products/twitter-api/)[academic-research](academic-research) the dominant data source for social media analysis of human behavior due to its data accessibility via a variety of application programming interfaces (API) . Several review articles have tried to create an overview of the countless num˜ ber of studies that have utilized Twitter data (Yu and MunozJusticia 2020; Williams, Terras, and Warwick 2013; Zimmer and Proferes 2014; Jungherr 2015; Sinnenberg et al. 2017; Karami et al. 2020; Pradyumn, Kapoor, and Tabrizi 2018) . For most of the data driven studies in recent years, two main data access strategies have been applied. Tweets were collected either with ﬁltering queries (e.g. keywords, accounts, geo-locations) or as random 1% or 10% samples of all Tweets. Both data collection strategies relied on real-time data collection. With the Academic API, researchers are now able to perform full-archive searches, which has created a wide variety

FAQs

What are the two main questions addressed about Twitter's Academic API?

The study asks whether datasets collected with the Academic API are complete and how much data is lost over time because historic Tweets and accounts may be deleted or removed from the platform.

How does the paper assess completeness and temporal reliability?

It re-collects historic data, uses API error messages for missing Tweets, and compares Academic API samples against data collected with the Twitter Premium API using the same search queries.

What evidence does the paper provide about different API endpoints?

It reports that the Academic API can produce (almost) complete Twitter data samples using varied search terms and that the v2 endpoint delivers better samples than the previously used v1.1 endpoint.

Sep 13, 2026

Upload by Lute