CROSS-PLATFORM PERFORMANCE OF MACHINE LEARNING MODELS - OR CU N ERD EM

Sep 17, 2026

Uploaded by Aldword

This thesis investigates the use of multiple machine learning models to predict author attributes on social media platforms. The target attributes are gender and MBTI personality type, using data from Reddit and Twitter. Models are trained and evaluated within each platform, then assessed cross-domain by training on one platform and testing on the other. Four models are compared: Logistic Regression, Random Forest, Linear SVC, and BERT. Preprocessing includes removing Twitter URLs and retweets, and using TF-IDF with word and character n-grams. GridSearch with 3-fold cross-validation finds parameters, showing strong within-domain accuracy that declines cross-domain, indicating limited generalization.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages41

Reading Time1 h 43 min

AccessFree to download

Abstract

Acknowledgments

1 Data source, ethics, code, and technology statement

1.1 Source/Code/Ethics/Technology Statement Example

2 Introduction

Research Questions

CROSS-PLATFORM PERFORMANCE OF MACHINE LEARNING MODELSON REDDIT AND TWITTER OR CU N ERD EM thesis submitted in partial fulfillment of the requirements for the degree of master of science in data science & society at the school of humanities and digital sciences of tilburg university student number 876525 committee dr. Chris Emmery dr. Dimitar Shterionov location Tilburg University School of Humanities and Digital Sciences Department of Cognitive Science & Artificial Intelligence Tilburg, The Netherlands date June 24th, 2024 word count 8155 acknowledgments I would like to thank my supervisor dr. Chris Emmery for his help and feedback during the creation of my thesis. He provided me with helpful feedback and information for me to be able to continue working on my research. I would also like to thank Tilburg University for this opportunity. CROSS-PLATFORM PERFORMANCE OF MACHINE LEARNING MODELSON REDDIT AND TWITTER orcun erdem Abstract This thesis investigates the use of multiple different machine learning models to predict attributes of an author on social media platforms. The attributes are gender and the MBTI personality type. Data from two different social media platforms will be used. Firstly Reddit, which is a fairly lesser researched social media platform that has been used for author profiling tasks. Secondly Twitter, which is amore widely used social media platform and has also been used for multiple PAN tasks. These profiling tasks will firstly only focus on both domains individually, by predicting the attributes of an author with using the data as training and testing. Afterwards the crossdomain performance of the models will be evaluated, to see how they perform by using one platform as training data and the other as testing. Four different models will be made and tested to predict the attributes of the user. These are Logistic Regression, Random Forest, Linear SVC and BERT. Before using the data in combination with these models, some preprocessing steps will be taken. So for example URLs and retweets are removed from the Twitter dataset. Methods used in combination with the models are Term FrequencyInverse Document Frequency (TF-IDF) in combination with word and character n-grams to transform the textual data in numerical vectors, which can be used in the models. To find the optimal parameters of a model, GridSearch has been ran with some possible parameters in combination with a cross-validation of 3, to increase the validity of the parameters. The results show a highest accuracy of 0.90 and 0.76 for gender on Reddit and Twitter, respectively. With the highest accuracy on a single dimension in the MBTI indicator of 0.95 and 0.80 on Reddit and Twitter respectively. The cross-domain accuracy of models is decreased compared to single domain performance. Which indicates models do not generalize as well cross-domain. 1 data source, ethics, code, and technology statement 2 1 data source, ethics, code, and technology statement 1.1 Source/Code/Ethics/Technology Statement Example Data Source: The Reddit data has been acquired from dr. Chris Emmery by submitting a data agreement. Twitter data has been acquired by the help of dr. Chris Emmery, but is from TwiSty. A corpus to aid for research in author profiling, by Ben Verhoeven, Walter Daelemans and Barbara Plank. Work on this thesis did not require collecting data from human participants or animals. Consent for Reddit has been given, after signing a data agreement, Twitter data was available. The original owner of the data and code used in this thesis retains ownership of the data and code during and after the completion of this thesis. The thesis code can be accessed through the GitHub repository following the link [’[https://github.com/NucroQ/Author-Profiling](https://github.com/NucroQ/Author-Profiling)’]. Some code has been taken from [’[https://github.com/google-research/bert](https://github.com/google-research/bert)’

FAQs

What author attributes does the thesis predict?

The thesis predicts gender and MBTI personality type for authors on social media platforms.

Which social media platforms and datasets are used?

It uses Reddit data and Twitter data, comparing model performance both within each domain and across domains.

Which machine learning models are tested, and how are they evaluated?

The models tested are Logistic Regression, Random Forest, Linear SVC, and BERT. Evaluation is done first on single-domain training/testing, then on cross-domain performance using one platform for training and the other for testing.

Sep 17, 2026

Uploaded by Aldword