Identification of Potentially Misclassified Crash Narratives using Machine Learning (ML) and Deep Learning (DL)

Aug 5, 2026

This research investigates the efficacy of machine learning (ML) and deep learning (DL) methods in detecting misclassified intersection-related crashes within police-reported narratives. Using 2019 crash data from the Iowa Department of Transportation, the study implements and compares SVM, XGBoost, BERT sentence and word embeddings, and Albert-based models. Performance is validated against expert reviews of potentially misclassified narratives, ensuring rigorous classification accuracy. Results show hybrid strategies that combine narrative text with structured crash data reduce error rates by 54.2%, improving crash data quality for transportation safety management.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages32

Reading Time1 h 21 min

AccessFree to download

Abstract

Introduction

Crash data quality and misclassification

Role of crash narratives

Limitations of manual narrative review

Need for automated ML/DL approaches

arXiv :2507 .03066v 1 [ cs .CL] 3 Jul 2025 Identification of Potentially Misclassified Crash Narratives using Machine Learning (ML) and Deep Learning (DL) Sudesh Bhagat*1 , Ibne Farabi Shihab*2 , and Jonathan Wood3 1 Department of Civil, Construction and Environmental Engineering, , Iowa State University, 813 Bissell Road, Ames, IA 50011, USA. , [bhagat@iastate. edu](bhagat@iastate. edu) 2 Department of Computer Science, , Iowa State University, Ames, IA 50011, USA. , [ishihab@iastate. edu](ishihab@iastate. edu) 3 Department of Civil, Construction and Environmental Engineering, , Iowa State University, 813 Bissell Road, Ames, IA 50011, USA. , [jwood2@iastate. edu](jwood2@iastate. edu) Abstract This research investigates the efficacy of machine learning (ML) and deep learning (DL) methods in detecting misclassified intersection-related crashes in police-reported narratives. Using 2019 crash data from the Iowa Department of Transportation, we implemented and compared a comprehensive set of models including Support Vector Machine (SVM), XGBoost, BERT Sentence Embeddings, BERT Word Embeddings, and Albert Model. Model performance was systematically validated against expert reviews of potentially misclassified narratives, providing a rigorous assessment of classification accuracy. Results demonstrated that while traditional ML methods exhibited superior overall performance compared to some DL approaches, the Albert Model achieved the highest agreement with expert classifications (73% with Expert 1) and original tabular data (58%) . Statistical analysis revealed that the Albert Model maintained performance levels similar to inter-expert consistency rates, significantly outperforming other approaches particularly on ambiguous narratives. This work addresses a critical gap in transportation safety research through multi-modal integration analysis, which achieved a 54.2% reduction in error rates by combining narrative text with structured crash data. We conclude that hybrid approaches combining automated classification with targeted expert review offer a practical methodology for improving 0 *These authors contributed equally to this work. crash data quality, with substantial implications for transportation safety management and policy development. Keywords: Misclassification, crash narratives, machine learning, deep learning, natural language processing, expert opinion 1 Introduction Police-reported crash data are fundamental to transportation safety management, informing activities ranging from crash prediction to countermeasure evaluation and policy development (AASHTO, 2010) . The accuracy of these data critically influences the reliability of subsequent safety analyses, yet data quality issues—particularly misclassification of key variables—remain prevalent and understudied (Montella et al. , 2013; Abay, 2015; Pasindu, 2019) . When crash features such as intersection involvement are incorrectly coded, this can significantly impact safety program effectiveness and resource allocation decisions (Pasindu, 2019; Abdulhafedh, 2017) . Crash reports typically include both structured data elements and narrative descriptions. While structured data provide quantifiable information, crash narratives offer detailed accounts that may contain critical contextual information about crash circumstances (Kim et al., 2021; Trueblood et al., 2019) . These narratives represent a valuable but underutilized resource that could potentially identify and correct misclassifications in the structured data. Previous research has demonstrated the utility of crash narratives for frequency analysis (Boggs et al., 2020), severity analysis (Arteaga et al., 2020), network screening (Ambros Jiriet al., 2016), and countermeasure selection (Saha, 2022) . However, the manual review of narratives is time-intensive and impractical at scale, creating a need for automated methods (McCullough and Smith, 1998; Williamson et al., 2001) . T

FAQs

What problem does the study address in transportation safety data?

It addresses the misclassification of intersection-related crash variables in police-reported data, which can undermine safety analyses and decision-making.

Which models are compared for detecting potentially misclassified narratives?

The study compares SVM, XGBoost, BERT sentence embeddings, BERT word embeddings, and the Albert model using 2019 Iowa crash data.

How are model results validated in the research?

Model performance is systematically validated against expert reviews of potentially misclassified crash narratives, and also compared with original tabular data.

Aug 5, 2026

Upload by Seraphina