From Job Descriptions to Occupations - Using Neural Language Models to Code Job Data

Sep 13, 2026

Occupation is a core concept in social and policy research, yet mapping job descriptions to standardized occupational categories is difficult and error-prone. Manual expert coding translates detailed and often ambiguous text into fixed categories, but it is labor-intensive and costly. Recent computational autocoding offers efficient alternatives, though existing tools often depend on supervised models and string matching without deep semantic understanding. This work introduces a T5-based occupational classifier (T5-OCC) trained on 2019 American Community Survey write-ins, improving accuracy and discussing adaptation for future neural language models.

LanguageEnglish

CategoryResumes

FormatPDF

Pages51

Reading Time18 min

AccessFree to download

Abstract

1 Introduction

Background and manual coding

Automated coding with computational techniques

Paper overview and existing autocoding tools

From Job Descriptions to Occupations: Using Neural Language Models to Code Job Data Lynda Laughlin U.S. Census Bureau [lynda.l.laughlin@census.gov](lynda.l.laughlin@census.gov) Xi Song University of Pennsylvania [xisong@sas.upenn.edu](xisong@sas.upenn.edu) Megan Wisniewski University of Pennsylvania [meganwis@sas.upenn.edu](meganwis@sas.upenn.edu) Jiahui Xu Pennsylvania State University [jpx5053@psu.edu](jpx5053@psu.edu) April 8, 2024 Keywords: Jobs and occupations, natural language processing, automated coding, coding errors Word count (not including references, footnotes, tables, and appendices): 10,862 words, 4 tables, 3 figures * The authorship is alphabetic. Xi Song acknowledges support from the University of Pennsylvania Population Studies Center, which receives support from the Eunice Shriver Kennedy National Institute of Child Health and Human Development Population Research Infrastructure Program (NIH P2C-HD044964) . We are grateful to Zack Han and Aadit Juneja for their excellent research assistance. Any views expressed are those of the authors and not those of the U.S. Census Bureau. Abstract Occupation is a fundamental concept in social and policy research, but classifying job descriptions into occupational categories can be challenging and susceptible to errors. Traditionally, this involved expert manual coding, translating detailed, often ambiguous job descriptions to standardized categories, a process both laborious and costly. However, recent advances in computational techniques offer efficient automated coding alternatives. Existing autocoding tools, including the O*NET-SOC AutoCoder, the NIOCCS AutoCoder, and the SOCcer AutoCoder, rely on supervised machine learning methods and string-matching algorithms. Yet these autocoders are not designed to understand semantic meanings in occupational write-in text. We develop a new autocoder based on Google’s Text-to-Text Transfer Transformer (T5) model. Like GPT and other large language models, T5 is pretrained on vast amounts of text data. We develop a T5-based occupational classifier (T5-OCC) model with fine-tuned model parameters and training data from occupation write-ins from the 2019 American Community Survey. By comparing our T5-OCC with existing methods, we show that the autocoding accuracy rate increases from 61 .8% to 71 . 1% . Considering the rapid change in neural language models, we conclude by offering suggestions on how to adapt our method for the development of occupational autocoding models in future research. 1 Introduction Coding job descriptions into occupational categories is a common but challenging task for researchers, analysts, and organizations to classify job roles systematically (Cain and Treiman 1981; Miller, Treiman, Cain, and Roos 1980; Treiman 1979) . The process involves translating the detailed content of a job description, sometimes abridged and ambiguous, into a standardized occupational category. Traditionally, the gold standard for such occupational classification has been manual coding by expert coders. This approach involves a meticulous process where the coder begins by thoroughly reading the job description. Once familiarized with the content, the coder matches the job titles and duties specified in the description against predefined criteria in a chosen classification system. Based on this evaluation, the job description is matched with the most fitting category or code. Although this traditional approach can adeptly categorize a vast number of job descriptions into distinct occupations, spanning from a few to hundreds of categories, it is labor-intensive, time-consuming, and often costly. With the recent development of computational techniques for automated language understanding and production, automated coding methods become a useful alternative to the manual coding method (Census Bureau 2015) . These techniques stem from the fields of machine learning and artificial intelligence, s

FAQs

Why is coding job descriptions into occupational categories challenging?

Job descriptions often contain detailed, ambiguous information, and mapping them to standardized categories is error-prone. Traditional manual coding is also labor-intensive and costly.

What limitation affects existing occupational autocoding tools?

Many rely on supervised machine learning and string-matching methods, which are not designed to capture semantic meanings in free-text job write-ins.

How does the proposed method improve occupational autocoding accuracy?

The paper develops a T5-based classifier (T5-OCC) fine-tuned on occupation write-ins from the 2019 American Community Survey, and compares it with existing approaches to show higher accuracy.

Sep 13, 2026

Upload by Taylor Morgan