Teaching LLMs a Low-Resource Language Enhancing Code Completion in Pharo

Jul 16, 2026

Upload by Noah

Large Language Models enable automated code writing and power most code completion tools, yet their performance is uneven across programming languages. Low-resource languages suffer from limited training data, leaving their communities with weaker completion support. Using Pharo as a case study, a Smalltalk-inspired low-resource language whose IDE offers only single-token completion, the work presents an end-to-end LLM pipeline, tailored benchmarks, and empirical results showing specialized models improve Pharo completion and reach real-time in-IDE feasibility.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages13

Reading Time33 min

Introduction

Motivation and background

Problem statement

Approach and pipeline

Data curation

Continued pre-training and fine-tuning

Pharo benchmarks

Syntax learning evaluation

Masked code completion evaluation

Experimental results

Specialized model gains

Comparison with larger code LLMs

Conclusion

arXiv :2607 .04939v 1 [ cs . SE] 6 Jul 2026 Teaching LLMs a Low-Resource Language: Enhancing Code Completion in Pharo Kilian Kier∗ [kilian.kier@student.tugraz.at](kilian.kier@student.tugraz.at)[ ](kilian.kier@student.tugraz.at)Graz University of Technology and Lifeware SA Graz, Austria Oleksandr Zaitsev [oleksandr.zaitsev@cirad.fr](oleksandr.zaitsev@cirad.fr)[ ](oleksandr.zaitsev@cirad.fr)CIRAD, UMR SENS Montpellier, France Alessandro Giagnorio∗ [alessandro.giagnorio@usi.ch](alessandro.giagnorio@usi.ch)[ ](alessandro.giagnorio@usi.ch)Software Institute – USI Università della Svizzera italiana Lugano, Switzerland Robert Peharz [robert.peharz@tugraz.at](robert.peharz@tugraz.at)[ ](robert.peharz@tugraz.at)Graz University of Technology Graz, Austria Omar AbedelKader∗ [omar.abedelkader@inria.fr](omar.abedelkader@inria.fr)[ ](omar.abedelkader@inria.fr)Univ. Lille, Inria, CNRS, Centrale Lille UMR 9189 CRIStAL Lille, France Romain Robbes [romain.robbes@labri.fr](romain.robbes@labri.fr)[ ](romain.robbes@labri.fr)CNRS, University of Bordeaux Bordeaux INP, LaBRI, UMR5800 Talence, France Gabriele Bavota [gabriele.bavota@usi.ch](gabriele.bavota@usi.ch)[ ](gabriele.bavota@usi.ch)Software Institute – USI Università della Svizzera italiana Lugano, Switzerland Stéphane Ducasse [stephane.ducasse@inria.fr](stephane.ducasse@inria.fr)[ ](stephane.ducasse@inria.fr)Univ. Lille, Inria, CNRS, Centrale Lille UMR 9189 CRIStAL Lille, France ∗These authors contributed equally. Abstract—Large Language Models (LLMs) unlocked new possibilities in automated code writing, becoming the backbone of most code completion tools. While LLMs excel in mainstream languages, they often lack support for the so-called low-resource languages where training data is scarce. As a result, these languages lag behind in the quality of code completion tooling available to their communities. A concrete example is Pharo, a Smalltalkinspired language whose IDE currently offers only single-token completion. In this work, we report on our experience bringing LLM-based code completion to Pharo. First, we describe an end-to-end pipeline that combines Pharo-specific data curation, continued pre-training and fine-tuning of open code LLMs. Second, we introduce a set ofPharo code completion benchmarks designed to evaluate whether models (i) learn Pharo’s syntax and (ii) accurately complete masked Pharo code from realworld GitHub repositories. Third, we show empirically that Pharo-specialized models substantially outperform their original base checkpoints and also exceed the accuracy of substantially larger code LLMs on Pharo completion. Overall, our case study demonstrates the feasibility of bringing strong LLM-based code completion to low-resource programming languages, with models small enough to provide “real-time” in-IDE support. Index Terms—Code completion, Low-resource programming languages, Pharo, Smalltalk I. INTRODUCTION Code completion is a core Integrated Development Environment (IDE) feature that helps developers write code faster, boosts productivity, and cuts manual effort [1],[2] . Early codecompletion approaches typically relied on heuristics [3], [4] or machine learning [5] to recommend the most probable next token. Early n-gram and neural language models followed [6], [7] . However, they struggled to capture broader context and were limited in producing richer, multi-token completions beyond the immediate next token [8], [9] . The advent of Large Language Models (LLMs) has reshaped the landscape of developer tooling, including code completion. LLMs learn statistical regularities from massive source code corpora, and can predict likely continuations of partially written programs, going well beyond next-token suggestions [10] . However, the benefits of LLM-based code completion are not evenly distributed across programming languages. Code LLMs [11]–[13] are usually trained on publicly available repositories (e.g., those present on forges

FAQs

Why do LLM-based code completion tools underperform for low-resource programming languages like Pharo?

They are typically trained on large public code repositories, and low-resource languages have far fewer available repositories. This imbalance limits the model’s exposure to syntax and coding patterns, reducing completion quality.

What is the main limitation of Pharo’s current IDE code completion?

Pharo’s IDE currently provides only single-token completion, lacking the multi-token, context-aware suggestions expected from modern developer tools.

How does the proposed work improve LLM-based code completion for Pharo?

It builds an end-to-end pipeline that includes Pharo-specific data curation plus continued pre-training and fine-tuning of open code LLMs. It also introduces Pharo benchmarks to evaluate syntax learning and masked code completion accuracy from real GitHub repositories.

Jul 16, 2026

Upload by Noah