The INRIA DataLake: A Generic and Scalable Ecosystem of Pipelines for HAL Applied to Software Mentions Tracking

Jul 16, 2026

Upload by Violet

Research repositories store abundant scientific knowledge, yet structured access to articles and specialized assets—such as datasets and software metadata—remains limited. The INRIA DataLake project delivers an ecosystem of scalable, interconnected pipelines to prepare scientific literature, extract structured information, and support specialized downstream treatments. Deployed on Grid’5000/ABACA, the project demonstrates software-mention extraction from daily-deposited articles and their validated visualization in the HAL portal, enabling efficient large-volume processing, user validation, interoperability, and extensibility for open science impact tracking.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages10

Reading Time25 min

Abstract

Introduction

System Architecture

arXiv :2607 .09824v 1 [ cs .DL] 10 Jul 2026 The INRIA DataLake: A Generic and Scalable Ecosystem of Pipelines for HAL Applied to Software Mentions Tracking Luca Foppiano 1 ,3 ,5[0000−0002−6114−6164]⋆ , Vipul Gupta2[0000−0002−6311−4422], Samuel Scalbert3[0009−0002−0423−9281], Estelle Niveaut3[0000−0003−0630−5633], Kumar Guha3[0009−0005−6649−9515], Yannick Barborini4[0000−0003−3756−8647], Alain Monteil3[0000−0003−3150−4837], and Laurent Romary3[0000−0002−0756−0508] 1 ScienciaLAB, Portugal 2 Institute of Materials Physics, Helmholtz-Zentrum Hereon, Germany 3 Inria, France 4 CCSD, CNRS, France 5 Common Crawl Foundation, United States Abstract. Research repositories contain a large amount of scientific knowledge, but access to structured articles and specialised information, such as datasets or software metadata, remains limited. In this paper, we present the INRIA DataLake project, which provides an ecosystem of scalable and interconnected pipelines for preparing scientific literature, extracting structured information, and applying specialised treatments. Using a large-scale shared infrastructure, Grid’5000/ABACA, we demonstrate our ecosystem through a concrete use case: extracting software mentions from scientific articles deposited daily and visualising them after validation in the HAL research portal. Our results show that the system can efficiently process large volumes of scientific literature while supporting user validation and interoperability with external systems. Designed to grow by integrating additional pipelines and sharing the preparation effort across research groups, this project already contributes to open science through improved visibility and tracking of research software. Keywords: HAL · Open Science · Information Extraction · Digital Libraries · Software Mentions 1 Introduction Digital repositories host a rapidly growing volume of scientific knowledge. However, most of this information remains embedded and dispersed in scholarly articles [12], which limits its accessibility and reuse. Scientific research suffers from ⋆ Corresponding author: [luca@sciencialab.com](luca@sciencialab.com) This preprint has not undergone peer review or any post-submission improvements or corrections. This work has been submitted to TPDL 2026 (Lecture Notes in Computer Science, Springer) for consideration. 2 Foppiano et al. a persistent lack of data mutualisation, leading teams to repeatedly duplicate efforts in data preparation. While traditional search systems support keywordbased retrieval, they do not provide access to structured data or to important research outputs, such as software, datasets, or experimental details. In recent years, open science policies have emphasised the need to better capture and measure research production beyond publications. For example, in France, the Open Science Monitor [3] reflects this evolution by aiming to quantify not only citation-based impact, but also the production of software and datasets. Achieving this objective requires scalable methods to extract structured information from large document collections. Data lakes have emerged as a dominant paradigm for storing and processing large volumes of heterogeneous data [9,4] . While classical data lake architectures defer transformation to the point of use, scientific literature processing benefits from early structured extraction—converting PDFs into machine-readable representations that downstream pipelines can consume without repeated parsing effort. Existing data lake solutions remain largely general-purpose and do not address these domain-specific needs, such as integration with scholarly repositories and open science infrastructures. In this paper, we present an ecosystem of generic, scalable, and interconnected pipelines for processing scholarly articles, extracting structured information, and enabling post-hoc analytics and insight generation. The system is designed to work at scale, and to support m

FAQs

What problem does the INRIA DataLake address in scientific repositories?

Scientific knowledge is often embedded in scholarly PDFs and dispersed across articles, making structured access and reuse of specialized outputs like software metadata difficult.

How does the project process scientific literature at scale?

It uses a generic ecosystem of modular, interconnected pipelines executed on Grid’5000/ABACA to prepare documents, extract structured information, and enable asynchronous post-hoc analytics.

What is the concrete use case demonstrated with HAL?

The system extracts software mentions from pre-processed structured scientific articles deposited daily, supports human validation, and visualizes results in the HAL research portal while disseminating them to external services.

Jul 16, 2026

Upload by Violet