A Virtual Member of a Community of Practice for the Society of Petroleum Engineers - From Prototype to Deployment

Sep 15, 2026

Uploaded by Levi

This paper describes the evolution of ATHENA, a virtual assistant supporting knowledge capture, retrieval, and dissemination for a Community of Practice in the Oil & Gas sector. Evaluation of a first prototype with 75 Society of Petroleum Engineers professionals shows large gains in productivity and task performance on realistic well-planning activities versus a state-of-the-art RAG baseline. The work also identifies improvement areas and reports technical advances in multi-document retrieval, answer validation, and more targeted proactive dissemination.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages7

Reading Time18 min

AccessFree to download

Introduction

ATHENA as a virtual CoP member

Knowledge management challenges in Oil & Gas

Document corpus and evaluation setup

Initial ATHENA Capability and Evaluation

Initial ATHENA capability

Agentic RAG approach

A Virtual Member of a Community of Practice for the Society of Petroleum Engineers: From Prototype to Deployment John Boden 1, Joshua Eckroth1, Dayne Freitag2, Skyler Gipson1, Jonathan Keefe3, Karen Myers2, Eric Schoen1, Pedro Sequeira2, Reid Smith1, Michael Wessel2 1i2k Connect Inc. 2SRI International 3Pacific Science & Engineering Abstract We describe the evolution of a virtual assistant, called ATHENA, designed to support the capture, retrieval, and dissemination of knowledge for members of a Community of Practice (CoP) related to the Oil & Gas sector. An evaluation of a first prototype involving 75 professionals from the Society of Petroleum Engineering (SPE) showed that ATHENA dramatically improved both their productivity and performance quality on a set of realistic well-planning tasks compared to their use of a state-of-the-art RAG baseline system. However, the evaluation also identified areas for improvement. This paper describes technical advances to our first prototype in the areas of multi-document retrieval, support for answer validation, and more focused proactive dissemination. Evaluation results show that this enhanced version of ATHENA provides better support for completing knowledge-intensive tasks related to well planning than does a state-of-the-art baseline. ATHENA has been integrated into the SPE Research Portal and is being deployed for use by the society’s membership. Introduction We previously developed an intelligent assistant called ATHENA that operates as a virtual member of a Community of Practice (CoP) (Wenger 1998; Allee 2000) by modeling how experts naturally seek and share knowledge (Eckroth et al. 2025) . ATHENA combines three core capabilities embedded within a chat-based user interface that mirrors the natural workflow of person-to-person knowledge sharing: • an agentic approach to information retrieval that delivers accurate answers rather than raw data, along with rationale and links to source material; • in-the-flow capture of experiential knowledge, or “insights”, that can be shared within a CoP; • proactive dissemination of insights tailored to an individual’s expertise level and task context via user cognitive modeling and large language models (LLMs) . Our objective was to develop capabilities that could be used broadly. However, to ensure their utility in real-world settings we focused from the start on addressing knowledge management challenges for the Oil & Gas sector with a particular focus on well planning. Well planning involves designing and mapping out the wellbore, estimating costs, and identifying the optimal drilling approach to access hydrocarbons efficiently and safely (Devereux 1998) . These tasks require a comprehensive analysis of geological data, regulatory guidelines, and potential environmental impacts. Finding the right information is paramount, as accurate data helps minimize risks, optimize resources, reduce costs, and ensure compliance with safety and environmental standards. We worked with a corpus of more than 27,000 documents containing general petroleum engineering knowledge drawn from PetroWiki (OnePetro 2025), regulatory matter (US Government 2025), and an extensive data set comprising logs, daily and end-of-well reports, and studies from drilling operations in the Volve North Sea oil field from 2008 to 2016 (Equinor 2025) . Our focus on knowledge management for Oil & Gas was facilitated by members ofour team having decades of experience in knowledge management for that sector, including having developed the current research portal for the Society of Petroleum Engineers (SPE), which is the premiere professional society devoted to petroleum engineering with 132,000 members across 146 countries. Leveraging those connections, we evaluated our initial ATHENA prototype using 75 volunteers from the SPE engaged in realistic well-planning tasks. The results showed that ATHENA significantly improved outcomes (152% increase in task sco

FAQs

What is ATHENA and what role does it play for the SPE Community of Practice?

ATHENA is a virtual assistant designed to support how CoP members capture, retrieve, and share knowledge. It models expert knowledge seeking and sharing through a chat-based workflow.

How was ATHENA evaluated in the first prototype study?

A first prototype was evaluated with 75 Society of Petroleum Engineers professionals completing realistic well-planning tasks. Results compared ATHENA to a state-of-the-art RAG baseline.

What technical improvements are described beyond the initial prototype?

The paper describes advances including chat-based multi-document retrieval, improved support for answer validation, and more focused proactive dissemination of insights tailored to user context.

Sep 15, 2026

Uploaded by Levi