Using an LLM to Help With Code Understanding - Promising In-IDE Prompt-less Interaction

Oct 22, 2026

Upload by dawugda

Understanding code is challenging, especially in new and complex development environments where documentation is sparse or hard to navigate. The study investigates an IDE plugin that provides LLM-based conversational support for code understanding. It uses GPT-3.5-turbo with prompt-less requests to explain code, APIs, key domain terms, and usage examples, plus open-ended contextual prompts. A user study with 32 participants shows improved task completion over web search, and analyzes differing benefits for students versus professionals.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages13

Reading Time33 min

AccessFree to download

Abstract

Introduction

Using an LLM to Help With Code Understanding Daye Nam Carnegie Mellon University U.S.A. [dayen@cs.cmu.edu](dayen@cs.cmu.edu) Andrew Macvean Google, Inc. U.S.A. [amacvean@google.com](amacvean@google.com) Vincent Hellendoorn Carnegie Mellon University U.S.A. [vhellend@andrew.cmu.edu](vhellend@andrew.cmu.edu) Bogdan Vasilescu Carnegie Mellon University U.S.A. [vasilescu@cmu.edu](vasilescu@cmu.edu) Brad Myers Carnegie Mellon University U.S.A. [bam@cs.cmu.edu](bam@cs.cmu.edu) arXiv :2307 .08 177v 3 [ cs . SE] 16 Jan 2024 ABSTRACT Understanding code is challenging, especially when working in new and complex development environments. Code comments and documentation can help, but are typically scarce or hard to navigate. Large language models (LLMs) are revolutionizing the process of writing code. Can they do the same for helping understand it? In this study, we provide a first investigation of an LLM-based conversational UI built directly in the IDE that is geared towards code understanding. Our IDE plugin queries OpenAI’s GPT-3.5-turbo model with four high-level requests without the user having to write explicit prompts: to explain a highlighted section of code, provide details of API calls used in the code, explain key domainspecific terms, and provide usage examples for an API. The plugin also allows for open-ended prompts, which are automatically contextualized to the LLM with the program being edited. We evaluate this system in a user study with 32 participants, which confirms that using our plugin can aid task completion more than web search. We additionally provide a thorough analysis of the ways developers use, and perceive the usefulness of, our system, among others finding that the usage and benefits differ between students and professionals. We conclude that in-IDE prompt-less interaction with LLMs is a promising future direction for tool builders. ACM Reference Format: Daye Nam, Andrew Macvean, Vincent Hellendoorn, Bogdan Vasilescu, and Brad Myers. 2024. Using an LLM to Help With Code Understanding. In 2024 IEEE/ACM 46th International Conference on Software Engineering (ICSE’24), April 14–20, 2024, Lisbon, Portugal. ACM, New York, NY, USA, 13 pages. [https://doi.org/10.1145/3597503.3639187](https://doi.org/10.1145/3597503.3639187) 1 INTRODUCTION Building and maintaining software systems requires a deep understanding of a codebase. Consequently, developers spend a significant amount of time searching and foraging for the information they need and organizing and digesting the information they Permission to make digital or hard copies of part or all ofthis work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s) . ICSE ’24, April 14–20, 2024, Lisbon, Portugal © 2024 Copyright held by the owner/author(s) . ACM ISBN 979-8-4007-0217-4/24/04 . [https://doi.org/10.1145/3597503.3639187](https://doi.org/10.1145/3597503.3639187) find [30, 31, 34, 44, 48, 57] . Understanding code, however, is a challenging task; developers need to assimilate a large amount of information about the semantics of the code, the intricacies of the APIs used, and the relevant domain-specific concepts. Such information is often scattered across multiple sources, making it challenging for developers, especially novices or those working with unfamiliar APIs, to locate what they need. Furthermore, much of the relevant information is inadequately documented or spread across different formats and mediums, where it often becomes outdated. With the growing popularity of large language model (LLM) based code generation tools [26, 54, 67], the need for information support for code understanding is arguably growing even higher. These tools can

FAQs

What problem does the study address in code understanding?

Understanding code is difficult because needed information is scattered across sources and often poorly documented, especially for novices and unfamiliar APIs.

How does the IDE tool interact with the LLM?

The plugin provides prompt-less, high-level requests to an LLM to explain highlighted code, APIs, key terms, and give usage examples, while also supporting open-ended prompts contextualized to the current program.

What evidence supports the effectiveness of the approach?

A user study with 32 participants confirms that using the plugin helps with task completion more than web search, and the paper analyzes how usage and perceived usefulness differ between students and professionals.

Oct 22, 2026

Upload by dawugda