Delta Debugging in the Absence of Test Oracles Through Metamorphic Testing

Jul 16, 2026

Upload by Anda

Delta debugging provides an automatic way to minimize a program input while preserving a specified property, yet its usability depends on test oracles that verify whether a reduced input still maintains correctness. This oracle dependency restricts existing delta debugging methods for oracle-deficient programs. The paper introduces DDMT, which redesigns an oracle-independent test function and integrates it into delta debugging reductions. DDMT uses metamorphic testing to validate property preservation without requiring a traditional test oracle. Experiments on 66 subjects across oracle-available and oracle-deficient settings show improved applicability with maintained or enhanced reduction effectiveness and query efficiency.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages13

Reading Time33 min

Introduction

Delta Debugging for Input Simplification

Prior Work and Optimization Approaches

Delta Debugging in the Absence of Test Oracles Through Metamorphic Testing Mingyue Jiang, Yongqiang Tian, Tsong Yueh Chen arXiv :2607 .00929v 1 [ cs . SE] 1 Jul 2026 Abstract—Delta debugging provides an automatic way to minimize a program input while preserving a certain property. However, its effectiveness fundamentally relies on the availability of test oracles to determine whether a reduced input still preserves the specific property. Consequently, the oracle problem substantially limits the applicability of existing delta debugging techniques, particularly for oracle-deficient programs where output correctness cannot be directly determined. To address this problem, this paper proposes a novel approach, DDMT, to enhance the applicability of delta debugging, especially facilitating its application to oracle-deficient programs. Our key insight is to redesign an oracle-independent test function and incorporate it into the reduction procedure of delta debugging such that the property-preservation validation can be accomplished without requiring a test oracle. To this end, DDMT employs the technique of metamorphic testing, which is a property-based and oracleindependent testing method. It establishes a metamorphic testingbased test function, using it as a replacement for the original test function adopted by delta debugging. The experiments evaluate DDMT on 66 subjects across both oracle-available and oracledeficient scenarios, with different delta debugging approaches. The results positively confirm that DDMT can enhance the applicability of delta debugging while often preserving or improving reduction effectiveness and query efficiency. Furthermore, compared to the relevant delta debugging approaches, DDMT is also able to achieve performance improvements with proper configurations. Index Terms—Delta debugging, Metamorphic testing, Oracle problem. I. INTRODUCTION TEST inputs are key artifacts for program debugging. A failure-causing input that reveals program failures can aid the debugging process in finding the bug, understanding the root cause of the bug, fixing the bug, and so on. However, such an input may be overly large and complex, because it may contain some information that is irrelevant to the cause of failure. The debugging of a program using such an input may be unnecessarily cumbersome and inefficient. Therefore, a simplified input that preserves the failure-revealing capability of the original input can facilitate more effective debugging. M. Jiang is with School of Information Science and Technology, Zhejiang Sci-Tech University, Hangzhou, China ([e-mail: mjiang@zstu.edu.cn](e-mail: mjiang@zstu.edu.cn)) Y. Tian is with Department of Software Systems & Cybersecurity, Monash University, Melbourne, Australia ([e-mail: Yongqiang.Tian@monash.edu](e-mail: Yongqiang.Tian@monash.edu)) T. Y. Chen is with the School of Science, Computing and Emerging Technologies, Swinburne University of Technology, Hawthorn, VIC 3122 Australia (e-mail: [tychen@swin.edu.au](tychen@swin.edu.au)) Corresponding author: Yongqiang Tian Preprint—under review. This is the authors’ version. This research was partially supported by use of the Nectar Research Cloud, a collaborative Australian research platform supported by the NCRIS-funded Australian Research Data Commons (ARDC) . It was also supported by the National Nature Science Foundation of China (Grant No.61802349) . Delta debugging (DD) [1], [2], [3] is a systematic approach towards the automated input simplification. It automatically reduces an input of the target software system into the essential minimum that still preserves the property of interest (e.g. the capability of revealing a specific type of failures) . DD is traditionally built upon the ddmin algorithm, which follows a divide-and-conquer strategy by recursively partitioning inputs and eliminating irrelevant elements. To simplify program inputs, the algorithm repeatedly generates reduced candidate inp

FAQs

What problem does the paper address in delta debugging?

Delta debugging effectiveness is constrained by the availability of test oracles needed to confirm that reduced inputs still preserve the desired property.

How does DDMT enable delta debugging without test oracles?

DDMT redesigns the validation test function to be oracle-independent by using metamorphic testing, embedding that function into the reduction procedure.

What do the experiments evaluate, and what are the main results?

The experiments assess DDMT on 66 subjects under both oracle-available and oracle-deficient scenarios, comparing different delta debugging approaches. Results indicate improved applicability while often preserving or improving reduction effectiveness and query efficiency, and sometimes improving performance with proper configurations.

Jul 16, 2026

Upload by Anda