LCEN - A Novel Feature Selection Algorithm for Nonlinear, Interpretable Machine Learning Models - preprint

Aug 4, 2026

Upload by Oliver

Interpretable machine learning models strengthen trust in high-stakes applications, yet common sparse linear architectures like LASSO and elastic net are limited to linear predictions and can struggle with feature selection. The LCEN algorithm (LASSO-Clip-EN) enables nonlinear, interpretable models while producing accurate, sparser solutions. LCEN is tested across artificial and empirical datasets and remains robust to noise, multicollinearity, data scarcity, and hyperparameter variation, rediscovering multiple physical laws from data.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages20

Reading Time50 min

AccessFree to download

Abstract

1 Introduction

Statistical models and interpretability

Feature selection overview

Limitations of common interpretable sparse models

Extending sparse models to nonlinear settings

LCEN: A Novel Feature Selection Algorithm for Nonlinear, Interpretable Machine Learning Models arXiv :2402 . 17 120v2 [ cs .LG] 6 Jun 2024 Pedro Seber Department of Chemical Engineering Massachusetts Institute of Technology Cambridge, MA 02139 [pseber@mit.edu](pseber@mit.edu) Richard D. Braatz Department of Chemical Engineering Massachusetts Institute of Technology Cambridge, MA 02139 [braatz@mit.edu](braatz@mit.edu) Abstract Interpretable architectures can have advantages over black-box architectures, and interpretability is essential for the application of machine learning in critical settings, such as aviation or medicine. However, the simplest, most commonly used interpretable architectures, such as LASSO or elastic net (EN), are limited to linear predictions and have poor feature selection capabilities. In this work, we introduce the LASSO-Clip-EN (LCEN) algorithm for the creation of nonlinear, interpretable machine learning models. LCEN is tested on a wide variety of artificial and empirical datasets, frequently creating more accurate, sparser models than other architectures, including those for building sparse, nonlinear models. LCEN is robust against many issues typically present in datasets and modeling, including noise, multicollinearity, data scarcity, and hyperparameter variance. LCEN is also able to rediscover multiple physical laws from empirical data and, for processes with no known physical laws, LCEN achieves better results than many other dense and sparse methods – including using 10.8-fold fewer features than dense methods and 8.1-fold fewer features than EN on one dataset, and is comparable to or better than ANNs on multiple datasets. 1 Introduction Statistical models are powerful tools to explain, predict, or describe natural phenomena [1] . They connect independent variables (also called “inputs” or “features”) to dependent variables (also called“outputs” or “labels”) to test causal hypotheses, predict novel outputs from known inputs, or summarizing the data structure [1] . Many model architectures exist, including linear, ensemble-based, and deep learning models. Complex architectures are claimed to have greater capability to model phenomena due to their lower bias, but their intricate and numerous mathematical transformations prevent humans from understanding how an output was predicted by a model, or the relative or absolute importance of the inputs. Moreover, a lack of transparency may prevent the model from being trusted in critical or sensitive applications [2] . As summarized by Ref. [3], there are two main methods to increase interpretability: the use of model-agnostic algorithms, which extract interpretable explanations a posteriori and work for any architecture, or the direct use of interpretable architectures. Interpretable architectures include “decision trees, rules, additive models, attention-based networks, and sparse linear models” [3] . It should be noted that nonlinear models may also be made sparse, and even interpretable, as described later in this section and the rest of this work. As elaborated in the review of Ref. [4], interpretable architectures can have many advantages over black-box or a posteriori explanations, including the ability to assist researchers in refining the model and data, or better highlighting scenarios in which the model fails or lacks robustness. Special attention should be given to sparse models, which identify Preprint. Under review. the most important features, can make the model more robust to variations in the input data, and can significantly improve the model’s interpretability if an interpretable architecture is used [4] . At the same time, even a linear model or decision tree/rules can become unwieldy and uninterpretable if hundreds or thousands of coefficients or rules are present. Feature selection is the process of selecting the most important features in a model to increase its robustness or interpretability. Many criter

FAQs

What problem does LCEN address in interpretable machine learning?

LCEN targets the limitation of common interpretable sparse linear methods like LASSO and elastic net, which are restricted to linear predictions and often underperform in feature selection.

How does LCEN perform feature selection and model interpretability?

LCEN creates nonlinear, interpretable machine learning models and frequently yields more accurate, sparser solutions than other architectures, including approaches for sparse nonlinear modeling.

What kinds of dataset and modeling challenges is LCEN robust against?

LCEN is reported to be robust to noise, multicollinearity, data scarcity, and hyperparameter variance, which often degrade model performance and reliability.

Aug 4, 2026

Upload by Oliver