New Machine Learning Techniques for Simulation-Based Inference - InferoStatic Nets, Kernel Score Estimation, and Kernel Likelihood Ratio Estimation

Aug 6, 2026

Upload by Jiven

New Machine Learning Techniques for Simulation-Based Inference presents an intuitive machine-learning framework for multiparameter inference when probability densities can be sampled but cannot be computed directly. The InferoStatic Networks (ISN) method employs a neural network to model an inferostatic potential underlying score and likelihood-ratio estimators. The work further develops Kernel Score Estimation and Kernel Likelihood Ratio Estimation to learn these functions from simulated data, and it validates the ideas using toy examples and comparisons to prior approaches.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages35

Reading Time1 h 28 min

AccessFree to download

Introduction

Applications of Estimated Scores and Likelihood Ratios

Related Techniques and New Contributions in This Work

Methodology: InferoStatic Networks (ISNs)

Methodology: Kernel Score Estimation

Intuition and Motivation

Kernel Score Approximation

Kernel Score Estimation using ML

Alternative Version of Kernel Score Approximation and Estimation

Methodology: Kernel Likelihood Ratio Estimation

Experiments and Results

Tasks

NN Architecture and Training Details

Results

Conclusions and Outlook

New Loss Functions to Utilize Latent Information

Feed-Forward Nature of the Gradient Network

Derivation of the Kernel Score Estimation Technique

Narrowing Down the Choices for KSA

arXiv :2210 .01680v1 [ stat .ML] 4 Oct 2022 FERMILAB-PUB-22-741-SCD SciPost Physics Submission New Machine Learning Techniques for Simulation-Based Inference: InferoStatic Nets, Kernel Score Estimation, and Kernel Likelihood Ratio Estimation Kyoungchul Kong1 , Konstantin T. Matchev2 , Stephen Mrenna3 , and Prasanth Shyamsundar4?, 1 Department of Physics and Astronomy, University of Kansas, Lawrence, KS 66045, USA 2 Institute for Fundamental Theory, Physics Department, University of Florida, Gainesville, FL 32611, USA 3 Scientiﬁc Computing Division, Fermi National Accelerator Laboratory, Batavia, IL 60510, USA 4 Fermilab Quantum Institute, Fermi National Accelerator Laboratory, Batavia, IL 60510, USA? [prasanth@fnal.gov](prasanth@fnal.gov) October 4, 2022 Abstract We propose an intuitive, machine-learning approach to multiparameter inference, dubbed the InferoStatic Networks (ISN) method, to model the score and likelihood ratio estimators in cases when the probability density can be sampled but not computed directly. The ISN uses a backend neural network that models a scalar function called the inferostatic potential . In addition, we ' introduce new strategies, respectively called Kernel Score Estimation (KSE) and Kernel Likelihood Ratio Estimation (KLRE), to learn the score and the likelihood ratio functions from simulated data. We illustrate the new techniques with some toy examples and compare to existing approaches in the literature. We mention en passant some new loss functions that optimally incorporate latent information from simulations into the training procedure. Contents 1 Introduction 2 1.1 Applications of Estimated Scores and Likelihood Ratios 3 1.2 Related Techniques and New Contributions in This Work 4 2 Methodology: InferoStatic Networks (ISNs) 5 3 Methodology: Kernel Score Estimation 8 3.1 Intuition and Motivation 8 3.2 Kernel Score Approximation 11 3.3 Kernel Score Estimation using ML 12 3.4 Alternative Version of Kernel Score Approximation and Estimation 14 4 Methodology: Kernel Likelihood Ratio Estimation 15 5 Experiments and Results 17 5.1 Tasks 18 5.2 NN Architecture and Training Details 20 5.3 Results 20 6 Conclusions and Outlook 23 A New Loss Functions to Utilize Latent Information 25 A.1 Background 25 A.2 New Loss Functions 26 A.3 Proofs 26 B Feed-Forward Nature of the Gradient Network 28 C Derivation of the Kernel Score Estimation Technique 30 D Narrowing Down the Choices for KSA 31 D.1 Bias of the Kernel Score Approximation 31 D.2 Local Variance of the Regression Target 32 D.3 Choosing the Kernel and Difference Function 32 References 34 1 Introduction Inference in physical sciences, such as particle physics, relies on comparing detailed predictions from computationally expensive simulations to data. These predictions depend upon input parameters that are the objects of interest in parameter-estimation analyses. Classical inference techniques for parameter measurement include the analysis of histograms of summary statistics, the matrix element method, optimal observables, etc. (see [1, 2] for recent reviews and a guide to the literature) . More recently, there has been an explosion of interest in corresponding Machine Learning (ML) techniques for parameter measurement, which rely only on samples generated at different parameter values. The basic appeal of the ML approach is that it can leverage highdimensional information not captured by summary statistics. An up-to-date compendium of the literature on ML applications in particle physics is maintained at [3] . The ML problem at hand can be described as follows. Let x = (x 1 , . . . , xD) be a D-dimensional random variable (datapoint; collision event in the context of collider physics) whose unit-normalized distribution under a given theory model is p (x ; 􀀒 ), where 􀀒 􀀑 (􀀒1 , . . . , 􀀒 d) is a d-dimensional continuous parameter of the model. A standard problem,

FAQs

What problem does InferoStatic Networks (ISN) address in simulation-based inference?

ISN targets multiparameter inference settings where probability densities can be sampled but not directly computed, making conventional likelihood-based methods difficult to apply.

How does ISN represent the estimators used for score and likelihood ratio estimation?

ISN trains a backend neural network to model a scalar inferostatic potential, which is then used to form score and likelihood-ratio estimators.

What are Kernel Score Estimation (KSE) and Kernel Likelihood Ratio Estimation (KLRE)?

KSE and KLRE are strategies that learn the score function and the likelihood-ratio function from simulated data, using kernel-based approaches to approximate or estimate the required quantities.

Aug 6, 2026

Upload by Jiven