[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-81500-en":3,"doc-seo-81500-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},81500,1099513958607,"Jiven","https://ap-avatar.wpscdn.com/avatar/100002390cf8733938c?x-image-process=image/resize,m_fixed,w_180,h_180&k=1778829742770036399",8,"Research & Report","LDPKiT: Superimposing Remote Queries for Privacy-Preserving Distillation","LDPKiT is a privacy-preserving model distillation framework for regulated settings where model owners keep training data and parameters private while users must send potentially sensitive inputs for remote inference. The method uses local differential privacy to bound per-query privacy leakage and introduces a novel superimposition technique that generates approximately in-distribution augmented samples. Experiments on Fashion-MNIST, SVHN, and PathMNIST show improved utility under stronger noise, and sensitivity plus latent-space analyses explain accuracy gains.","arXiv :2405 . 1636 1v4 [ cs .LG] 10 Jul 2026  \nLDPKiT: Superimposing Remote Queries for Privacy-Preserving Distillation  \nKexin Li 1􀀌, Aastha Mehta2, and David Lie 1  \n1 University of Toronto, Toronto, ON, Canada  \n2 The University of British Columbia, Vancouver, BC, Canada  \n[cassiekx.li@mail.utoronto.ca](cassiekx.li@mail.utoronto.ca)  \nAbstract. To protect privacy in regulated domains such as healthcare and finance, model owners may allow only remote API access while keeping both the training data and model parameters private. However, model users performing inference on such remotely hosted models may be required to transmit potentially sensitive inputs, raising privacy concerns. In this work, we present LDPKiT, a framework for non-adversarial, privacypreserving model distillation that leverages a user’s private in-distribution data while bounding privacy leakage. LDPKiT introduces a novel superimposition technique that generates approximately in-distribution samples, enabling effective knowledge transfer under local differential privacy (LDP) . Experiments on Fashion-MNIST, SVHN, and PathMNIST demonstrate that LDPKiT consistently improves utility while maintaining privacy, with benefits that become more pronounced at stronger noise levels. For example, on SVHN, LDPKiT achieves nearly the same inference accuracy at ϵ = 1 .25 as at ϵ = 2 .0, yielding stronger privacy guarantees with less than a 2% accuracy reduction. We further conduct sensitivity analyses to examine the effect of dataset size on performance and provide a systematic analysis of latent space representations, offering intuitive and empirical insights into the accuracy gains of LDPKiT.  \nKeywords: Local differential privacy · Inference data privacy protection  \n· Privacy-preserving knowledge distillation · Data augmentation  \n1 Introduction  \nA machine learning (ML) model, trained on a dataset, is a representation of the data that can be used to make predictions and offer insights on other data that is of a similar distribution to the original training set. As a result, even in cases where the training data itself cannot be shared due to privacy restrictions, there is still a strong incentive to share access to the ML model. Moreover, techniques such as PATE [27] and federated learning [42] have been proposed to enable training such models while limiting the privacy impact on the training set. The privacy protection such techniques confer is maximized when offering only black-box access (i. e. , keeping the weights private) to such models, so that the intermediate activations computed during inference remain hidden [30] .  \n2 K. Li et al.  \nSensitive data is sometimes shared for socially beneficial reasons, particularly in regulated sectors where organizations must disclose privacy-sensitive information to protect public welfare. For example, hospitals may be required to share patient information during disease outbreaks, and financial institutions may need to report accounts suspected of money laundering. ML could enable similar benefits while reducing the need to share sensitive datasets directly: organizations with large proprietary datasets could train predictive models using privacy-protective techniques [27, 42] and provide controlled access to entities without comparable data. For instance, a healthcare network could offer smaller clinics access to a model that predicts disease risk, while a large financial institution could provide local banks with a model for detecting money laundering.  \nHowever, while black-box models trained using privacy-preserving methods protect the training set, inference in the black-box setting still requires the model user to transmit potentially sensitive inputs. While various methods for protecting privacy during inference have been proposed, none satisfy the requirements of efficiency and ease of adoption. For instance, homomorphic encryption [14] can impose severe overheads and typically targets an honest-but-curious adve","cbCaieK4fL5rNDJj","https://ap.wps.com/l/cbCaieK4fL5rNDJj","pdf",7435603,4,1,18,"English","en",105,"# Introduction\n## Problem setting: black-box privacy vs sensitive inputs\n## Key idea: LDP-protected distillation with superimposed queries\n## Research questions and contributions","[{\"question\":\"What problem does LDPKiT address in privacy-preserving distillation?\",\"answer\":\"LDPKiT addresses the privacy risk that arises when users must transmit sensitive inputs to a remote black-box model during inference, even if model parameters remain private.\"},{\"question\":\"How does LDPKiT protect user query data?\",\"answer\":\"It applies local differential privacy to bound privacy leakage of each query while enabling effective knowledge transfer from the remote model.\"},{\"question\":\"Why does the superimposition technique help?\",\"answer\":\"Superimposition generates augmented queries that are approximately in-distribution, improving distillation effectiveness even when users only have small private datasets.\"}]",1784173831,45,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"ldpkit-superimposing-remote-queries-for-privacy-preserving-distillation","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":20},"https://docshare.wps.com/document/ldpkit-superimposing-remote-queries-for-privacy-preserving-distillation/81500/",{"url":52,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-24","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does LDPKiT address in privacy-preserving distillation?","Question",{"text":75,"@type":76},"LDPKiT addresses the privacy risk that arises when users must transmit sensitive inputs to a remote black-box model during inference, even if model parameters remain private.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does LDPKiT protect user query data?",{"text":80,"@type":76},"It applies local differential privacy to bound privacy leakage of each query while enabling effective knowledge transfer from the remote model.",{"name":82,"@type":73,"acceptedAnswer":83},"Why does the superimposition technique help?",{"text":84,"@type":76},"Superimposition generates augmented queries that are approximately in-distribution, improving distillation effectiveness even when users only have small private datasets.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]