[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-119736-en":3,"doc-seo-119736-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},119736,687197100911,"Himbo","https://ap-avatar.wpscdn.com/avatar/a000239b6f1da00475?x-image-process=image/resize,m_fixed,w_180,h_180&k=1785132997149421697",8,"Research & Report","A Distributed and Real-time Machine Learning Framework for Smart Meter Big Data","Advanced metering infrastructure enables smart meters to collect high-resolution energy consumption data, supporting smart grid applications and multi-level insights for both consumers and utilities. Developing machine learning frameworks for this data is difficult due to big-data scale and privacy constraints that do not exist in classic theoretical settings. This thesis proposes multi-level algorithm comparisons, an offline-to-online functional analysis framework for real-time forecasting, and a distributed federated learning approach combining FederatedNILM with differential privacy for appliance-level privacy guarantees.","A Distributed and Real-time Machine Learning Framework for Smart Meter Big  \nData  \nShuang Dai  \nA thesis submitted for the degree of Doctor of Philosophy in  \nData Science  \nDepartment of Mathematical Sciences University of Essex  \nJanuary 2023  \nAbstract  \nThe advanced metering infrastructure allows smart meters to collect high-resolution consumption data, thereby enabling consumers and utilities to understand their energy usage at different levels, which has led to numerous smart grid applications. Smart meter data, however, poses different challenges to developing machine learning frameworks than classic theoretical frameworks due to their big data features and privacy limitations.  \nTherefore, in this work, we aim to address the challenges of building machine learning frameworks for smart meter big data. Specifically, our work includes three parts: 1) We first analyze and compare different learning algorithms for multi-level smart meter big data. A daily activity pattern recognition model has been developed based on non-intrusive load monitoring for appliance-level smart meter data. Then, a consensus-based load profiling and forecasting system has been proposed for individual building level and higher aggregated level smart meter data analysis; 2) Following discussion of multi-level smart meter data analysis from an offline perspective, a universal online functional analysis model has been proposed for multi-level real-time smart meter big data analysis. The proposed model consists of a multiscale load dynamic profiling unit based on functional clustering and a multi-scale online load forecasting unit based on functional deep neural networks. The two units enable online tracking of the dynamic cluster trajectories and online forecasting of daily multi-scale demand; 3) To enable smart meter data analysis in the distributed environment, FederatedNILM was proposed, which is then combined with differential privacy to provide privacy guarantees for the appliance-level distributed machine learning framework. Based on federated deep learning enhanced with two schemes, namely the utility optimization scheme and the privacy-preserving scheme, the proposed distributed and privacy-preserving machine learning framework enables electric utilities and service providers to offer smart meter services on a large scale.  \nNomenclature  \nADLs Activities of daily life  \nAMI Advanced metering infrastructure ANN Artificial neural network ARMA Auto-regressive moving-average  \nARMIA Auto-regressive integrated moving average BIC Bayesian information criterion  \nBP Backpropagation CH Calinski-Harabasz  \nCNN Convolutional neural network CO Combinatorial optimization CVI Cluster validity indice  \nDB-index Davies-Buldin validity index DNN Deep neural network  \nDPFL Differential private federated learning DTW Dynamic time warping distance FDA Functional data analysis  \nFDN Functional deep neural network  \nFHMM factorial hidden Markov model  \nFL Federated learning FN False negative FP False positive  \nFPCA Functional principal component analysis  \nFR Functional regression FTL Federated transfer learning GB Gradient boosting  \nGDPFL Global differential private federated learning GDPR General data protection regulation  \nGMM Gaussian mixture model HFL Horizontal federated learning HMM Hidden Markov model  \nHR Hit rate  \nIoT Internet of things IR Improvement rate KNR K-nearest regression  \nLDPFL Local differential private federated learning LSTM Long short-term memory  \nMA Moving average  \nMAE Mean absolute error  \nMAPE Mean absolute percentage error  \nMFPCA Multivariate functional principal components analysis MLR Multiple linear regression  \nMSE Mean squared error  \nNILM Non-intrusive load monitoring OL Online learning  \nPA Passive-aggressive  \nPCA Principal component analysis PSP-Net Pyramid scene parsing network RBF Radial basis function  \nRBFN Radial basis function network  \nRNNs Recurrent neural networks SAE Signal aggregated error SBD Shape-based d","cbCaiaFCpGeZlbtP","https://ap.wps.com/l/cbCaiaFCpGeZlbtP","pdf",6657782,1,185,"English","en",105,"# 1 Introduction\n## 1.1 Context and Problem Statement\n## 1.2 Research Aims and Objectives\n## 1.3 Contributions\n## 1.4 Thesis Outline\n## 1.5 List of Publications\n# 2 Background and Literature Review\n## 2.1 Load Analysis\n## 2.2 Multi-level Load Analysis with Smart Meter Data","[{\"question\":\"What main challenges does smart meter big data introduce for machine learning frameworks?\",\"answer\":\"The thesis highlights big-data scale and privacy limitations, which make framework development harder than classic theoretical machine learning settings.\"},{\"question\":\"What are the three core parts of the proposed work?\",\"answer\":\"It includes (1) analysis and comparison of multi-level learning algorithms plus an activity recognition model and a consensus-based profiling/forecasting system, (2) an offline-to-online functional analysis model for real-time multi-level demand forecasting, and (3) a distributed approach using FederatedNILM combined with differential privacy.\"},{\"question\":\"How does the distributed privacy-preserving learning approach provide privacy guarantees?\",\"answer\":\"FederatedNILM is combined with differential privacy, and the framework is enhanced with utility optimization and privacy-preserving schemes for large-scale smart meter services.\"}]","A Distributed and Real-time Machine Learning Framework for Smart Meter Big Data | PDF",1785726018,466,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"a-distributed-and-real-time-machine-learning-framework-for-smart-meter-big-data","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/a-distributed-and-real-time-machine-learning-framework-for-smart-meter-big-data/119736/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What main challenges does smart meter big data introduce for machine learning frameworks?","Question",{"text":75,"@type":76},"The thesis highlights big-data scale and privacy limitations, which make framework development harder than classic theoretical machine learning settings.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What are the three core parts of the proposed work?",{"text":80,"@type":76},"It includes (1) analysis and comparison of multi-level learning algorithms plus an activity recognition model and a consensus-based profiling/forecasting system, (2) an offline-to-online functional analysis model for real-time multi-level demand forecasting, and (3) a distributed approach using FederatedNILM combined with differential privacy.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the distributed privacy-preserving learning approach provide privacy guarantees?",{"text":84,"@type":76},"FederatedNILM is combined with differential privacy, and the framework is enhanced with utility optimization and privacy-preserving schemes for large-scale smart meter services.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]