[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85748-en":3,"doc-seo-85748-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85748,5909877438554,"Maeve","https://ap-avatar.wpscdn.com/avatar/5600025385ad2bf12a7?_k=1778553567797529272",8,"Research & Report","Nonparametric Bayesian Inverse Reinforcement Learning with Data-Parallel Gibbs Sampling","Inverse Reinforcement Learning infers reward functions from expert demonstrations, but pooling data from multiple experts yields parametric models that recover only an average reward that fits none. This work presents nonparametric Bayesian IRL with a Dirichlet Process prior, jointly inferring the number of latent reward types and their rewards. Collapsed Gibbs inference combines Chinese Restaurant Process updates for clusters with Metropolis-Hastings updates for reward weights plus soft value iteration planning. Experiments on ObjectWorld evaluate K=2 and K=3 using ARI metrics and analyze an ObjectWorld placement requirement for reliable recovery. A data-parallel Ray-based implementation achieves up to 4.79× speedup while exposing throughput–accuracy tradeoffs from consensus merging; code and a containerized environment support replication.","Nonparametric Bayesian Inverse Reinforcement  \nLearning with Data-Parallel Gibbs Sampling  \nSai Anirudh Katupilla  \nUniversity of Maryland, College Park[anir@umd.edu](anir@umd.edu)  \nShreeya Dasa Lakshminath  \nUniversity of Maryland, College Park[slakshm2@umd.edu](slakshm2@umd.edu)  \narXiv :2607 .09886v 1 [ cs .LG] 10 Jul 2026  \nAbstract—Inverse Reinforcement Learning recovers reward functions from expert demonstrations, but standard formulations assume that all demonstrations come from a single expert. When demonstrations are pooled from multiple experts with distinct preferences, parametric methods recover an averaged reward that fits no individual expert well. We implement Nonparametric Bayesian Inverse Reinforcement Learning with a Dirichlet Process prior over reward functions, allowing the number of latent reward types to be inferred jointly with the rewards themselves. Inference uses a collapsed Gibbs sampler combining a Chinese Restaurant Process update for cluster assignments with a Metropolis-Hastings update for reward weights, and soft value iteration as the inner planning routine. We evaluate on a 10 × 10 ObjectWorld grid with two and three ground-truth reward types. The serial sampler recovers K = 2 with Adjusted Rand Index of 1.000, substantially outperforming a Maximum Entropy IRL baseline (ARI=0.000). Extension to K = 3 shows that the sampler correctly identifies the number of clusters in all runs; assignment ARI of 0.48–0.58 reflects behavioral overlap between expert types that persists across grid instantiations, revealing that reliable K=3 evaluation on ObjectWorld requires controlled object placement rather than random seeding. We further parallelize the sampler across CPU cores using Rayon HPC hardware, achieving a peak speedup of 4.79 × at 8 workers, and characterize a throughput-versus-accuracy tradeoff arising from the consensus merge heuristic used during state aggregation. Code and a containerized environment are available at [https://github.com/dasashreeya/np](https://github.com/dasashreeya/np) bayes irl.  \nIndex Terms—Inverse Reinforcement Learning, Bayesian Nonparametrics, Dirichlet Process, Gibbs Sampling, Parallel MCMC, JAX, Ray  \nI. INTRODUCTION  \nReinforcement learning algorithms learn control policies from a reward function, but in many practical domains the reward function itself is unknown and must be inferred from observed expert behavior. Inverse Reinforcement Learning (IRL) addresses this inverse problem: given a Markov Decision Process and a set of expert trajectories, recover the reward function that best explains the observed actions. IRL has applications in robotics, autonomous driving, medical decision support, and human-AI interaction, where engineering an explicit reward function is impractical but expert demonstrations are available [1] .  \nMost IRL formulations encounter two fundamental difficulties. The modeling problem: real demonstration datasets rarely come from a single homogeneous expert. Fitting one reward  \nto a mixed pool produces an average that represents nobody. Parametric Bayesian IRL [2] provides principled uncertainty over the reward but still requires the number of reward types to be fixed in advance—an assumption that is unreasonable when the goal is to discover latent structure. A Dirichlet Process prior removes this requirement, letting the number of components grow as the data demands. The computational problem: MCMC inference over reward posteriors is expensive because each sample requires evaluating trajectory likelihood via inner planning. For a nonparametric model the inference cost scales with both trajectory count and cluster count, making parallelization essential for practical use.  \nWe address both problems in a single system. Our contributions are:  \n• A complete open-source implementation of nonparametric Bayesian IRL using a collapsed Gibbs sampler with Chinese Restaurant Process cluster updatesand Metropolis-Hastings weight updates, verified agains","cbCaifFaIkzDdZ9f","https://ap.wps.com/l/cbCaifFaIkzDdZ9f","pdf",819013,1,6,"English","en",105,"# Introduction\n# Literature Survey\n## Inverse Reinforcement Learning\n## Bayesian and Nonparametric IRL\n## Dirichlet Process Mixture Models","[{\"question\":\"为什么传统参数化IRL在多专家演示数据上表现不佳？\",\"answer\":\"当演示来自多个偏好不同的专家时，参数化方法通常会拟合一个“平均奖励”，该奖励难以同时符合任何单一专家的行为，因此恢复质量会受限。\"},{\"question\":\"文中如何用Dirichlet Process避免预先固定奖励类型数量？\",\"answer\":\"通过对奖励函数设置Dirichlet Process先验，模型允许潜在的奖励成分数量随数据自动增长，从而与未知的潜在结构规模相匹配。\"},{\"question\":\"如何实现并行推断以及并行带来什么取舍？\",\"answer\":\"作者在CPU核心上并行化collapsed Gibbs采样流程，采用Ray进行数据并行，并指出由于状态聚合中的consensus merge启发式，会出现吞吐量与准确率之间的权衡。\"}]",1784205999,15,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"nonparametric-bayesian-inverse-reinforcement-learning-with-data-parallel-gibbs-sampling","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/nonparametric-bayesian-inverse-reinforcement-learning-with-data-parallel-gibbs-sampling/85748/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"为什么传统参数化IRL在多专家演示数据上表现不佳？","Question",{"text":75,"@type":76},"当演示来自多个偏好不同的专家时，参数化方法通常会拟合一个“平均奖励”，该奖励难以同时符合任何单一专家的行为，因此恢复质量会受限。","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"文中如何用Dirichlet Process避免预先固定奖励类型数量？",{"text":80,"@type":76},"通过对奖励函数设置Dirichlet Process先验，模型允许潜在的奖励成分数量随数据自动增长，从而与未知的潜在结构规模相匹配。",{"name":82,"@type":73,"acceptedAnswer":83},"如何实现并行推断以及并行带来什么取舍？",{"text":84,"@type":76},"作者在CPU核心上并行化collapsed Gibbs采样流程，采用Ray进行数据并行，并指出由于状态聚合中的consensus merge启发式，会出现吞吐量与准确率之间的权衡。","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]