[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-81634-en":3,"doc-seo-81634-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},81634,8796095360427,"Lucas Martin","https://ap-avatar.wpscdn.com/davatar_994ba38a5ba835b3df7d355c54d3ed8d",8,"Research & Report","Rectification Difficulty and Optimal Sample Allocation in LLM-Augmented Surveys","Large Language Models can generate synthetic survey responses cheaply, yet their accuracy varies unpredictably across questions. The work studies how to allocate a fixed budget of human respondents among estimation tasks when LLM predictions are available for each task. It introduces rectification difficulty to characterize question-specific residual uncertainty, derives a closed-form optimal allocation rule that favors tasks where the LLM is least reliable, and proposes meta-learning to predict rectification difficulty for new tasks without pilot human data. The framework extends to general M-estimation and is validated on two datasets, delivering 61–79% of achievable efficiency gains with 11.4% and 10.5% MSE reductions.","Rectification Difficulty and Optimal Sample Allocation in  \nLLM-Augmented Surveys  \nZikun Ye*  \nUniversity of Washington  \nHema Yoganarasimhan University of Washington  \narXiv :2604 . 17267v2 [ cs .AI] 9 Jul 2026  \nThis version: July 9, 2026  \nAbstract  \nLarge Language Models can generate synthetic survey responses at low cost, but their accuracy varies unpredictably across questions. We study the design problem of allocating a fixed budget of human respondents across estimation tasks when cheap LLM predictions are available for every task. Our framework combines three components. First, building on Prediction-Powered Inference, we characterize a question-specific rectification difficulty that governs how quickly the estimator’s variance decreases with human sample size. Second, we derive a closed-form optimal allocation rule that directs more human labels to tasks where the LLM is least reliable. Third, since rectification difficulty depends on unobserved human responses for new surveys, we propose a meta-learning approach, trained on historical data, that predicts it for entirely new tasks without pilot data. The framework extends to general M-estimation, covering regression coefficients and multinomial logit partworths for conjoint analysis. We validate the framework on two datasets spanning different domains, question types, and LLMs, showing that our approach captures 61–79% of the theoretically attainable efficiency gains, achieving 11.4% and 10.5% MSE reductions without requiring any pilot human data for the target survey.  \nKeywords: Market Research, Survey Design, Large Language Models, Prediction-Powered Inference.  \n*We thank Olivier Toubia and the participants of the UW–UBC Marketing Conference for their helpful comments and feedback. Please address all correspondence to: zikunye@uw.edu and hemay@uw.edu.  \n1 Introduction  \n1.1 The promise and challenge of LLMs as synthetic respondents  \nSurveys play an important role in market research, from brand tracking and conjoint analysis to willingnessto-pay estimation. However, fielding them is expensive. This cost pressure has led both researchers and firms to explore whether Large Language Models (LLMs) can serve as low-cost synthetic respondents. Industry interest is growing rapidly: synthetic-research startups have attracted major funding, market-research firms are developing digital-twin panels, and many firms are investing in tools that simulate consumer behavior, forecast customer reactions, and substitute for conventional surveys (Index Ventures, 2026 ; Ipsos, 2025b,a; Ipsos Digital, 2026 ; Accenture, 2025 ; Maier et al., 2025) . Academic research has developed in parallel, showing that LLMs can generate plausible survey responses at negligible marginal cost and, in some settings, replicate aggregate human patterns across marketing and social-science tasks (e.g., Brand et al., 2023 ; Argyle et al., 2023 ; Wang et al., 2024) .  \nAt the same time, LLM responses exhibit systematic biases, insufficient within-population heterogeneity, and sensitivity to topics, populations, prompts, and question formats (Bisbee et al., 2024 ; Anthis et al., 2025 ; Brucks and Toubia, 2025) . In some settings, LLMs approximate aggregate human responses well; in others, they produce biased or near-constant answers that fail to capture respondent-level variation (Motoki et al., 2024 ; Peng et al., 2025) . The central empirical fact is therefore heterogeneity: LLM predictions are useful for some questions and nearly useless for others, and this variation is difficult to anticipate ex ante (Toubia et al., 2025) . LLMs should therefore be viewed not as wholesale substitutes for human respondents, but as unevenly informative auxiliary signals. Thus, the relevant managerial question is how to use those signals to determine where scarce human responses are most valuable.  \nIn a multi-question survey, this becomes a pre-fielding budget-allocation problem. A firm can cheaply generate LLM prediction","cbCaij7NBU1QTCLW","https://ap.wps.com/l/cbCaij7NBU1QTCLW","pdf",682768,4,1,57,"English","en",105,"# Introduction\n## The promise and challenge of LLMs as synthetic respondents\n## An efficient design framework for LLM-augmented surveys","[{\"question\":\"What is the key problem addressed in LLM-augmented survey design?\",\"answer\":\"How to allocate a fixed number of human respondents across multiple survey questions when LLM predictions are available but vary in usefulness and accuracy by question.\"},{\"question\":\"What does rectification difficulty mean in this framework?\",\"answer\":\"A question-specific residual uncertainty that remains after optimally using the LLM signal, determining how quickly estimator variance decreases as human sample size increases.\"},{\"question\":\"How does the method handle new tasks without pilot human data?\",\"answer\":\"It uses meta-learning trained on historical survey data to predict rectification difficulty for entirely new tasks, enabling optimal allocation without collecting pilot responses.\"}]",1784175007,144,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"rectification-difficulty-and-optimal-sample-allocation-in-llm-augmented-surveys","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":20},"https://docshare.wps.com/document/rectification-difficulty-and-optimal-sample-allocation-in-llm-augmented-surveys/81634/",{"url":52,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-25","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is the key problem addressed in LLM-augmented survey design?","Question",{"text":75,"@type":76},"How to allocate a fixed number of human respondents across multiple survey questions when LLM predictions are available but vary in usefulness and accuracy by question.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What does rectification difficulty mean in this framework?",{"text":80,"@type":76},"A question-specific residual uncertainty that remains after optimally using the LLM signal, determining how quickly estimator variance decreases as human sample size increases.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the method handle new tasks without pilot human data?",{"text":84,"@type":76},"It uses meta-learning trained on historical survey data to predict rectification difficulty for entirely new tasks, enabling optimal allocation without collecting pilot responses.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]