[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-273524-105":59,"doc-detail-273524-en":131},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":124,"head_meta":126,"extra_data":128,"updated_unix":130},105,"en","effects-of-answer-format-variation-on-gender-bias-in-large-language-models","Effects of Answer Format Variation on Gender Bias in Large Language Models","","Gender bias and related social biases in large language models are often assessed using question answering or survey benchmarks that require predefined response formats. Survey methodology shows that answer format can substantially change responses, yet the effect of answer-format changes on measuring gender bias in LLMs has received little systematic study. This work compares closed-ended, Likert-scaled, and open-ended formats on BBQ and OpinionQA, revealing large differences in measured bias and alignment, including ranking reversals. Results support treating answer format as a core evaluation component.",{"@graph":69,"@context":123},[70,84,106],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/effects-of-answer-format-variation-on-gender-bias-in-large-language-models/273524/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/effects-of-answer-format-variation-on-gender-bias-in-large-language-models/273524.png","ImageObject",300,407,{"name":92,"@type":93},"Hazel","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-10-27","2026-09-15",true,{"@type":102,"interactionType":103,"userInteractionCount":105},"InteractionCounter",{"@type":104},"ViewAction",11,{"@type":107,"mainEntity":108},"FAQPage",[109,115,119],{"name":110,"@type":111,"acceptedAnswer":112},"Why can answer format change measured gender bias in large language models?","Question",{"text":113,"@type":114},"Different answer formats elicit distinct response behaviours, such as forced-choice selection, Likert-scale distribution patterns, and refusal or variability in free-text generation. These behavioural shifts directly affect the resulting bias measurements.","Answer",{"name":116,"@type":111,"acceptedAnswer":117},"Which datasets and models are used to study answer-format effects?",{"text":118,"@type":114},"The study evaluates three instruction-tuned models on the BBQ benchmark and OpinionQA survey-derived data. It uses parallel versions of the same questions with systematically varied answer formats.",{"name":120,"@type":111,"acceptedAnswer":121},"What answer formats are compared in the evaluation?",{"text":122,"@type":114},"Closed-ended multiple-choice, Likert-scaled graded response formats, and open-ended free-text formats are compared. Together, they represent complementary evaluation settings and constrain responses differently.","https://schema.org",{"og:url":83,"og:type":125,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":127,"canonical":83},"index,follow",{"doc_id":129,"site_id":62},273524,1789471879,{"code":4,"msg":5,"data":132},{"doc_id":129,"user_id":133,"nickname":92,"user_avatar":134,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":105,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":140,"language":141,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":67,"update_tm":130,"read_time":145,"preview_page_count":4},137441390410,"https://ap-avatar.wpscdn.com/avatar/2000252f4ab5702993?_k=1776741390130283984","Effects of Answer Format Variation on Gender Bias in Large Language Models  \nKsenia Merzlyakova Sebastian Padó Franziska Weeber  \nInstitute for Natural Language Processing, University of Stuttgart { [ksenia.merzlyakova | pado | franziska.weeber}@ims.uni-stuttgart.de](ksenia.merzlyakova | pado | franziska.weeber}@ims.uni-stuttgart.de)  \narXiv : 2608 . 175 16v 1 [cs .CL] 18 Aug 2026  \nAbstract  \nGender bias or other social biases in large language models (LLMs) are frequently evaluated with question answering or survey benchmarks where the LLM needs to give a response in a predefined answer format. It is well known in survey science that the answer format has a substantial impact on answers, just as LLMs are sensitive to the prompt wording. However, to our knowledge it has not been studied yet how changes in answer format impact the measurement of gender bias in LLMs and their alignment with human response distributions.  \nWe evaluate three instruction-tuned models on the BBQ benchmark and OpinionQA survey data across closed-ended, Likert-scaled and open-ended formats, comparing bias measurement and distributional alignment under otherwise identical conditions. We find that answer format does substantially alter measured outcomes, including reversals in order rankings. These differences arise because each format elicits distinct response behaviours, such as forced-choice selection, scale-based distributions and refusal in free-text generation. Our findings highlight the importance of treating answer format as a substantive component of LLM evaluation and motivate multi-format designs for more robust model assessment.  \n1 Introduction  \nSocial biases in large language models (LLMs), such as gender bias, require careful evaluation by developers and researchers to avoid unfair and offensive model behaviour. One approach to evaluation is to present the model with example scenarios or questions and record its preference among provided answer options, treating the resulting choice as the measure of bias. However, previous metastudies on social bias showed that effect sizes for bias metrics are highly sensitive to phrasing and formatting changes that do not affect the semantics of evaluation examples, such as paraphrases,  \nnegations, answer order or punctuation differences (Hudson and Al Moubayed, 2021 ; Zhao et al., 2021 ; Alzahrani et al., 2024 ; Röttger et al., 2024) .  \nOne aspect that has not been evaluated yet in detail is the format of the answer options provided to the LLM: existing evaluations commonly rely on single-format assessments (e.g. Nangia et al., 2020 ; Nadeem et al., 2021 ; Parrish et al., 2022 ; Myrzakhan et al., 2024) . However, comparative work shows that closed-ended and open-ended formats can yield inconsistent and weakly correlated estimates (Jin et al., 2025), and that benchmark outcomes depend strongly on task formulation and measurement choices more broadly (GoldfarbTarrant et al., 2021 ; Ceron et al., 2024 ; Hida et al., 2025 ; Simpson et al., 2025) .  \nFrom a social science perspective, answer formats are even more consequential: it is well established that question format can substantially influence how humans respond (Kalton and Schuman, 1982 ; Tourangeau et al., 2000) . This may or may not transfer to LLMs, given that LLMs do not reliably replicate human response behaviour in a mechanistic sense, including in their handling of uncertainty and refusal behaviour (Tjuatja et al., 2024) . In either case, it remains problematic that much research builds on the results for single answer formats to draw inferences about models’ general underlying behaviour. The results for one format may rather reflect the specific constraints and response mechanisms induced by the chosen format, such that a different format could yield a substantially different estimate of the same bias construct.  \nThis study therefore investigates how answer format variation affects the measurement and manifestation of gender bias in LLMs","cbCaid68OMnUaW43","https://ap.wps.com/l/cbCaid68OMnUaW43","pdf",1495014,23,"English","# Introduction\n## Research questions and motivation\n## Evaluation approach and formats\n## Answer formats compared","[{\"question\":\"Why can answer format change measured gender bias in large language models?\",\"answer\":\"Different answer formats elicit distinct response behaviours, such as forced-choice selection, Likert-scale distribution patterns, and refusal or variability in free-text generation. These behavioural shifts directly affect the resulting bias measurements.\"},{\"question\":\"Which datasets and models are used to study answer-format effects?\",\"answer\":\"The study evaluates three instruction-tuned models on the BBQ benchmark and OpinionQA survey-derived data. It uses parallel versions of the same questions with systematically varied answer formats.\"},{\"question\":\"What answer formats are compared in the evaluation?\",\"answer\":\"Closed-ended multiple-choice, Likert-scaled graded response formats, and open-ended free-text formats are compared. Together, they represent complementary evaluation settings and constrain responses differently.\"}]","Effects of Answer Format Variation on Gender Bias in Large Language Models | PDF",58]