[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82457-en":3,"doc-seo-82457-105":30,"detail-sidebar-cat-0-en-105":95},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},82457,1099513958607,"Jiven","https://ap-avatar.wpscdn.com/avatar/100002390cf8733938c?x-image-process=image/resize,m_fixed,w_180,h_180&k=1778829742770036399",8,"Research & Report","Evaluating and Mitigating Errors in LLM-Generated Web API Integrations","Web API integrations are foundational to digital infrastructure, yet generating correct invocation code remains difficult—especially as APIs evolve. This work introduces WAPIIBench, a dataset and evaluation pipeline to measure how well large language models (LLMs) generate web API invocation code. Experiments on multiple open-source LLMs show frequent failures: hallucinated endpoints, improper argument usage, and low task success (none above 40%). The study further proposes constrained decoding via automatic translation from API specifications to constraints, boosting correctness on starter-code variants by about 90%–135%.","arXiv :2509 .20172v6 [ cs . SE] 18 Dec 2025  \nEvaluating and Mitigating Errors in LLM-Generated Web API Integrations  \nDANIEL MANINGER, Technische Universität Darmstadt, Germany and Hessian Center for Artificial Intelligence (hessian.AI), Germany  \nLEON CHEMNITZ, Pariton AI, Germany and Technische Universität Darmstadt, Germany AMIR MOLZAM SHARIFLOO, Technische Universität Darmstadt, Germany TUSHAR LAMBA, Technische Universität Darmstadt, Germany  \nJANNIS BRUGGER, Technische Universität Darmstadt, Germany and Hessian Center for Artificial Intelligence (hessian.AI), Germany  \nMIRA MEZINI, Technische Universität Darmstadt, Germany, Hessian Center for Artificial Intelligence (hessian.AI), Germany, and National Research Center for Applied Cybersecurity ATHENE, Germany  \nAPI integration is a cornerstone of our digital infrastructure, enabling software systems to connect and interact. However, as shown by many studies, writing or generating correct code to invoke APIs, particularly web APIs, is challenging. Although large language models (LLMs) have become popular in software development, their effectiveness in automating the generation of web API integration code remains unexplored. In order to address this, we present WAPIIBench, a dataset and evaluation pipeline designed to assess the ability of LLMs to generate web API invocation code. Our experiments with several open-source LLMs reveal that generating API invocations poses a significant challenge, resulting in hallucinated endpoints, incorrect argument usage, and other errors. None of the evaluated open-source models was able to solve more than 40% of the tasks. Motivated by those findings, we explore the potential of constrained decoding for generating API invocations. To this end, we propose an automatic translation from API specifications to constraints. Our approach prevents violations of API usage rules and significantly increases the overall correctness of the generated code, on average by 90% and 135%, depending on the provided starter code.  \nCCS Concepts: • Software and its engineering → Automatic programming; • Computing methodologies → Neural networks;  \n• Information systems → RESTful web services.  \nAdditional Key Words and Phrases: artificial intelligence, software engineering, large language models, code generation, web APIs, benchmarks, constrained decoding  \nACM Reference Format:  \nDaniel Maninger, Leon Chemnitz, Amir Molzam Sharifloo, Tushar Lamba, Jannis Brugger, and Mira Mezini. 2026. Evaluating and Mitigating Errors in LLM-Generated Web API Integrations. ACM Trans. Softw. Eng. Methodol. 37, 4, Article 111 (August 2026), 40 pages.  \n[https://doi](https://doi.org/XXXXXXX.XXXXXXX)[.](https://doi.org/XXXXXXX.XXXXXXX)[org/XXXXXXX](https://doi.org/XXXXXXX.XXXXXXX)[.](https://doi.org/XXXXXXX.XXXXXXX)[XXXXXXX](https://doi.org/XXXXXXX.XXXXXXX)  \nAuthors’ Contact Information: Daniel Maninger, Technische Universität Darmstadt, Darmstadt, Germany and Hessian Center for Artificial Intelligence (hessian.AI), Darmstadt, Germany, [daniel](daniel.maninger@tu-darmstadt.de)[.](daniel.maninger@tu-darmstadt.de)[maninger@tu-darmstadt](daniel.maninger@tu-darmstadt.de)[.](daniel.maninger@tu-darmstadt.de)[de](daniel.maninger@tu-darmstadt.de); Leon Chemnitz, ParitonAI, Berlin, Germany and Technische Universität Darmstadt, Darmstadt, Germany, [leon](leon.chemnitz@pariton.ai)[.](leon.chemnitz@pariton.ai)[chemnitz@pariton](leon.chemnitz@pariton.ai)[.](leon.chemnitz@pariton.ai)[ai](leon.chemnitz@pariton.ai); Amir Molzam Sharifloo, Technische Universität Darmstadt, Darmstadt, Germany, amir.molzam@tu[darmstadt](darmstadt.de)[.](darmstadt.de)[de](darmstadt.de); Tushar Lamba, Technische Universität Darmstadt, Darmstadt, Germany, [tushar](tushar.lamba@stud.tu-darmstadt.de)[.](tushar.lamba@stud.tu-darmstadt.de)[lamba@stud](tushar.lamba@stud.tu-darmstadt.de)[.](tushar.lamba@stud.tu-darmstadt.de)[tu-darmstadt](tushar.lamba@stud.tu-darmstadt.de)[.](tushar.lamba@stud.tu-darmstadt.de)[de](tushar.","cbCaiqmHiVuKY67r","https://ap.wps.com/l/cbCaiqmHiVuKY67r","pdf",989942,4,1,40,"English","en",105,"# Introduction\n## Background and Motivation\n## WAPIIBench Benchmarking Framework\n## Error Analysis in LLM-Generated Integrations\n## Constrained Decoding for Error Mitigation","[{\"question\":\"What challenge does the document address in LLM-based web API integrations?\",\"answer\":\"Correctly generating web API invocation code is difficult for LLMs, leading to hallucinated endpoints, incorrect arguments, and other integration errors.\"},{\"question\":\"What is WAPIIBench used for?\",\"answer\":\"WAPIIBench is a dataset and evaluation pipeline that assesses LLMs’ ability to generate web API invocation code.\"},{\"question\":\"How well do the evaluated open-source LLMs perform on the benchmark?\",\"answer\":\"No evaluated open-source model solves more than 40% of the tasks.\"},{\"question\":\"How does the proposed constrained decoding method improve correctness?\",\"answer\":\"It translates API specifications into constraints and prevents violations of API usage rules, increasing overall generated-code correctness by roughly 90% to 135% depending on the provided starter code.\"}]",1784180558,101,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":90,"head_meta":92,"extra_data":94,"updated_unix":28},"evaluating-and-mitigating-errors-in-llm-generated-web-api-integrations","",{"@graph":36,"@context":89},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":20},"https://docshare.wps.com/document/evaluating-and-mitigating-errors-in-llm-generated-web-api-integrations/82457/",{"url":52,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-23","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81,85],{"name":72,"@type":73,"acceptedAnswer":74},"What challenge does the document address in LLM-based web API integrations?","Question",{"text":75,"@type":76},"Correctly generating web API invocation code is difficult for LLMs, leading to hallucinated endpoints, incorrect arguments, and other integration errors.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is WAPIIBench used for?",{"text":80,"@type":76},"WAPIIBench is a dataset and evaluation pipeline that assesses LLMs’ ability to generate web API invocation code.",{"name":82,"@type":73,"acceptedAnswer":83},"How well do the evaluated open-source LLMs perform on the benchmark?",{"text":84,"@type":76},"No evaluated open-source model solves more than 40% of the tasks.",{"name":86,"@type":73,"acceptedAnswer":87},"How does the proposed constrained decoding method improve correctness?",{"text":88,"@type":76},"It translates API specifications into constraints and prevents violations of API usage rules, increasing overall generated-code correctness by roughly 90% to 135% depending on the provided starter code.","https://schema.org",{"og:url":52,"og:type":91,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":93,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":96},[97,101,105,109,114,119,123,126,131,134,138],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Literature",80,"literature",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":106,"show_sort_weight":107,"slug":108},"Exam",70,"exam",{"id":110,"doc_module":4,"doc_module_name":46,"category_name":111,"show_sort_weight":112,"slug":113},5,"Comic",60,"comic",{"id":115,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},6,"Technology",50,"technology",{"id":120,"doc_module":4,"doc_module_name":46,"category_name":121,"show_sort_weight":22,"slug":122},7,"Healthcare","healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":124,"slug":125},30,"research-report",{"id":127,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":129,"slug":130},9,"Religion & Spirituality",20,"religion-spirituality",{"id":129,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":129,"slug":133},"World Cup","world-cup",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":135,"slug":137},10,"Lifestyle","lifestyle",{"id":139,"doc_module":4,"doc_module_name":46,"category_name":140,"show_sort_weight":110,"slug":141},19,"General","general"]