[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82772-en":3,"doc-seo-82772-105":29,"detail-sidebar-cat-0-en-105":83},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82772,137441390410,"Hazel","https://ap-avatar.wpscdn.com/avatar/2000252f4ab5702993?_k=1776741390130283984",8,"Research & Report","Enhancement of E-commerce Sponsored Search Relevancy with LLM","Sponsored search is a key revenue channel for search engines, yet matching advertisers’ bid keywords to users’ evolving queries is difficult due to large keyword spaces, ambiguous intent, and diverse topics and languages. For e-commerce, accurate sponsored search results directly affect user satisfaction and operational efficiency. This work builds an Ad Relevance Model using a pretrained large language model, adapting LLAMA2 7B via LoRA and introducing a \u003Cquery, ad title> classifier to label interactions across three relevance levels. Fine-tuning achieves 89.43% accuracy, improving precision, cost efficiency, and privacy for large marketplaces.","Enhancement of E-commerce Sponsored Search Relevancy with LLM  \nMd Omar Faruk Rokon1, * , Andrei Simion1 , Weizhi Du1 , Musen Wen1 , Hong Yao1 and Kuang-chih Lee 1  \n1 Walmart AdTech, Sunnyvale, CA, USA  \nAbstract  \nSponsored search plays a crucial role as a revenue stream for search engines, wherein advertisers competitively bid on keywords that align with the users’ search queries. The task of matching relevant keywords to these queries is complicated by the vast and ever-evolving space of keywords, the ambiguity of user and advertiser intentions, and the wide range of topics and languages involved. Consequently, ensuring that ads are pertinent to user queries presents significant challenges. In the fast-paced world of e-commerce, the accuracy of sponsored search results is vital for boosting user satisfaction and optimizing business operations. This paper presents the development of an advanced Ad Relevance Model within a sponsored search framework, utilizing the power of a pretrained large language model. We detail a pioneering adaptation of the LLAMA2 7B model through Low-Rank Adaptation (LoRA), which markedly enhances search precision and operational efficiency, thus opening new avenues for improving user interactions in extensive online marketplaces [such as Walmart.com. We introduce a novel](such as Walmart.com. We introduce a novel)[ ](such as Walmart.com. We introduce a novel)\u003Cquery, ad title> classifier, which discerns the relevance of search interactions across three categories: Relevant, Partially Relevant, and Irrelevant. Our approach involved adapting the pretrained model specifically for the e-commerce sponsored search context, training it on a large dataset. The fine-tuned model demonstrated a marked improvement in ad relevance accuracy, achieving 89.43% accuracy on a comprehensive test dataset—outperforming both the baseline model and other advanced language models like GPT-4 . The integration of LoRA with the based model represents a significant stride in customizing language models for e-commerce applications, resulting in enhanced search accuracy, cost efficiency, and operational privacy—a triad essential for the modern digital marketplace.  \nKeywords  \nLLM, LLaMa, LoRA, Relevance, Sponsored Search  \n1. Introduction  \nThe advent of e-commerce has revolutionized the retail landscape, creating a pressing need for advanced technological solutions to improve the user experience. A pivotal aspect of this experience is the relevance of sponsored product searches—a factor that directly influences customer satisfaction and retention[1, 2, 3] . Traditionally, search relevance in e-commerce platforms has been tackled using various algorithmic approaches, but these often fall short in understanding the nuanced language of consumer queries [4, 5, 6] .  \neCom’24: ACM SIGIR Workshop on eCommerce, July 18, 2024, Washington, DC, USA  \n* Corresponding author.  \n$ [mdomarfaruk.rokon@walmart.com](mdomarfaruk.rokon@walmart.com) (M. O. F. Rokon); [andrei.simion@walmart.com](andrei.simion@walmart.com) (A. Simion);  \n[weizhi.du@walmart.com](weizhi.du@walmart.com) (W. Du); [musen.wen@walmart.com](musen.wen@walmart.com) (M. Wen); [hong.yao0@walmart.com](hong.yao0@walmart.com) (H. Yao);  \n[kuang-chih.lee@walmart.com](kuang-chih.lee@walmart.com) (K. Lee)  \n © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0) .  \nRecent advances in Natural Language Processing (NLP), particularly the advent of large language models (LLMs), have brought a revolution in solving complex information retrieval problems. These models have significantly improved our ability to interpret and respond to the intricate nuances of human language, presenting new opportunities for enhancing search relevance. Despite this progress, there remains a significant gap in the application of these models within the unique constraints of e-commerce search environments, where the interpretatio","cbCaiqH7waoDX46S","https://ap.wps.com/l/cbCaiqH7waoDX46S","pdf",1092120,1,16,"English","en",105,"# Introduction\n## Problem motivation and challenges\n## Proposed approach with LLAMA2 7B and LoRA\n## Query–ad title relevance classification scheme","[{\"question\":\"What performance improvement is reported for the fine-tuned model?\",\"answer\":\"The fine-tuned LLAMA2 7B model reaches 89.43% accuracy on a comprehensive test dataset, outperforming a baseline and other advanced language models such as GPT-4.\"}]",1784182839,40,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":78,"head_meta":80,"extra_data":82,"updated_unix":27},"enhancement-of-e-commerce-sponsored-search-relevancy-with-llm","",{"@graph":35,"@context":77},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/enhancement-of-e-commerce-sponsored-search-relevancy-with-llm/82772/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71],{"name":72,"@type":73,"acceptedAnswer":74},"What performance improvement is reported for the fine-tuned model?","Question",{"text":75,"@type":76},"The fine-tuned LLAMA2 7B model reaches 89.43% accuracy on a comprehensive test dataset, outperforming a baseline and other advanced language models such as GPT-4.","Answer","https://schema.org",{"og:url":51,"og:type":79,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":81,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":84},[85,89,93,97,102,107,111,114,119,122,126],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":86,"show_sort_weight":87,"slug":88},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":90,"show_sort_weight":91,"slug":92},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Exam",70,"exam",{"id":98,"doc_module":4,"doc_module_name":45,"category_name":99,"show_sort_weight":100,"slug":101},5,"Comic",60,"comic",{"id":103,"doc_module":4,"doc_module_name":45,"category_name":104,"show_sort_weight":105,"slug":106},6,"Technology",50,"technology",{"id":108,"doc_module":4,"doc_module_name":45,"category_name":109,"show_sort_weight":28,"slug":110},7,"Healthcare","healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":112,"slug":113},30,"research-report",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},9,"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":45,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":98,"slug":129},19,"General","general"]