[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84156-en":3,"doc-seo-84156-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84156,2336464648746,"Skyler","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Ad Headline Generation using Self-Critical Masked Language Model","Programmatic generation of e-commerce advertisement headlines is difficult to scale while maintaining creative quality. The document proposes a reinforcement learning (RL) policy-gradient training approach applied to Transformer-based masked language models to generate product advertising headlines. Headlines are produced by jointly conditioning on multiple products within a single campaign, enabling generalization across shared attributes. Experiments report improvements over Transformer and LSTM+RL baselines using overlap metrics and quality/grammar audits, and outperform human-submitted headlines in creative quality.","Ad Headline Generation using Self-Critical Masked Language Model  \nYashal Shakti Kanungo  \n[yashalk@amazon.com](yashalk@amazon.com)  \nSumit Negi  \n[suminegi@amazon.com](suminegi@amazon.com)  \nAruna Rajan  \n[rajarna@amazon.com](rajarna@amazon.com)  \narXiv :2607 .068 18v 1 [ cs .CL] 7 Jul 2026  \nAbstract  \nFor any E-commerce website it is a nontrivial problem to build enduring advertisements that attract shoppers. It is hard to pass the creative quality bar of the website, especially at a large scale. We thus propose a programmatic solution to generate product advertising headlines using retail content. We propose a state of the art application of Reinforcement Learning (RL) Policy gradient methods on Transformer (Vaswani et al., 2017) based Masked Language Models (Devlin et al., 2019) . Our method creates the advertising headline by jointly conditioning on multiple products that a seller wishes to advertise. We demonstrate that our method outperforms existing Transformer and LSTM + RL methods in overlap metrics and quality audits. We also show that our model-generated headlines outperform human submitted headlines in terms of both grammar and creative quality as determined by audits.  \n1 Introduction  \nThere are a various types of ads. A set of example ads that showcase products selected by sellers along with headlines that advertise them are shown in Figure 1. Sellers create multiple ad campaigns for multiple products, bid in an auction to advertise and pay for clicks on the ad.  \nAn E-Commerce product catalog may have millions of products which can be advertised. To ease the ad headline writing process, humans resort to programmatically padding keywords, or repasting the retail catalog content in the advertisement.  \nTemplated creatives such as “Save Now on ...\" or“Buy more (product) of (brand)\" save the creative effort but fail to create any excitement or brand identity in the minds of shoppers. High quality headlines are more attractive to shoppers and offer better value proposition. In this paper, we describe how we built a Natural Language Generation (NLG) system to generate instantaneous, attractive  \nand brand identity building headlines for advertisements that intend to promote a wide range of products offered by a brand.  \nThe content associated with a retail product has challenging characteristics. Some product titles have poor structure, grammatical issues, or partial phrases. The product titles also include varying number of product features such as “Hyper Tough 18V Cordless Drill, 3/8 inch Chuck, Variable Speed, with 1.2Ah Nickel Cadmium Battery, Charger, Bit Holder LED Light\" along with titles such as “ZIPIT Grillz Backpack, Camo Grey\".  \nThe generated headlines need to capture the information present in the retail attributes and at the same time be different and uniquely attractive. Advertisers select multiple related products that are advertised as part of a single ad campaign. The ad campaign headline is then shared across all of these related products. Thus, the headline also needs to generalize the shared characteristics of the products and cannot be specific to a single product within the campaign.  \nThe key contributions of our work are:  \n• We use Masked Language Model (MLM) for the generation of advertisement headlines using multiple products at the same time. Extensive test-set metrics, quality and grammar audits show that the proposed model outperforms all the baselines and the humansubmitted headlines in terms of quality and grammar.  \n• The novel usage of RL for the training of MLM allows us to directly optimize the MLM for improved headline quality metrics without changing inference setup or latency. Our method can also be applied to any other NLG task such as summarization, translation etc.  \n• Our model reduces the extensive effort and time that is required to manually create headlines and has low latency.  \nFigure 1: Examples of different product ads from multiple websites across the internet. A ","cbCainJiX4nb3Gc3","https://ap.wps.com/l/cbCainJiX4nb3Gc3","pdf",1440665,1,9,"English","en",105,"# Introduction\n# Related Work","[{\"question\":\"How does the proposed method generate ad headlines for e-commerce campaigns?\",\"answer\":\"It generates headlines using a Transformer-based masked language model jointly conditioned on multiple products that a seller wants to advertise in a single campaign.\"},{\"question\":\"What training strategy is used to improve headline quality?\",\"answer\":\"The method uses reinforcement learning with self-critical policy gradient to directly optimize masked language model objectives for improved headline quality metrics without changing inference setup or latency.\"},{\"question\":\"How does the model perform compared with existing baselines and human-written headlines?\",\"answer\":\"Results show better overlap metrics and quality audits than existing Transformer and LSTM+RL methods, and audits indicate the generated headlines outperform human-submitted headlines in both grammar and creative quality.\"}]",1784193507,23,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"ad-headline-generation-using-self-critical-masked-language-model","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/ad-headline-generation-using-self-critical-masked-language-model/84156/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"How does the proposed method generate ad headlines for e-commerce campaigns?","Question",{"text":75,"@type":76},"It generates headlines using a Transformer-based masked language model jointly conditioned on multiple products that a seller wants to advertise in a single campaign.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What training strategy is used to improve headline quality?",{"text":80,"@type":76},"The method uses reinforcement learning with self-critical policy gradient to directly optimize masked language model objectives for improved headline quality metrics without changing inference setup or latency.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the model perform compared with existing baselines and human-written headlines?",{"text":84,"@type":76},"Results show better overlap metrics and quality audits than existing Transformer and LSTM+RL methods, and audits indicate the generated headlines outperform human-submitted headlines in both grammar and creative quality.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]