TSI: an Ad Text Strength Indicator using Text-to-CTR and Semantic-Ad-Similarity - Paper

Sep 27, 2026

Ad text generation is time-consuming and difficult for small businesses with limited advertising expertise. To support onboarding with low-quality ads, an Ad Text Strength Indicator (TSI) is proposed that (i) predicts click-through-rate from input ad text, (ii) retrieves semantically similar existing ads to form a neighborhood, and (iii) compares predicted CTRs to judge whether the input ad is weak or strong. TSI also suggests anonymized stronger ads. A BERT text-to-CTR model and a sentence-BERT semantic similarity model achieve strong offline prediction and retrieval results, with sub-second online latency in Yahoo’s ad platform.

LanguageEnglish

CategoryTechnology

FormatPDF

Pages10

Reading Time25 min

AccessFree to download

Introduction

Core idea of TSI

Two major components: text-to-CTR and similar ads retrieval

TSI: an Ad Text Strength Indicator using Text-to-CTR and Semantic-Ad-Similarity Shaunak Mishra∗ Yahoo Research USA [shaunakm@verizonmedia.com](shaunakm@verizonmedia.com) Changwei Hu∗ Yahoo Research USA [changweih@verizonmedia.com](changweih@verizonmedia.com) Manisha Verma Yahoo Research USA [manishav@verizonmedia.com](manishav@verizonmedia.com) arXiv :2108 .08226v1 [ cs .CL] 18 Aug 2021 Kevin Yen Yahoo Research USA [kevinyen@verizonmedia.com](kevinyen@verizonmedia.com) Yifan Hu Yahoo Research USA [yifanhu@verizonmedia.com](yifanhu@verizonmedia.com) Maxim Sviridenko Yahoo Research USA [sviri@verizonmedia.com](sviri@verizonmedia.com) ABSTRACT Coming up with effective ad text is a time consuming process, and particularly challenging for small businesses with limited advertising experience. When an inexperienced advertiser onboards with a poorly written ad text, the ad platform has the opportunity to detect low performing ad text, and provide improvement suggestions. To realize this opportunity, we propose an ad text strength indicator (TSI) which: (i) predicts the click-through-rate (CTR) for an input ad text,(ii) fetches similar existing ads to create a neighborhood around the input ad,(iii) and compares the predicted CTRs in the neighborhood to declare whether the input ad is strong or weak. In addition, as suggestions for ad text improvement, TSI shows anonymized versions of superior ads (higher predicted CTR) in the neighborhood. For (i), we propose a BERT based text-to-CTR model trained on impressions and clicks associated with an ad text. For (ii), we propose a sentence-BERT based semantic-ad-similarity model trained using weak labels from ad campaign setup data. Offline experiments demonstrate that our BERT based text-to-CTR model achieves a significant lift in CTR prediction AUC for cold start (new) advertisers compared to bag-of-words based baselines. In addition, our semantic-textual-similarity model for similar ads retrieval achieves a precision@1 of 0.93 (for retrieving ads from the same product category); this is significantly higher compared to unsupervised TF-IDF, word2vec, and sentence-BERT baselines. Finally, we share promising online results from advertisers in the Yahoo (Verizon Media) ad platform where a variant of TSI was implemented with sub-second end-to-end latency. CCS CONCEPTS • Information systems → Online advertising. ∗ Equal contribution. Permission to make digital or hard copies of part or all ofthis work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s) . CIKM’21, November 1–5, 2021, Virtual Event, QLD, Australia © 2021 Copyright held by the owner/author(s) . ACM ISBN 978-1-4503-8446-9/21/11 . [https://doi.org/10.1145/3459637.3481957](https://doi.org/10.1145/3459637.3481957) KEYWORDS Online advertising; CTR prediction; semantic textual similarity; multilingual; ad creative; ad text optimization. ACM Reference Format: Shaunak Mishra, Changwei Hu, Manisha Verma, Kevin Yen, Yifan Hu, and Maxim Sviridenko. 2021. TSI: an Ad Text Strength Indicator using Text-to-CTR and Semantic-Ad-Similarity. In Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM’21), November 1–5, 2021, Virtual Event, QLD, Australia. ACM, New York, NY, USA, 10 pages. [https://doi.org/10.1145/3459637.3481957](https://doi.org/10.1145/3459637.3481957) 1 INTRODUCTION Effective ad text can go a long way in attracting online users, and nudging them towards a purchase. However, writing effective ad text is an inherently creative process, and may require years of experience. This can be challenging for small businesses, who have limited advertising budgets and may n

FAQs

How does TSI find similar ads and generate improvement suggestions?

TSI uses a sentence-BERT semantic-ad-similarity model to retrieve similar ads, then compares predicted CTRs in the neighborhood; it also provides anonymized versions of superior ads as inspiration for better ad writing.

Sep 27, 2026

Upload by Nguyễn Văn Học