[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-157370-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-157370-en":131},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","sketchrl-finetuning-generative-sketch-models-with-visual-rewards","SketchRL - Finetuning Generative Sketch Models with Visual Rewards","","SketchRL fine-tunes a pretrained SketchRNN decoder for vector sketch generation by using reinforcement learning with image-based visual rewards. The approach tackles the gap between stroke-level imitation and semantic recognizability of the final rendered sketch, which standard SketchRNN optimizes only indirectly. Three reward functions are compared: SSIM to a target image, sparse CLIP at episode end, and dense CLIP computed at intermediate stroke checkpoints. Dense CLIP yields the strongest semantic gains while better preserving diversity, validated with CLIP metrics, FID, and an independent classifier.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/sketchrl-finetuning-generative-sketch-models-with-visual-rewards/157370/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/sketchrl-finetuning-generative-sketch-models-with-visual-rewards/157370.png","ImageObject",300,407,{"name":42,"@type":43},"Emma Mercer","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-27","2026-08-29",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",12,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What problem does SketchRL address compared with standard SketchRNN training?","Question",{"text":63,"@type":64},"Standard SketchRNN trains on stroke-level supervision and lacks an objective that directly optimizes whether the final rendered sketch is recognizable. SketchRL adds reinforcement learning with visual rewards to improve semantic quality.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"Which reward functions are compared in the paper?",{"text":68,"@type":64},"The paper compares SSIM reward against a target image, sparse CLIP reward computed only at the end of the episode, and dense CLIP reward computed at intermediate stroke checkpoints with discounted returns.",{"name":70,"@type":61,"acceptedAnswer":71},"What are the main findings from the evaluation?",{"text":72,"@type":64},"Dense CLIP reward achieves the highest CLIP score and improves classifier-based recognition while better preserving diversity than sparse reward. All RL methods increase FID, indicating a tradeoff between CLIP-optimized semantic style and similarity to the original QuickDraw distribution.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},157370,1787979890,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,127],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":25,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":128,"doc_module":4,"doc_module_name":25,"category_name":129,"show_sort_weight":97,"slug":130},19,"General","general",{"code":4,"msg":82,"data":132},{"doc_id":79,"user_id":133,"nickname":42,"user_avatar":134,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":112,"language":140,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":12,"update_tm":80,"read_time":118,"preview_page_count":4},962084925502,"https://ap-avatar.wpscdn.com/davatar_6f874abed73319feea01a86fa6f0fab8","Extended Abstract  \nMotivation SketchRNN generates vector sketches one stroke at a time by imitating human drawings from the Quick, Draw! dataset (Ha and Eck, 2017), with no objective tied to the quality of the final rendered image. The model captures stroke statistics well but has no awareness of whether the completed sketch is recognizable. We investigate whether reinforcement learning with image-based rewards can close this gap, improving semantic quality while preserving the diversity of the pretrained model.  \nMethod We fine-tune the SketchRNN decoder using REINFORCE with the encoder frozen, comparing three reward functions: (1) an SSIM reward measuring pixel-level similarity to a fixed target owl image; (2) a sparse CLIP reward scoring the final rendered sketch against the prompt “a sketch of an owl” using CLIP ViT-B/32 (Radford et al., 2021); and (3) a dense CLIP reward computing CLIP scores every k = 25 strokes throughout the episode and summing them into a discounted return G = Pt γT −t · rt. To reduce gradient variance, batches of 16 episodes share a single latent vector z, isolating decoder stochasticity as the sole source of within-batch reward variance and enabling within-batch advantage normalization.  \nImplementation We built on the pretrained SketchRNN owl checkpoint from Google Magenta’s Quick, Draw! model zoo. CLIP rewards use softmax confidence against four negative prompts (cat, dog, bird, fish) . Dense rewards rasterize partial stroke sequences at each checkpoint before scoring with CLIP. All experiments train for 5,000 episodes with batch size 16, Adam at lr = 10−5 , gradient clipping to [−1, 1], sampling temperature 0 .5, and 3 warm-up batches. We also debugged a renderer issue present in our milestone experiments where the pen was incorrectly staying down between strokes, and re-ran all experiments with the fixed renderer. Training ran on Modal cloud GPUs (NVIDIA T4) .  \nResults We evaluated 2,000 generated sketches per method across CLIP softmax confidence, FID vs. QuickDraw owls, precision and recall, and an independent ResNet18 QuickDraw classifier. Dense CLIP reward achieved the highest CLIP score (0 .644 vs. 0.478 baseline, +35%) with a 96 .4% classifier rate, confirming gains were not CLIP-gaming. All RL methods increased FID relative to baseline (81.47), indicating a tradeoff between CLIP-optimized quality and similarity to the QuickDraw distribution. Sparse CLIP reward mode-collapsed (recall 0.156 vs. 0.254 baseline); dense reward partially mitigated this (recall 0.244) . SSIM reward failed to generalize, producing the lowest CLIP score (0 .284) and classifier rate (86 .0%) .  \nDiscussion Dense rewards provided a meaningful improvement over sparse rewards, likely by giving early strokes better credit assignment. The FID increase across all RL methods suggests that CLIP optimization pushes sketches toward a visual style CLIP finds more owl-like, but farther from human Quick Draw sketches—an expected tradeoff. The recall gap between sparse and dense CLIP reward (0.156 vs. 0.244) suggests denser feedback during generation helps preserve diversity. One surprising result was that SSIM preserved more diversity than CLIP at the milestone stage, suggesting semantic reward optimization can cause stronger mode collapse than pixel-level similarity optimization.  \nConclusion SketchRL shows that RL with image-based rewards can meaningfully improve the semantic quality of generative sketch models beyond stroke-level imitation learning. Dense intermediate rewards outperform sparse terminal rewards on semantic quality while better preserving diversity. Independent classifier evaluation confirms these gains reflect genuine improvement rather than reward hacking.  \nSketchRL: Finetuning Generative Sketch Models with  \nVisual Rewards  \nMallika Parulekar  \nDepartment of Computer Science Stanford University  \n[mallika.parulekar@stanford.edu](mallika.parulekar@stanford.edu)  \nHannah Levin  \nDepartment of Computer Scie","cbCaiqo1wI9fG7aX","https://ap.wps.com/l/cbCaiqo1wI9fG7aX","pdf",2825688,"English","# Introduction\n## Motivation\n## Method\n## Implementation\n## Results\n## Discussion\n## Conclusion","[{\"question\":\"What problem does SketchRL address compared with standard SketchRNN training?\",\"answer\":\"Standard SketchRNN trains on stroke-level supervision and lacks an objective that directly optimizes whether the final rendered sketch is recognizable. SketchRL adds reinforcement learning with visual rewards to improve semantic quality.\"},{\"question\":\"Which reward functions are compared in the paper?\",\"answer\":\"The paper compares SSIM reward against a target image, sparse CLIP reward computed only at the end of the episode, and dense CLIP reward computed at intermediate stroke checkpoints with discounted returns.\"},{\"question\":\"What are the main findings from the evaluation?\",\"answer\":\"Dense CLIP reward achieves the highest CLIP score and improves classifier-based recognition while better preserving diversity than sparse reward. All RL methods increase FID, indicating a tradeoff between CLIP-optimized semantic style and similarity to the original QuickDraw distribution.\"}]","SketchRL - Finetuning Generative Sketch Models with Visual Rewards | PDF"]