[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-150703-en":3,"doc-seo-150703-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},150703,687197207919,"Theodora","https://ap-avatar.wpscdn.com/avatar/a000253d6f5f7c60be?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779446848396160552",8,"Research & Report","Topic Scene Graph Generation - Attention Distillation from Caption","This document describes a topic-aware scene graph generation approach that derives relational captions from image content by distilling attention signals from caption generation. It specifies Transformer-based encoder-decoder architecture details and the parameter settings governing visual, semantic, and attention embeddings. Training follows two stages: captioning module optimization with Adam/Noam and a subsequent alignment module optimization with SGD while freezing captioning parameters, using ground-truth objects to reduce detector noise. Experiments analyze how varying a key hyperparameter affects image-level and relationship-recall metrics, compare likelihood-based versus random relationship evaluation, and study pooling and input-feature configurations to explain performance trends.","Topic Scene Graph Generation by Attention Distillation from Caption  \nWenbin Wang 1,2 , Ruiping Wang 1,2,3 , Xilin Chen 1,2  \n1 Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, Beijing, 100190, China  \n2University of Chinese Academy of Sciences, Beijing, 100049, China  \n3Beijing Academy of Artificial Intelligence, Beijing, 100084, China  \n[wenbin.wang@vipl.ict.ac.cn](wenbin.wang@vipl.ict.ac.cn) , fwangruiping, [xlchen](xlcheng@ict.ac.cn)[g](xlcheng@ict.ac.cn)[@ict.ac.cn](xlcheng@ict.ac.cn)  \nTable 1 . The details of the parameters settings.  \n\n| Params. | Meanings | Values |\n| --- | --- | --- |\n| dv | dimension of the visual features vi | 4,096 |\n| du | dimension of the union visual features vij | 512 |\n| dl | dimension of the transformed features | 1,024 |\n| dh | dimension of the hidden states in UD model [1] | 512 |\n| da | dimension of the attention embedding in UD model | 1,024 |\n| de | dimension of the word embedding in UD model | 1,024 |\n| dtr | dimension of the input/output inside the Transformer [6] | 512 |\n| ds | dimension of the query/key of the Transformer | 512 |\n| dsem | dimension of the semantic embedding | 200 |\n| TR | the maximum length of the relational captions | 18 |\n| TC | the maximum length of the image caption | 18 |\n\nFigure 1 . The details of the Transformer.  \n1. Implementation Details  \nThe parameters settings mentioned in the main paper are shown in Table 1.  \nIn Figure 1 we show the details of the Transformer. The Transformer has 6 encoder layers and 6 decoder layers, both of which have one attention head for simplicity. Only the attentions from the last decoder layer are collected. It is noted that when generating the relational captions, we still feed all of the object transformed features (i.e., V 0 2 Rdl 􀀂 n ) into the encoder, but only select the subject and object features from the output of the encoder and their union feature to assemble the V 􀀃 2 Rdtr 􀀂 3 as the input of the decoder.  \nTable 2 . Image captioning results. B1, B4, M, R, C, S denote the BLEU-1, BLEU-4, METEOR, ROUGE-L, CIDErD, and SPICE respectively.“-ICRC” denotes that the model is trained with image captions and relational captions.  \n\n| Model | 􀀕 | B1 | B4 | M | R | C | S |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| UD [1] | - | 69.8 | 29.6 | 25.0 | 52.3 | 94.1 | 18.0 |\n| UD-ICRC | 0.1 | 71.1 | 30.4 | 25.1 | 52.6 | 95.3 | 18.3 |\n|  | 0.3 | 70.7 | 30.0 | 24.9 | 52.5 | 94.6 | 18.1 |\n|  | 0.7 | 70.5 | 30.1 | 25.0 | 52.4 | 94.8 | 18.2 |\n|  | 1.0 | 71.0 | 30.0 | 24.8 | 52.5 | 93.5 | 17.9 |\n|  | 3.0 | 70.0 | 29.2 | 24.3 | 51.8 | 91.4 | 17.5 |\n|  | 5.0 | 69.6 | 28.9 | 23.9 | 51.3 | 89.6 | 17.2 |\n|  | 7.0 | 69.0 | 28.3 | 23.6 | 50.9 | 87.4 | 16.8 |\n|  | 10.0 | 68.7 | 27.4 | 23.2 | 50.3 | 84.8 | 16.7 |\n| Transformer [6] | - | 68.8 | 26.8 | 23.5 | 50.4 | 85.6 | 17.3 |\n| Transformer-ICRC | 0.7 | 70.3 | 28.6 | 24.4 | 51.7 | 91.5 | 18.0 |\n\nThe scene graph training is divided into two stages. In the first stage we train the captioning module for 30 epochs with the Adam [3] optimizer and the Noam policy [6] . The initial learning rate is set as 5e-4 and the batch size is 12 . In the second stage, the attention alignment module is trained with the SGD optimizer for 12 epochs during which the parameters of captioning module are frozen. The initial learning rate is set as 2e-2 . We use the ground truth objects (bounding boxes and categories) for training and evaluation to eliminate the interference of the error from detector.  \n2. Experiments about Topic Scene Graph  \nIn Table 2 and Table 3, we provide more results as the 􀀕 changes. From Table 2, it further proves the conclusion mentioned in the main paper that mixed training actually brings benefit to the image captioning, but as the 􀀕 increases, this benefit will slightly drop. When the 􀀕 is larger than 1, it will be harmful to the performance. In Table 3, asthe 􀀕 increases, the UD-ICRC roughly performs bet","cbCaikaNhUxDyLE0","https://ap.wps.com/l/cbCaikaNhUxDyLE0","pdf",2661171,1,7,"English","en",105,"# Implementation Details\n## Parameters and Transformer Structure\n## Two-Stage Training Protocol\n# Experiments about Topic Scene Graph\n## Hyperparameter Sensitivity and Metrics\n## Relationship Evaluation Strategies\n## Configurations: Pooling and Input Features","[{\"question\":\"How is the Transformer used to generate relational captions in the system?\",\"answer\":\"The model uses a Transformer with 6 encoder and 6 decoder layers. Only attentions from the last decoder layer are collected, while the decoder input is assembled from selected subject/object encoder outputs and their union feature.\"},{\"question\":\"What are the two training stages and how are modules updated?\",\"answer\":\"Stage one trains the captioning module for 30 epochs using Adam with a Noam learning-rate policy. Stage two trains the attention alignment module for 12 epochs using SGD while freezing the captioning module’s parameters, and ground-truth object boxes and categories are used for training and evaluation.\"},{\"question\":\"How does changing the key hyperparameter (σ) affect performance?\",\"answer\":\"Mixed training provides benefits for image captioning, but the benefit decreases as σ increases; performance becomes harmful when σ is larger than 1. Relationship recall metrics reach a peak at σ = 0.1, and when σ exceeds 7.0 performance begins to drop.\"}]","Topic Scene Graph Generation - Attention Distillation from Caption | PDF",1787825926,18,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"topic-scene-graph-generation-attention-distillation-from-caption","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/topic-scene-graph-generation-attention-distillation-from-caption/150703/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-27",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"How is the Transformer used to generate relational captions in the system?","Question",{"text":75,"@type":76},"The model uses a Transformer with 6 encoder and 6 decoder layers. Only attentions from the last decoder layer are collected, while the decoder input is assembled from selected subject/object encoder outputs and their union feature.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What are the two training stages and how are modules updated?",{"text":80,"@type":76},"Stage one trains the captioning module for 30 epochs using Adam with a Noam learning-rate policy. Stage two trains the attention alignment module for 12 epochs using SGD while freezing the captioning module’s parameters, and ground-truth object boxes and categories are used for training and evaluation.",{"name":82,"@type":73,"acceptedAnswer":83},"How does changing the key hyperparameter (σ) affect performance?",{"text":84,"@type":76},"Mixed training provides benefits for image captioning, but the benefit decreases as σ increases; performance becomes harmful when σ is larger than 1. Relationship recall metrics reach a peak at σ = 0.1, and when σ exceeds 7.0 performance begins to drop.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]