[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-209935-en":3,"doc-seo-209935-105":31,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},209935,24189269381491,"Bill Black","https://ap-avatar.wpscdn.com/avatar/160000cf11732dd8392?x-image-process=image/resize,m_fixed,w_180,h_180&k=1788146458752108895",8,"Research & Report","Automatic Evaluation of ENEM-like Essays - An Analysis of Benchmarks, Gaming Susceptibility, Transferability and Feedback","Automatic Evaluation of ENEM-like Essays explores how to automatically assess essay quality using computational benchmarks and evaluates three major risks and transfer behaviors. The work analyzes gaming susceptibility, measuring how models can be exploited by adversarial or strategically written responses, and studies transferability to new settings or distributions. It also investigates the role of automatic feedback, connecting evaluation outputs with actionable guidance for improving student writing performance. The thesis presents an analytical framework and findings for designing more robust essay assessment systems.","Automatic Evaluation of ENEM-like Essays: An Analysis of Benchmarks, Gaming Susceptibility, Transferability and  \nFeedback  \nIgor Cataneo Silveira  \nThesis presented to the  \nInstitute of Mathematics, Statistics, and Computer Science of the University of São Paulo  \nin partial fulfillment  \nof the requirements  \nfor the degree of  \nDoctor of Science  \nProgram: Ciência da Computação  \nAdvisor: Prof. Dr. Denis Deratani Mauá  \nDurante o desenvolvimento deste trabalho o autor recebeu auxílio financeiro da CAPES  \nSão Paulo  \nApril, 2026  \nAutomatic Evaluation of ENEM-like Essays: An Analysis of Benchmarks, Gaming Susceptibility, Transferability and  \nFeedback  \nIgor Cataneo Silveira  \nThis version of the thesis includes the corrections and modifications suggested by the Examining Committee during the defense of the original version of the work, which took place on April 28, 2026 .  \nA copy of the original version is available at the Institute of Mathematics,  \nStatistics, and Computer Science of the University ofSão Paulo.  \nExamining Committee:  \nProf. Dr. Denis Deratani Mauá (advisor)– IME-USP Prof. Dr. Aline Paes – UFF  \nProf. Dr. Evelin Amorim – INESCTEC  \nProf. Dr. Fábio Gagliardi Cozman – POLI-USP Prof. Dr. Valéria Delisandra Feltrim – UEM  \nThe content of this work is published under the CC BY 4 .0 license  \n(Creative Commons Attribution 4.0 International License)  \nFicha catalográfica elaborada com dados inseridos pelo(a) autor(a) Biblioteca Carlos Benjamin de Lyra Instituto de Matemática e Estatística Universidade de São Paulo  \nSilveira, Igor Cataneo  \nAutomatic Evaluation of ENEM-like Essays: An Analysis of Benchmarks, Gaming Susceptibility, Transferability and Feedback / Igor Cataneo Silveira; orientador, Denis Deratani Mauá . -São Paulo, 2026 .  \n95 p . : il .  \nTese (Doutorado) - Programa de Pós-Graduação em Ciência da Computação / Instituto de Matemática, Estatística e Ciênciada Computação / Universidade de São Paulo .  \nBibliografia  \n1 . Correção Automática de Redação . 2 . ENEM . 3 . Ataques Adversa riais . 4 . Transferência de Conhecimento . 5 . Feedback Automático . I . Mauá, Denis Deratani . II . Título .  \nBibliotecárias do Serviço de Informação e Biblioteca Carlos Benjamin de Lyra do IME-USP, responsáveis pela estrutura de catalogação da publicação de acordo com a AACR2:  \nMaria Lúcia Ribeiro CRB-8/2766; Stela do Nascimento Madruga CRB 8/7534 .  \nΓνῶθι µαθών. (Stobaeus, 5th c.)  \ni  \nAgradecimentos  \nConforme prometido na dissertação de mestrado: dedico esse doutorado, acima de tudo, à minha família. Aos meus avôs — Ciro e Waldir —já falecidos, que ficaram naminha memória como exemplos a serem seguidos. Às minhas avós, Rose — por tudo emais um pouco – e Leonor —pelos “filho, come o feijão que é da hora”,“filho, vem comer que tá uma delícia”,“lembra que, quando você era pequeno, você adorava a canjiquinhada vó?” Aos meus pais, Sandro e Lilian, que sempre me apoiaram nesse longo processoda pós-graduação. Todos meus tios, tias e primas. O Adri e a Yasmim, com quem tenhosempre assunto infinito. A Gê e o Wandy, que também fazem tudo e mais um pouco por mim. A Pri, o Bira e a Rafa, que me recebem todo domingo na casa deles para almoço. A Ângela, o Luciano e a Allana, com quem acabo interagindo menos, mas nem por issonão são importantes. O Ju e a Karla, que sempre me convidam e recebem para a virada deano — foi, inclusive, em uma dessas que eu escrevi o primeiro artigo que se tornaria essatese. E meu irmão Iuri, que foi prejudicado pela escrita desse trabalho, tendo que ceder o quarto e perdendo uma hora de jogatina noturna — foi mal, piá .  \nGostaria de escrever um pouco sobre cada uma das pessoas (não-familiares) que marcaram toda essa jornada, mas infelizmente não vou ter como, então fica a menção honrosa de agradecimento e a torcida para que a pessoa saiba o porquê de estar sendo nomeada:  \nAlana “Astreisky” Streisky da Silva, Aline “F.” Carolina Oliveira, Amanda “Futura Ministra”Simões, Ana Paula Stienen, Ande","cbCaia7rbMXkN1SN","https://ap.wps.com/l/cbCaia7rbMXkN1SN","pdf",3245897,3,1,115,"English","en",105,"# Acknowledgements\n# References","[{\"question\":\"What topic does the thesis address?\",\"answer\":\"The thesis focuses on automatic evaluation of ENEM-like essays, including benchmark analysis and the quality of generated assessment and feedback.\"},{\"question\":\"What does “gaming susceptibility” mean in this work?\",\"answer\":\"It refers to how easily an essay evaluation system can be manipulated by adversarial or strategically written inputs.\"},{\"question\":\"How does the thesis evaluate transferability and feedback?\",\"answer\":\"It analyzes how evaluation behavior generalizes across contexts or distributions and examines how automatic feedback supports improvement in essay writing.\"}]","Automatic Evaluation of ENEM-like Essays - An Analysis of Benchmarks, Gaming Susceptibility, Transferability and Feedback | PDF",1788637120,290,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":29},"automatic-evaluation-of-enem-like-essays-an-analysis-of-benchmarks-gaming-susceptibility-transferability-and-feedback","",{"@graph":37,"@context":86},[38,54,69],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,51],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":20},"https://docshare.wps.com/document/research-report/",{"item":52,"name":13,"@type":44,"position":53},"https://docshare.wps.com/document/automatic-evaluation-of-enem-like-essays-an-analysis-of-benchmarks-gaming-susceptibility-transferability-and-feedback/209935/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":42,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-09-11","2026-09-05",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What topic does the thesis address?","Question",{"text":76,"@type":77},"The thesis focuses on automatic evaluation of ENEM-like essays, including benchmark analysis and the quality of generated assessment and feedback.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"What does “gaming susceptibility” mean in this work?",{"text":81,"@type":77},"It refers to how easily an essay evaluation system can be manipulated by adversarial or strategically written inputs.",{"name":83,"@type":74,"acceptedAnswer":84},"How does the thesis evaluate transferability and feedback?",{"text":85,"@type":77},"It analyzes how evaluation behavior generalizes across contexts or distributions and examines how automatic feedback supports improvement in essay writing.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":47,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":47,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":47,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":47,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":47,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":47,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":47,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]