[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-121746-en":3,"doc-seo-121746-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},121746,34359740700684,"Finn","https://ap-avatar.wpscdn.com/avatar/1f400023980c374ae676?_k=1777273430885731487",8,"Research & Report","Transformers in Machine Learning - Literature Review","This literature review presents an approach to methods in Transformer-based machine learning. Transformers are described as deep neural network architectures for modeling contextual relationships between words, connecting encoder and decoder via context vectors. The review surveys applications across NLP and beyond, including text compression, sentiment and emotion detection on social media, chemical image recognition (reported 96% accuracy), abusive-language classification in Indonesian news comments, and other tasks such as machine translation and health-related analysis. Comparative reporting focuses on dataset, analysis method, year, and achieved accuracy to identify the highest-accuracy options and guide future research opportunities.","Transformers in Machine Learning: Literature Review  \nThoyyibah T1*, Wasis Haryono1, Achmad Udin Zailani1, Yan Mitha Djaksana1, Neny Rosmawarni2, Nunik Destria Arianti3  \n1 Faculty of Computer Science, Informatics Engineering Study Program, Universitas Pamulang, Banten, Indonesia.  \n2 Faculty of Computer Science, Informatics Engineering Study Program, Universitas Pembangunan Nasional Veteran, Jakarta, Indonesia.  \n3 Faculty of Computer Engineering and Design, Information System Study Program, Nusa Putra University, Bandung, Indonesia.  \nReceived: July 15, 2023  \nRevised: August 20, 2023  \nAccepted: September 25, 2023  \nPublished: September 30, 2023  \nCorresponding Author: Thoyyibah T  \n[doseno01116@npam.ac.id](doseno01116@npam.ac.id)  \n[DOI: 10.29303/jppipa.v9i9.5040](DOI: 10.29303/jppipa.v9i9.5040)  \n© 2023 The Authors. This open access article is distributed under a (CC-BY License)  \nIntroduction  \nAbstract: In this study, the researcher presents an approach regarding methods in Transformer Machine Learning. Initially, transformers are neural network architectures that are considered as inputs. Transformers are widely used in various studies with various objects. The transformer is one of the deep learning architectures that can be modified. Transformers are also mechanisms that study contextual relationships between words. Transformers are used for text compression in readings. Transformers are used to recognize chemical images with an accuracy rate of 96% . Transformers are used to detect a person's emotions. Transformer to detect emotions in social media conversations, for example, on Facebook with happy, sad, and angry categories. Figure 1 illustrates the encoder and decoder process through the input process and produces output. the purpose of this study is to only review literature from various journals that discuss transformers. This explanation is also done by presenting the subject or dataset, data analysis method, year, and accuracy achieved. By using the methods presented, researchers can conclude results in search of the highest accuracy and opportunities for further research.  \nKeywords: Accuracy; Machine learning; Transformer  \nTransformer is part of NLP, an open-source library (Singla et al. ,2020) . The Natural Language process uses  \nAt first, the transformer is a neural network architecture that is considered as input (Reinauer et al., 2021; Grechishnikova, 2021; Ramos-Pérez et al., 2021; Moutik et al., 2023; Röder, 2023; Lin et al., 2022; Abed et al., 2023) . In principle, a transformer is needed to solve sequential problems such as sentences, which is an artificial neural network architecture (Luitse et al., 2021; Taye, 2023; Yang & Wang, 2020) . The transformer is incharge of connecting the encoder and decoder to the text in stages (Singla et al., 2020; Alqudsi et al., 2019; Liu & Chen, 2022) input and output are interconnected via context vectors. The autoencoder is capable of converting input into output. Besides being used in Natural Language Process transformers, it is also used in computer vision (Ghojogh et al., 2020; He et al., 2023) . Transformer is a concept in Natural language Processing (NLP) in Deep Learning (Singla et al., 2020) .  \nmany words that are processed through transformers (Singla et al., 2020; Khurana et al., 2023; Caucheteux & King, 2022) . Several studies use transformers, namely the application of machine learning using the transformer method in the health sector (Alqudsi et al., 2019; Arshed et al., 2023; Sanmarchi et al., 2023) . Transformers used to examine people's views through politics on social media apply BERT, BERT, LSTM, Support Vector Machine, Decision Trees, Naïve Bayes, and Electra. In this study, the highest accuracy value used the Electra model with a value of 70%(Öztürk et al., 2022) . The use of TurnGPT transformer model is used in spoken dialogue (Ekstedt et al., 2020) . Research that applies transformers to musical chord recognition using the bi-directional Tr","cbCaimP8JAq57zan","https://ap.wps.com/l/cbCaimP8JAq57zan","pdf",297006,1,7,"English","en",105,"# Introduction\n## Transformer fundamentals\n## Applications and reported accuracy\n## Theory and use cases\n## Research review dimensions (dataset, method, year, accuracy)","[{\"question\":\"What are transformers in machine learning used for according to the review?\",\"answer\":\"Transformers are used to model contextual relationships between words and support tasks such as text compression, sentiment/emotion detection, and other deep learning applications across NLP and computer vision.\"},{\"question\":\"How does the paper evaluate studies within the transformer literature review?\",\"answer\":\"It reviews literature by presenting the subject or dataset, the data analysis method, the year, and the accuracy achieved, enabling comparison across studies.\"},{\"question\":\"Which transformer-based models and tasks are mentioned in the document?\",\"answer\":\"The document mentions BERT, Electra, GPT-style encoder–decoder setups, and applications such as machine translation, film review sentiment analysis, chemical image recognition, and next-week influenza prediction.\"}]","Transformers in Machine Learning - Literature Review | PDF",1785806620,18,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"transformers-in-machine-learning-literature-review","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/transformers-in-machine-learning-literature-review/121746/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What are transformers in machine learning used for according to the review?","Question",{"text":75,"@type":76},"Transformers are used to model contextual relationships between words and support tasks such as text compression, sentiment/emotion detection, and other deep learning applications across NLP and computer vision.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the paper evaluate studies within the transformer literature review?",{"text":80,"@type":76},"It reviews literature by presenting the subject or dataset, the data analysis method, the year, and the accuracy achieved, enabling comparison across studies.",{"name":82,"@type":73,"acceptedAnswer":83},"Which transformer-based models and tasks are mentioned in the document?",{"text":84,"@type":76},"The document mentions BERT, Electra, GPT-style encoder–decoder setups, and applications such as machine translation, film review sentiment analysis, chemical image recognition, and next-week influenza prediction.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]