[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-123118-en":3,"doc-seo-123118-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},123118,687197207639,"Asher","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","Machine Learning-Based Analysis of Test Results - Master’s Thesis","A comprehensive understanding of overall test performance is essential in software testing for improving test schedules and enabling focused investigations. In ABB Bryne, the IPS software testing for paint robots is processed through the NAST test system. This thesis tackles two core challenges: ranking frequently failing test cases and clustering test cases by analyzing error-message patterns. Ranking compares AHP-WSM with LambdaMART, while clustering applies agglomerative methods using Jaccard and cosine similarity metrics to capture text relationships beyond exact term matches.","| \u003Cbr>Faculty of Science and Technology\u003Cbr>MASTER’S THESIS |  |  |  |  |  |  |\n| --- | --- | --- | --- | --- | --- | --- |\n| Study program/ Specialization:\u003Cbr>Reliable and Secure Systems |  |  |  |  | Autumn semester, 2023\u003Cbr> |  |\n|  |  |  |  |  | Open | access |\n| Writer: | Xiaoyan Sun |  |  |  | \u003Cbr>…………………………………………\u003Cbr>(Writer’s signature) |  |\n| Faculty supervisor: |  |  |  |  |  |  |\n|  |  | Morten Mossige |  |  |  |  |\n|  |  |  |  |  |  |  |\n|  |  |  |  |  |  |  |\n| Thesis title:\u003Cbr>Machine Learning-Based Analysis of Test Results |  |  |  |  |  |  |\n| Credits (ECTS) : 30 |  |  |  |  |  |  |\n| Key words:\u003Cbr>Ranking, AHP, WSM, LambdaMART, Agglomerative Clustering, Jaccard Similarity, Cosine Similariy |  |  |  |  | Pages: 52\u003Cbr>+ enclosure: 56\u003Cbr>Stavanger, 15 December, 2023 |  |\n|  |  |  |  |  |  |  |\n\nFront page for master thesis Faculty of Science and Technology Decision made by the Dean October 30th 2009  \nFaculty of Science and Technology  \nDepartment of Electrical Engineering and Computer Science  \nMachine Learning-Based Analysis of Test  \nResults  \nMaster’s Thesis in Computer Science  \nby  \nXiaoyan Sun  \nInternal Supervisors  \nMorten Mossige  \nDecember 12, 2023  \nAbstract  \nA comprehensive understanding of the overall performance of the tests is critical in software testing to make necessary adjustments to the test schedule or conduct targeted investigations. In ABB Bryne, the software testing is processed by a test system named NAST for the IPS software for paint robots. This thesis addresses the critical challenges by investigating two key aspects: ranking frequently failing test cases and clustering testcases based on error messages.  \nTwo models were employed for the ranking task: the conventional Analytic Hierarchy Process-Weighted Sum Model (AHP-WSM) and the Learning-to-rank model LambdaMART. The results highlight the superiority of the AHP-WSM model over LambdaMART, attributed to the influence of dataset quality and size in the machine learning technique.  \nAgglomerative clustering with Jaccard and cosine similarity metrics was applied for the clustering. The findings reveal that cosine similarity yielded superior outcomes due to its calculation characteristics to capture semantic relationships between texts based on overall content rather than relying on exact term matches.  \nThis research provides valuable insights into effective methodologies for addressing the complexities associated with test case prioritisation and clustering in software testing.  \nAcknowledgements  \nI sincerely thank my supervisor, Morten Mossige, whose continuous guidance has been invaluable throughout this research journey. Special thanks to Nina Svense, Lasse Ånestad, and Donjing Liu for their insightful and constructive feedback, which significantly enhanced the quality of this work.  \nI also deeply appreciate my friend, Ronbing Li and my boyfriend, Njål Brocker Næss, for their reliable support during the stressful periods of my master’s thesis. Their encouragement and understanding were invaluable throughout my academic journey.  \nContents  \nAbstract i  \nAcknowledgements ii  \nAbbreviations iv  \n1 Introduction 1  \n1. 1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1  \n1.2 Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2  \n1.2.1 Ranking Frequently Failed Tests . . . . . . . . . . . . . . . . . . . 2  \n1.2.2 Clustering Test Cases Based on Error Messages ........... 2  \n1.3 Objectives .................................... 3  \n1.4 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4  \n2 Background 5  \n2.1 Ranking and Learning to Rank . . . . . . . . . . . . . . . . . . . . . . . . 5  \n2.1. 1 AHP-WSM Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 5  \n2.1.2 LambdaMART . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9  \n2.2 Clustering .................................... 11  \n2.2.1 Clustering Method .......................","cbCaivTbnO6Ny8D8","https://ap.wps.com/l/cbCaivTbnO6Ny8D8","pdf",1786242,1,65,"English","en",105,"# Abstract\n# Acknowledgements\n# Abbreviations\n# Introduction\n## Motivation\n## Use Cases\n## Objectives\n## Outline\n# Background\n## Ranking and Learning to Rank\n## Clustering\n## Previous Work\n## Related Work\n# Approaches for Ranking Model\n## Data Preprocessing\n## AHP-WSM model\n## LambdaMART\n## Evaluation\n# Approaches for Clustering Model\n## Data Preprocessing\n## Modelling\n## Evaluation\n# Result\n## Ranking Results\n## Clustering Results\n# Discussion\n## Ranking Model Results Analysis","[{\"question\":\"What two key problems does the thesis address in software testing?\",\"answer\":\"It addresses ranking frequently failing test cases and clustering test cases based on error messages. These tasks aim to improve how testing is prioritized and investigated.\"},{\"question\":\"Which models are used for the test-case ranking task, and what is the outcome?\",\"answer\":\"The thesis uses AHP-WSM and the learning-to-rank model LambdaMART. The results indicate AHP-WSM performs better, influenced by dataset quality and size.\"},{\"question\":\"How is clustering performed and why does cosine similarity perform better?\",\"answer\":\"Agglomerative clustering is applied using Jaccard and cosine similarity metrics. Cosine similarity yields superior results because it better captures semantic relationships across text based on overall content rather than exact term matches.\"}]","Machine Learning-Based Analysis of Test Results - Master’s Thesis | PDF",1785814727,164,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"machine-learning-based-analysis-of-test-results-masters-thesis","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/machine-learning-based-analysis-of-test-results-masters-thesis/123118/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What two key problems does the thesis address in software testing?","Question",{"text":75,"@type":76},"It addresses ranking frequently failing test cases and clustering test cases based on error messages. These tasks aim to improve how testing is prioritized and investigated.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"Which models are used for the test-case ranking task, and what is the outcome?",{"text":80,"@type":76},"The thesis uses AHP-WSM and the learning-to-rank model LambdaMART. The results indicate AHP-WSM performs better, influenced by dataset quality and size.",{"name":82,"@type":73,"acceptedAnswer":83},"How is clustering performed and why does cosine similarity perform better?",{"text":84,"@type":76},"Agglomerative clustering is applied using Jaccard and cosine similarity metrics. Cosine similarity yields superior results because it better captures semantic relationships across text based on overall content rather than exact term matches.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]