[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118253-en":3,"doc-seo-118253-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118253,687197207639,"Asher","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","Taking the Law More Seriously - Investigating Design Choices in Machine Learning Prediction Research","Approaches to court case prediction using machine learning differ widely in both effectiveness and legal reasonableness due to challenges inherent to legal reasoning and to overlooked impacts of methodological design. The study builds four machine learning models for predicting cases from the European Court of Human Rights and experimentally evaluates how performance metrics, selected parts of the legal case, the degree of legal specialization, and temporal effects of past decisions shape both model performance and alignment with legal-domain characteristics, clarifying design limits.","University of Groningen  \nTaking the Law More Seriously by Investigating Design Choices in Machine Learning Prediction Research  \nSteging, Cor; Renooij, Silja; Verheij, Bart  \nPublished in:  \n6th Workshop on Automated Semantic Analysis of Information in Legal Text, ASAIL 2023  \nIMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document version below.  \nDocument Version  \nPublisher's PDF, also known as Version of record  \nPublication date: 2023  \nLink to publication in University of Groningen/UMCG research database  \nCitation for published version (APA):  \nSteging, C. , Renooij, S. , & Verheij, B. (2023) . Taking the Law More Seriously by Investigating Design Choices in Machine Learning Prediction Research. In 6th Workshop on Automated Semantic Analysis of Information in Legal Text, ASAIL 2023 (pp. 49-59) . (CEUR Workshop Proceedings; Vol. 3441) . CEUR Workshop Proceedings.  \nCopyright  \nOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons) .  \nThe publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license. More information can be found on the University of Groningen website: [https://www.rug.nl/library/open-access/self-archiving-pure/taverne](https://www.rug.nl/library/open-access/self-archiving-pure/taverne)amendment.  \nTake-down policy  \nIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.  \nDownloaded from the University of Groningen/U MCG research database (Pure): [http://www.rug. nl/research/portal. For technical reasons the](http://www.rug. nl/research/portal. For technical reasons the)[ ](http://www.rug. nl/research/portal. For technical reasons the)[number of authors shown on this cover page is limited to 10 maximum.](number of authors shown on this cover page is limited to 10 maximum.)  \nDownload date: 27-12-2024  \nTaking the Law More Seriously by Investigating Design Choices in Machine Learning Prediction Research  \nCor Steging1, * , Silja Renooij2 and Bart Verheij1  \n1 Bernoulli Institute of Mathematics, Computer Science and Artificial Intelligence, University of Groningen  \n2 Department of Information and Computing Sciences, Utrecht University  \nAbstract  \nApproaches to court case prediction using machine learning differ widely with varying levels of success and legal reasonableness. In part this is due to some aspects of law, such as justification, being inherently difficult for machine learning approaches. Another aspect is the effect of design choices and the extent to which these are legally reasonable, which has not yet been extensively studied. We create four machine learning models tasked with predicting cases from the European Court of Human Rights and we perform experiments in order to measure the role of the following four design choices and effects: the choice of performance metric; the effect of including different parts of the legal case; the effect of a more or less specialized legal focus; and the temporal effects of the available past legal decisions. Through this research, we aim to study design decisions and their limitations and how they affect the performance of machine learning models.  \nKeywords  \nCourt case prediction, design choices, machine learning  \n1. Introduction  \nRecently, much work has been done in the field of court case predictions. While automatically determining the outcome of court cases remains an academic exercise, the large variation in the ways that previous research has tackled the problem makes it nearly impossible to compare the approaches [1] . The law has unique characteristics, making it difficult to ap","cbCaibyhsxoMHcIf","https://ap.wps.com/l/cbCaibyhsxoMHcIf","pdf",1621303,1,12,"English","en",105,"# Introduction\n## Machine learning and court case prediction challenges\n## Research focus on design choices","[{\"question\":\"Why are court case prediction approaches hard to compare across prior work?\",\"answer\":\"Previous research varies substantially in how it tackles the problem, making comparisons difficult. The unique characteristics of law also complicate applying machine learning in legal settings.\"},{\"question\":\"Which design choices does the research investigate?\",\"answer\":\"It studies four factors: the choice of performance metric, the effect of including different parts of the legal case, the impact of a more or less specialized legal focus, and temporal effects of past legal decisions.\"},{\"question\":\"What data and models are used in the experiments?\",\"answer\":\"The work creates four machine learning models tasked with predicting cases using information from the European Court of Human Rights, then runs experiments to measure the role of the specified design choices.\"}]","Taking the Law More Seriously - Investigating Design Choices in Machine Learning Prediction Research | PDF",1785682652,30,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"taking-the-law-more-seriously-investigating-design-choices-in-machine-learning-prediction-research","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/taking-the-law-more-seriously-investigating-design-choices-in-machine-learning-prediction-research/118253/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why are court case prediction approaches hard to compare across prior work?","Question",{"text":75,"@type":76},"Previous research varies substantially in how it tackles the problem, making comparisons difficult. The unique characteristics of law also complicate applying machine learning in legal settings.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"Which design choices does the research investigate?",{"text":80,"@type":76},"It studies four factors: the choice of performance metric, the effect of including different parts of the legal case, the impact of a more or less specialized legal focus, and temporal effects of past legal decisions.",{"name":82,"@type":73,"acceptedAnswer":83},"What data and models are used in the experiments?",{"text":84,"@type":76},"The work creates four machine learning models tasked with predicting cases using information from the European Court of Human Rights, then runs experiments to measure the role of the specified design choices.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":29,"slug":121},"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]