[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118648-en":3,"doc-seo-118648-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118648,1099514068365,"Aurelia","https://ap-avatar.wpscdn.com/avatar/10000253d8d9f28188e?_k=1776742907772140068",8,"Research & Report","Taking the Law More Seriously - Investigating Design Choices in Machine Learning Prediction Research","Approaches to court case prediction using machine learning vary in success and legal reasonableness, in part because key legal factors such as justification are difficult to operationalize. This work examines how design choices influence both performance and legal alignment, focusing on four model-building decisions and effects. Four machine learning models predict cases from the European Court of Human Rights dataset to measure impacts of the chosen performance metric, inclusion of case parts, specialized legal focus, and temporal effects of prior decisions.","University of Groningen  \nTaking the Law More Seriously by Investigating Design Choices in Machine Learning Prediction Research  \nSteging, Cor; Renooij, Silja; Verheij, Bart  \nPublished in:  \n6th Workshop on Automated Semantic Analysis of Information in Legal Text, ASAIL 2023  \nIMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document version below.  \nDocument Version  \nPublisher's PDF, also known as Version of record  \nPublication date: 2023  \nLink to publication in University of Groningen/UMCG research database  \nCitation for published version (APA):  \nSteging, C. , Renooij, S. , & Verheij, B. (2023) . Taking the Law More Seriously by Investigating Design Choices in Machine Learning Prediction Research. In 6th Workshop on Automated Semantic Analysis of Information in Legal Text, ASAIL 2023 (pp. 49-59) . (CEUR Workshop Proceedings; Vol. 3441) . CEUR Workshop Proceedings.  \nCopyright  \nOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons) .  \nThe publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license. More information can be found on the University of Groningen website: [https://www.rug.nl/library/open-access/self-archiving-pure/taverne](https://www.rug.nl/library/open-access/self-archiving-pure/taverne)amendment.  \nTake-down policy  \nIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.  \nDownloaded from the University of Groningen/U MCG research database (Pure): [http://www.rug. nl/research/portal. For technical reasons the](http://www.rug. nl/research/portal. For technical reasons the)[ ](http://www.rug. nl/research/portal. For technical reasons the)[number of authors shown on this cover page is limited to 10 maximum.](number of authors shown on this cover page is limited to 10 maximum.)  \nDownload date: 01-01-2026  \nTaking the Law More Seriously by Investigating Design Choices in Machine Learning Prediction Research  \nCor Steging1, * , Silja Renooij2 and Bart Verheij1  \n1 Bernoulli Institute of Mathematics, Computer Science and Artificial Intelligence, University of Groningen  \n2 Department of Information and Computing Sciences, Utrecht University  \nAbstract  \nApproaches to court case prediction using machine learning differ widely with varying levels of success and legal reasonableness. In part this is due to some aspects of law, such as justification, being inherently difficult for machine learning approaches. Another aspect is the effect of design choices and the extent to which these are legally reasonable, which has not yet been extensively studied. We create four machine learning models tasked with predicting cases from the European Court of Human Rights and we perform experiments in order to measure the role of the following four design choices and effects: the choice of performance metric; the effect of including different parts of the legal case; the effect of a more or less specialized legal focus; and the temporal effects of the available past legal decisions. Through this research, we aim to study design decisions and their limitations and how they affect the performance of machine learning models.  \nKeywords  \nCourt case prediction, design choices, machine learning  \n1. Introduction  \nRecently, much work has been done in the field of court case predictions. While automatically determining the outcome of court cases remains an academic exercise, the large variation in the ways that previous research has tackled the problem makes it nearly impossible to compare the approaches [1] . The law has unique characteristics, making it difficult to ap","cbCaid9sGpB7HRLg","https://ap.wps.com/l/cbCaid9sGpB7HRLg","pdf",1656554,1,12,"English","en",105,"# Introduction\n## Motivation and challenges in ML-for-law\n## Study focus: design choices and evaluation aims","[{\"question\":\"Why is court case prediction difficult to do in a legally reasonable way with machine learning?\",\"answer\":\"Legal tasks like justification are inherently difficult for machine learning, and ML systems may use unsound reasoning. The legal domain also changes over time and requires argumentation for decisions.\"},{\"question\":\"What is the main research goal of the paper?\",\"answer\":\"To study how specific design choices and their effects affect both the performance of machine learning models and their alignment with characteristics of the legal domain.\"},{\"question\":\"Which four design choices and effects are investigated?\",\"answer\":\"The paper studies the choice of performance metric, the effect of including different parts of the legal case, the effect of a more or less specialized legal focus, and temporal effects of available past legal decisions.\"}]","Taking the Law More Seriously - Investigating Design Choices in Machine Learning Prediction Research | PDF",1785684691,30,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"taking-the-law-more-seriously-investigating-design-choices-in-machine-learning-prediction-research","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/taking-the-law-more-seriously-investigating-design-choices-in-machine-learning-prediction-research/118648/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is court case prediction difficult to do in a legally reasonable way with machine learning?","Question",{"text":75,"@type":76},"Legal tasks like justification are inherently difficult for machine learning, and ML systems may use unsound reasoning. The legal domain also changes over time and requires argumentation for decisions.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is the main research goal of the paper?",{"text":80,"@type":76},"To study how specific design choices and their effects affect both the performance of machine learning models and their alignment with characteristics of the legal domain.",{"name":82,"@type":73,"acceptedAnswer":83},"Which four design choices and effects are investigated?",{"text":84,"@type":76},"The paper studies the choice of performance metric, the effect of including different parts of the legal case, the effect of a more or less specialized legal focus, and temporal effects of available past legal decisions.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":29,"slug":121},"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]