[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118643-en":3,"doc-seo-118643-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118643,1099514068365,"Aurelia","https://ap-avatar.wpscdn.com/avatar/10000253d8d9f28188e?_k=1776742907772140068",8,"Research & Report","Taking the Law More Seriously - Investigating Design Choices in Machine Learning Prediction Research","Approaches to court case prediction using machine learning vary widely in both effectiveness and legal reasonableness. Differences stem from the intrinsic difficulty of legal aspects such as justification for standard ML methods. This work examines how design choices influence legally meaningful prediction outcomes by testing four ML models on cases from the European Court of Human Rights, analyzing the impact of performance metrics, legal-case feature inclusion, legal focus specialization, and temporal effects of past decisions.","University of Groningen  \nTaking the Law More Seriously by Investigating Design Choices in Machine Learning Prediction Research  \nSteging, Cor; Renooij, Silja; Verheij, Bart  \nPublished in:  \n6th Workshop on Automated Semantic Analysis of Information in Legal Text, ASAIL 2023  \nIMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document version below.  \nDocument Version  \nPublisher's PDF, also known as Version of record  \nPublication date: 2023  \nLink to publication in University of Groningen/UMCG research database  \nCitation for published version (APA):  \nSteging, C. , Renooij, S. , & Verheij, B. (2023) . Taking the Law More Seriously by Investigating Design Choices in Machine Learning Prediction Research. In 6th Workshop on Automated Semantic Analysis of Information in Legal Text, ASAIL 2023 (pp. 49-59) . (CEUR Workshop Proceedings; Vol. 3441) . CEUR Workshop Proceedings.  \nCopyright  \nOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons) .  \nThe publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license. More information can be found on the University of Groningen website: [https://www.rug.nl/library/open-access/self-archiving-pure/taverne](https://www.rug.nl/library/open-access/self-archiving-pure/taverne)amendment.  \nTake-down policy  \nIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.  \nDownloaded from the University of Groningen/U MCG research database (Pure): [http://www.rug. nl/research/portal. For technical reasons the](http://www.rug. nl/research/portal. For technical reasons the)[ ](http://www.rug. nl/research/portal. For technical reasons the)[number of authors shown on this cover page is limited to 10 maximum.](number of authors shown on this cover page is limited to 10 maximum.)  \nDownload date: 29-12-2025  \nTaking the Law More Seriously by Investigating Design Choices in Machine Learning Prediction Research  \nCor Steging1, * , Silja Renooij2 and Bart Verheij1  \n1 Bernoulli Institute of Mathematics, Computer Science and Artificial Intelligence, University of Groningen  \n2 Department of Information and Computing Sciences, Utrecht University  \nAbstract  \nApproaches to court case prediction using machine learning differ widely with varying levels of success and legal reasonableness. In part this is due to some aspects of law, such as justification, being inherently difficult for machine learning approaches. Another aspect is the effect of design choices and the extent to which these are legally reasonable, which has not yet been extensively studied. We create four machine learning models tasked with predicting cases from the European Court of Human Rights and we perform experiments in order to measure the role of the following four design choices and effects: the choice of performance metric; the effect of including different parts of the legal case; the effect of a more or less specialized legal focus; and the temporal effects of the available past legal decisions. Through this research, we aim to study design decisions and their limitations and how they affect the performance of machine learning models.  \nKeywords  \nCourt case prediction, design choices, machine learning  \n1. Introduction  \nRecently, much work has been done in the field of court case predictions. While automatically determining the outcome of court cases remains an academic exercise, the large variation in the ways that previous research has tackled the problem makes it nearly impossible to compare the approaches [1] . The law has unique characteristics, making it difficult to ap","cbCaini1Dhb5NPhB","https://ap.wps.com/l/cbCaini1Dhb5NPhB","pdf",1656555,1,12,"English","en",105,"# Introduction\n## Motivation: law vs. machine learning in court prediction\n## Design-choice focus and research aims","[{\"question\":\"Why are court case predictions difficult to make with machine learning in the legal domain?\",\"answer\":\"Legal tasks differ from typical ML assumptions: law is prospective, changes over time, includes wrong decisions, and requires justification that ML systems often struggle to handle reliably.\"},{\"question\":\"What design choices does the research investigate for machine learning prediction?\",\"answer\":\"The study evaluates four aspects: the choice of performance metric, the effect of including different parts of the legal case, the impact of a more or less specialized legal focus, and temporal effects from available past decisions.\"},{\"question\":\"What data and setting are used to test the machine learning models?\",\"answer\":\"Four models are built to predict cases using data from the European Court of Human Rights, followed by experiments measuring how the selected design choices affect performance and alignment with legal-domain characteristics.\"}]","Taking the Law More Seriously - Investigating Design Choices in Machine Learning Prediction Research | PDF",1785684666,30,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"taking-the-law-more-seriously-investigating-design-choices-in-machine-learning-prediction-research","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/taking-the-law-more-seriously-investigating-design-choices-in-machine-learning-prediction-research/118643/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why are court case predictions difficult to make with machine learning in the legal domain?","Question",{"text":75,"@type":76},"Legal tasks differ from typical ML assumptions: law is prospective, changes over time, includes wrong decisions, and requires justification that ML systems often struggle to handle reliably.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What design choices does the research investigate for machine learning prediction?",{"text":80,"@type":76},"The study evaluates four aspects: the choice of performance metric, the effect of including different parts of the legal case, the impact of a more or less specialized legal focus, and temporal effects from available past decisions.",{"name":82,"@type":73,"acceptedAnswer":83},"What data and setting are used to test the machine learning models?",{"text":84,"@type":76},"Four models are built to predict cases using data from the European Court of Human Rights, followed by experiments measuring how the selected design choices affect performance and alignment with legal-domain characteristics.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":29,"slug":121},"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]