[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118641-en":3,"doc-seo-118641-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118641,1099514068365,"Aurelia","https://ap-avatar.wpscdn.com/avatar/10000253d8d9f28188e?_k=1776742907772140068",8,"Research & Report","Taking the Law More Seriously - Investigating Design Choices in Machine Learning Prediction Research","Approaches to court case prediction using machine learning vary substantially in both success and legal reasonableness, in part because law—especially justification—is difficult for machine learning methods. A further factor is how modeling design choices affect legal acceptability, which remains insufficiently studied. This work builds four machine learning models for predicting European Court of Human Rights cases and runs experiments to quantify the impact of evaluation metrics, case evidence inclusion, specialization of legal focus, and temporal effects of prior decisions on model performance and legal alignment.","University of Groningen  \nTaking the Law More Seriously by Investigating Design Choices in Machine Learning Prediction Research  \nSteging, Cor; Renooij, Silja; Verheij, Bart  \nPublished in:  \n6th Workshop on Automated Semantic Analysis of Information in Legal Text, ASAIL 2023  \nIMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document version below.  \nDocument Version  \nPublisher's PDF, also known as Version of record  \nPublication date: 2023  \nLink to publication in University of Groningen/UMCG research database  \nCitation for published version (APA):  \nSteging, C. , Renooij, S. , & Verheij, B. (2023) . Taking the Law More Seriously by Investigating Design Choices in Machine Learning Prediction Research. In 6th Workshop on Automated Semantic Analysis of Information in Legal Text, ASAIL 2023 (pp. 49-59) . (CEUR Workshop Proceedings; Vol. 3441) . CEUR Workshop Proceedings.  \nCopyright  \nOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons) .  \nThe publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license. More information can be found on the University of Groningen website: [https://www.rug.nl/library/open-access/self-archiving-pure/taverne](https://www.rug.nl/library/open-access/self-archiving-pure/taverne)amendment.  \nTake-down policy  \nIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.  \nDownloaded from the University of Groningen/U MCG research database (Pure): [http://www.rug. nl/research/portal. For technical reasons the](http://www.rug. nl/research/portal. For technical reasons the)[ ](http://www.rug. nl/research/portal. For technical reasons the)[number of authors shown on this cover page is limited to 10 maximum.](number of authors shown on this cover page is limited to 10 maximum.)  \nDownload date: 29-12-2025  \nTaking the Law More Seriously by Investigating Design Choices in Machine Learning Prediction Research  \nCor Steging1, * , Silja Renooij2 and Bart Verheij1  \n1 Bernoulli Institute of Mathematics, Computer Science and Artificial Intelligence, University of Groningen  \n2 Department of Information and Computing Sciences, Utrecht University  \nAbstract  \nApproaches to court case prediction using machine learning differ widely with varying levels of success and legal reasonableness. In part this is due to some aspects of law, such as justification, being inherently difficult for machine learning approaches. Another aspect is the effect of design choices and the extent to which these are legally reasonable, which has not yet been extensively studied. We create four machine learning models tasked with predicting cases from the European Court of Human Rights and we perform experiments in order to measure the role of the following four design choices and effects: the choice of performance metric; the effect of including different parts of the legal case; the effect of a more or less specialized legal focus; and the temporal effects of the available past legal decisions. Through this research, we aim to study design decisions and their limitations and how they affect the performance of machine learning models.  \nKeywords  \nCourt case prediction, design choices, machine learning  \n1. Introduction  \nRecently, much work has been done in the field of court case predictions. While automatically determining the outcome of court cases remains an academic exercise, the large variation in the ways that previous research has tackled the problem makes it nearly impossible to compare the approaches [1] . The law has unique characteristics, making it difficult to ap","cbCait0fRXWlBKmu","https://ap.wps.com/l/cbCait0fRXWlBKmu","pdf",1656555,1,12,"English","en",105,"# Introduction\n## Research focus and motivation\n## Design choices and experimental evaluation","[{\"question\":\"Why is court case prediction difficult for machine learning approaches?\",\"answer\":\"Legal reasoning has unique properties: law is prospective and changes over time, includes erroneous or evolving decisions, and requires justification and argumentation. Machine learning is typically retrospective and relies on assumptions such as homogeneous, error-free data and limited ability to explain decisions.\"},{\"question\":\"What are the four design choices examined in this research?\",\"answer\":\"The study evaluates how performance metric choice affects results, how including different parts of the legal case impacts outcomes, how a more specialized versus less specialized legal focus changes performance, and how temporal effects from available past legal decisions influence predictions.\"},{\"question\":\"How is the research set up to measure the effect of design choices?\",\"answer\":\"Four machine learning models are created to predict cases from the European Court of Human Rights. Experiments are then performed to measure the role and effects of the specified design choices on prediction performance and alignment with legal-domain characteristics.\"}]","Taking the Law More Seriously - Investigating Design Choices in Machine Learning Prediction Research | PDF",1785684660,30,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"taking-the-law-more-seriously-investigating-design-choices-in-machine-learning-prediction-research","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/taking-the-law-more-seriously-investigating-design-choices-in-machine-learning-prediction-research/118641/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is court case prediction difficult for machine learning approaches?","Question",{"text":75,"@type":76},"Legal reasoning has unique properties: law is prospective and changes over time, includes erroneous or evolving decisions, and requires justification and argumentation. Machine learning is typically retrospective and relies on assumptions such as homogeneous, error-free data and limited ability to explain decisions.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What are the four design choices examined in this research?",{"text":80,"@type":76},"The study evaluates how performance metric choice affects results, how including different parts of the legal case impacts outcomes, how a more specialized versus less specialized legal focus changes performance, and how temporal effects from available past legal decisions influence predictions.",{"name":82,"@type":73,"acceptedAnswer":83},"How is the research set up to measure the effect of design choices?",{"text":84,"@type":76},"Four machine learning models are created to predict cases from the European Court of Human Rights. Experiments are then performed to measure the role and effects of the specified design choices on prediction performance and alignment with legal-domain characteristics.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":29,"slug":121},"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]