[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118063-en":3,"doc-seo-118063-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118063,137441390410,"Hazel","https://ap-avatar.wpscdn.com/avatar/2000252f4ab5702993?_k=1776741390130283984",8,"Research & Report","Taking the Law More Seriously by Investigating Design Choices in Machine Learning Prediction Research","Court case prediction using machine learning varies widely in both predictive success and legal reasonableness. This work examines how design choices shape outcomes in the legal domain, where tasks such as justification are difficult for ML and legal data and reasoning behave differently from typical assumptions. Four models predict cases from the European Court of Human Rights dataset to measure effects of performance metric, case-part inclusion, legal focus specialization, and temporal dependence on prior decisions, aiming to clarify limitations and alignment with legal characteristics.","University of Groningen  \nTaking the Law More Seriously by Investigating Design Choices in Machine Learning Prediction Research  \nSteging, Cor; Renooij, Silja; Verheij, Bart  \nPublished in:  \n6th Workshop on Automated Semantic Analysis of Information in Legal Text, ASAIL 2023  \nIMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document version below.  \nDocument Version  \nPublisher's PDF, also known as Version of record  \nPublication date: 2023  \nLink to publication in University of Groningen/UMCG research database  \nCitation for published version (APA):  \nSteging, C. , Renooij, S. , & Verheij, B. (2023) . Taking the Law More Seriously by Investigating Design Choices in Machine Learning Prediction Research. In 6th Workshop on Automated Semantic Analysis of Information in Legal Text, ASAIL 2023 (pp. 49-59) . (CEUR Workshop Proceedings; Vol. 3441) . CEUR Workshop Proceedings.  \nCopyright  \nOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons) .  \nThe publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license. More information can be found on the University of Groningen website: [https://www.rug.nl/library/open-access/self-archiving-pure/taverne](https://www.rug.nl/library/open-access/self-archiving-pure/taverne)amendment.  \nTake-down policy  \nIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.  \nDownloaded from the University of Groningen/U MCG research database (Pure): [http://www.rug. nl/research/portal. For technical reasons the](http://www.rug. nl/research/portal. For technical reasons the)[ ](http://www.rug. nl/research/portal. For technical reasons the)[number of authors shown on this cover page is limited to 10 maximum.](number of authors shown on this cover page is limited to 10 maximum.)  \nDownload date: 01-08-2026  \nTaking the Law More Seriously by Investigating Design Choices in Machine Learning Prediction Research  \nCor Steging1, * , Silja Renooij2 and Bart Verheij1  \n1 Bernoulli Institute of Mathematics, Computer Science and Artificial Intelligence, University of Groningen  \n2 Department of Information and Computing Sciences, Utrecht University  \nAbstract  \nApproaches to court case prediction using machine learning differ widely with varying levels of success and legal reasonableness. In part this is due to some aspects of law, such as justification, being inherently difficult for machine learning approaches. Another aspect is the effect of design choices and the extent to which these are legally reasonable, which has not yet been extensively studied. We create four machine learning models tasked with predicting cases from the European Court of Human Rights and we perform experiments in order to measure the role of the following four design choices and effects: the choice of performance metric; the effect of including different parts of the legal case; the effect of a more or less specialized legal focus; and the temporal effects of the available past legal decisions. Through this research, we aim to study design decisions and their limitations and how they affect the performance of machine learning models.  \nKeywords  \nCourt case prediction, design choices, machine learning  \n1. Introduction  \nRecently, much work has been done in the field of court case predictions. While automatically determining the outcome of court cases remains an academic exercise, the large variation in the ways that previous research has tackled the problem makes it nearly impossible to compare the approaches [1] . The law has unique characteristics, making it difficult to ap","cbCaifn9XInGVgfh","https://ap.wps.com/l/cbCaifn9XInGVgfh","pdf",1656872,1,12,"English","en",105,"# Introduction\n## Design choices and legal domain challenges\n# Model design and experiments\n## Performance metric and case-part inclusion\n## Legal focus specialization and temporal effects","[{\"question\":\"Why is court case prediction with machine learning difficult to do in a legally reasonable way?\",\"answer\":\"Legal tasks include characteristics such as justification that are inherently difficult for machine learning systems. The legal domain also differs from common ML assumptions and expects arguments that change over time.\"},{\"question\":\"What four design choices does the research evaluate in machine learning prediction models?\",\"answer\":\"The study measures (1) the choice of performance metric, (2) the effect of including different parts of the legal case, (3) the effect of using a more or less specialized legal focus, and (4) temporal effects of available past legal decisions.\"},{\"question\":\"Which dataset and task are used to test the proposed models?\",\"answer\":\"The experiments build four machine learning models tasked with predicting cases using data from the European Court of Human Rights, allowing assessment of how design choices affect model performance.\"}]","Taking the Law More Seriously by Investigating Design Choices in Machine Learning Prediction Research | PDF",1785681165,30,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"taking-the-law-more-seriously-by-investigating-design-choices-in-machine-learning-prediction-research","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/taking-the-law-more-seriously-by-investigating-design-choices-in-machine-learning-prediction-research/118063/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is court case prediction with machine learning difficult to do in a legally reasonable way?","Question",{"text":75,"@type":76},"Legal tasks include characteristics such as justification that are inherently difficult for machine learning systems. The legal domain also differs from common ML assumptions and expects arguments that change over time.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What four design choices does the research evaluate in machine learning prediction models?",{"text":80,"@type":76},"The study measures (1) the choice of performance metric, (2) the effect of including different parts of the legal case, (3) the effect of using a more or less specialized legal focus, and (4) temporal effects of available past legal decisions.",{"name":82,"@type":73,"acceptedAnswer":83},"Which dataset and task are used to test the proposed models?",{"text":84,"@type":76},"The experiments build four machine learning models tasked with predicting cases using data from the European Court of Human Rights, allowing assessment of how design choices affect model performance.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":29,"slug":121},"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]