[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-138559-105":59,"doc-detail-138559-en":130},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":123,"head_meta":125,"extra_data":127,"updated_unix":129},105,"en","continuous-arcade-learning-environment-cale-initial-paper","Continuous Arcade Learning Environment (CALE) - Initial paper","","Continuous Arcade Learning Environment (CALE) extends the Arcade Learning Environment (ALE) by keeping the Atari 2600 emulator while adding support for continuous actions. CALE enables benchmarking and evaluation of continuous-control agents and value-based agents on the same suite, facilitating study of action-space effects. The paper provides open research directions and initial baseline results using Soft Actor-Critic, and shares CALE as part of the ALE platform.",{"@graph":69,"@context":122},[70,84,105],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/continuous-arcade-learning-environment-cale-initial-paper/138559/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/continuous-arcade-learning-environment-cale-initial-paper/138559.png","ImageObject",300,407,{"name":92,"@type":93},"Sage","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-09-18","2026-08-23",true,{"@type":102,"interactionType":103,"userInteractionCount":29},"InteractionCounter",{"@type":104},"ViewAction",{"@type":106,"mainEntity":107},"FAQPage",[108,114,118],{"name":109,"@type":110,"acceptedAnswer":111},"What problem does CALE address compared with the original ALE?","Question",{"text":112,"@type":113},"CALE extends ALE by adding continuous actions while keeping the same Atari 2600 emulator, enabling unified evaluation across discrete and continuous control settings.","Answer",{"name":115,"@type":110,"acceptedAnswer":116},"Which kinds of agents can be evaluated using CALE?",{"text":117,"@type":113},"CALE supports continuous-control agents and also value-based agents on the same environment suite, enabling comparisons under a continuous action formulation.",{"name":119,"@type":110,"acceptedAnswer":120},"What baseline method is used for initial results in the paper?",{"text":121,"@type":113},"The paper reports initial baseline results using Soft Actor-Critic (SAC).","https://schema.org",{"og:url":83,"og:type":124,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":126,"canonical":83},"index,follow",{"doc_id":128,"site_id":62},138559,1787483826,{"code":4,"msg":5,"data":131},{"doc_id":128,"user_id":132,"nickname":92,"user_avatar":133,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":29,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":46,"language":139,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":140,"faqs":141,"seo_title":142,"seo_description":67,"update_tm":129,"read_time":31},687197207057,"https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0","CALE: Continuous Arcade Learning Environment  \narXiv :2410 .23810v1 [ cs .LG] 31 Oct 2024  \nJesse Farebrother  \nMcGill University Mila-Québec AI Institute Google DeepMind [jfarebro@cs.mcgill.ca](jfarebro@cs.mcgill.ca)  \nPablo Samuel Castro  \nGoogle DeepMind Université de Montréal Mila-Québec AI Institute [psc@google.com](psc@google.com)  \nAbstract  \nWe introduce the Continuous Arcade Learning Environment (CALE), an extension of the well-known Arcade Learning Environment (ALE) [Bellemare et al., 2013] . The CALE uses the same underlying emulator of the Atari 2600 gaming system (Stella), but adds support for continuous actions. This enables the benchmarking and evaluation of continuous-control agents (such as PPO [Schulmanet al., 2017] and SAC [Haarnoja et al., 2018]) and value-based agents (such as DQN [Mnih et al., 2015] and Rainbow [Hessel et al., 2018]) on the same environment suite. We provide a series of open questions and research directions that CALE enables, as well as initial baseline results using Soft Actor-Critic. CALE is available as part of the ALE at [https://github.com/Farama-Foundation/](https://github.com/Farama-Foundation/)[ ](https://github.com/Farama-Foundation/)Arcade-Learning-Environment.  \n1 Introduction  \nGenerally capable autonomous agents have been a principal objective of machine learning research, and in particular reinforcement learning, for many decades. General in the sense that they can handle a variety of challenges; capable in that they are able to “solve” or perform well on these challenges; and they are able to learn autonomously by interacting with the system or problem by exercising their agency (e.g. making their own decisions) . While deploying and testing on real systems is the ultimate goal, researchers usually rely on academic benchmarks to showcase their proposed methods. It is thus crucial for academic benchmarks to be able to test generality, capability, and autonomy. Bellemare et al. [2013] introduced the Arcade Learning Environment (ALE) as one such benchmark. The ALE is a collection of challenging and diverse Atari 2600 games where agents learn by directly playing the games; as input, agents receive a high dimensional observation (the “pixels” on the screen), and as output they select from one of 18 possible actions (see Section 2) . While some research had already been conducted on a few isolated Atari 2600 games [Cobo et al., 2011, Hausknecht et al., 2012, Bellemare et al., 2012], the ALE’s significance was to provide a unified platform for research and evaluation across more than 100 games. Using the ALE, Mnih et al. [2015] demonstrated, for the first time, that reinforcement learning (RL) combined with deep neural networks could play challenging Atari 2600 games with super-human performance. Much like how ImageNet [Deng et al., 2009] ushered in the era of Deep Learning [LeCun et al., 2015], the Arcade Learning Environment spawned the advent of Deep Reinforcement Learning.  \nIn addition to becoming one of the most popular benchmarks for evaluating RL agents, the ALE has also evolved with new extensions, including stochastic transitions [Machado et al., 2018], various game modes and difficulties [Machado et al., 2018, Farebrother et al., 2018], and multi-player support [Terry and Black, 2020] . What has remained constant is the suitability of this benchmark for testing generality (there is a wide diversity of games), capability (many games still prove challenging formost modern agents), and agency (learning typically occurs via playing the game) .  \n38th Conference on Neural Information Processing Systems (NeurIPS 2024) Track on Datasets and Benchmarks.  \nFigure 1: Left panel: Atari CX10 controller. Right panel: Discrete joystick positions (top left) versus continuous joystick positions with varying values of the threshold τ . The black circle corresponds toa joystick at position (r,θ) = (0 .61 , 2.53) .  \nThere are a number of design choices that have become standard when eva","cbCaiuuJawIJf7BG","https://ap.wps.com/l/cbCaiuuJawIJf7BG","pdf",3370054,"English","# Introduction\n## From Atari VCS to the Arcade Learning Environment","[{\"question\":\"What problem does CALE address compared with the original ALE?\",\"answer\":\"CALE extends ALE by adding continuous actions while keeping the same Atari 2600 emulator, enabling unified evaluation across discrete and continuous control settings.\"},{\"question\":\"Which kinds of agents can be evaluated using CALE?\",\"answer\":\"CALE supports continuous-control agents and also value-based agents on the same environment suite, enabling comparisons under a continuous action formulation.\"},{\"question\":\"What baseline method is used for initial results in the paper?\",\"answer\":\"The paper reports initial baseline results using Soft Actor-Critic (SAC).\"}]","Continuous Arcade Learning Environment (CALE) - Initial paper | PDF"]