[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-138561-105":59,"doc-detail-138561-en":130},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":123,"head_meta":125,"extra_data":127,"updated_unix":129},105,"en","continuous-arcade-learning-environment-cale-neurips-2024-datasets-and-benchmarks-track","Continuous Arcade Learning Environment (CALE) - NeurIPS 2024 Datasets and Benchmarks Track","","Continuous Arcade Learning Environment (CALE) extends the well-known Arcade Learning Environment (ALE) by keeping the Atari 2600 emulator (Stella) while adding support for continuous actions. The same environment suite enables benchmarking and evaluation of continuous-control agents and value-based agents side by side. The paper outlines open questions and research directions enabled by CALE and reports initial baseline results using Soft Actor-Critic. CALE is released as part of ALE, supporting broader study of action-space challenges across Atari games.",{"@graph":69,"@context":122},[70,84,105],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/continuous-arcade-learning-environment-cale-neurips-2024-datasets-and-benchmarks-track/138561/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/continuous-arcade-learning-environment-cale-neurips-2024-datasets-and-benchmarks-track/138561.png","ImageObject",300,407,{"name":92,"@type":93},"River Wang","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-09-20","2026-08-23",true,{"@type":102,"interactionType":103,"userInteractionCount":34},"InteractionCounter",{"@type":104},"ViewAction",{"@type":106,"mainEntity":107},"FAQPage",[108,114,118],{"name":109,"@type":110,"acceptedAnswer":111},"What problem does CALE solve compared with ALE?","Question",{"text":112,"@type":113},"CALE extends ALE by adding continuous actions while retaining the Atari 2600 emulator. This allows evaluation beyond discrete 18-action settings.","Answer",{"name":115,"@type":110,"acceptedAnswer":116},"Which kinds of agents can be evaluated using CALE?",{"text":117,"@type":113},"CALE supports both continuous-control agents (e.g., PPO, SAC) and value-based agents (e.g., DQN, Rainbow) on the same benchmark suite.",{"name":119,"@type":110,"acceptedAnswer":120},"What baseline results are provided in the paper?",{"text":121,"@type":113},"The paper provides initial baseline results using Soft Actor-Critic (SAC) to demonstrate CALE’s usefulness for continuous action evaluation.","https://schema.org",{"og:url":83,"og:type":124,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":126,"canonical":83},"index,follow",{"doc_id":128,"site_id":62},138561,1787483852,{"code":4,"msg":5,"data":131},{"doc_id":128,"user_id":132,"nickname":92,"user_avatar":133,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":34,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":46,"language":139,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":140,"faqs":141,"seo_title":142,"seo_description":67,"update_tm":129,"read_time":31},1099514067438,"https://ap-avatar.wpscdn.com/avatar/100002539ee87300030?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780474512215547542","CALE: Continuous Arcade Learning Environment  \nJesse Farebrother  \nMcGill University Mila-Québec AI Institute Google DeepMind [jfarebro@cs.mcgill.ca](jfarebro@cs.mcgill.ca)  \nPablo Samuel Castro  \nGoogle DeepMind Université de Montréal Mila-Québec AI Institute [psc@google.com](psc@google.com)  \nAbstract  \nWe introduce the Continuous Arcade Learning Environment (CALE), an extension of the well-known Arcade Learning Environment (ALE) [Bellemare et al., 2013] . The CALE uses the same underlying emulator of the Atari 2600 gaming system (Stella), but adds support for continuous actions. This enables the benchmarking and evaluation of continuous-control agents (such as PPO [Schulmanet al., 2017] and SAC [Haarnoja et al., 2018]) and value-based agents (such as DQN [Mnih et al., 2015] and Rainbow [Hessel et al., 2018]) on the same environment suite. We provide a series of open questions and research directions that CALE enables, as well as initial baseline results using Soft Actor-Critic. CALE is available as part of the ALE at [https://github.com/Farama-Foundation/](https://github.com/Farama-Foundation/)[ ](https://github.com/Farama-Foundation/)Arcade-Learning-Environment.  \n1 Introduction  \nGenerally capable autonomous agents have been a principal objective of machine learning research, and in particular reinforcement learning, for many decades. General in the sense that they can handle a variety of challenges; capable in that they are able to “solve” or perform well on these challenges; and they are able to learn autonomously by interacting with the system or problem by exercising their agency (e.g. making their own decisions) . While deploying and testing on real systems is the ultimate goal, researchers usually rely on academic benchmarks to showcase their proposed methods. It is thus crucial for academic benchmarks to be able to test generality, capability, and autonomy. Bellemare et al. [2013] introduced the Arcade Learning Environment (ALE) as one such benchmark. The ALE is a collection of challenging and diverse Atari 2600 games where agents learn by directly playing the games; as input, agents receive a high dimensional observation (the “pixels” on the screen), and as output they select from one of 18 possible actions (see Section 2) . While some research had already been conducted on a few isolated Atari 2600 games [Cobo et al., 2011, Hausknecht et al., 2012, Bellemare et al., 2012], the ALE's significance was to provide a unified platform for research and evaluation across more than 100 games. Using the ALE, Mnih et al. [2015] demonstrated, for the first time, that reinforcement learning (RL) combined with deep neural networks could play challenging Atari 2600 games with super-human performance. Much like how ImageNet [Deng et al., 2009] ushered in the era of Deep Learning [LeCun et al., 2015], the Arcade Learning Environment spawned the advent of Deep Reinforcement Learning.  \nIn addition to becoming one of the most popular benchmarks for evaluating RL agents, the ALE has also evolved with new extensions, including stochastic transitions [Machado et al., 2018], various game modes and difficulties [Machado et al., 2018, Farebrother et al., 2018], and multi-player support [Terry and Black, 2020] . What has remained constant is the suitability of this benchmark for testing generality (there is a wide diversity of games), capability (many games still prove challenging formost modern agents), and agency (learning typically occurs via playing the game) .  \n38th Conference on Neural Information Processing Systems (NeurIPS 2024) Track on Datasets and Benchmarks.  \nFigure 1: Left panel: Atari CX10 controller. Right panel: Discrete joystick positions (top left) versus continuous joystick positions with varying values of the threshold 􀀜 . The black circle corresponds to a joystick at position (r; 􀀒) = (0:61; 2:53) .  \nThere are a number of design choices that have become standard when evaluating agents on the ALE, and which affect the o","cbCairqKAFDmm7Lq","https://ap.wps.com/l/cbCairqKAFDmm7Lq","pdf",3264187,"English","# Abstract\n# Introduction\n# From Atari VCS to the Arcade Learning Environment","[{\"question\":\"What problem does CALE solve compared with ALE?\",\"answer\":\"CALE extends ALE by adding continuous actions while retaining the Atari 2600 emulator. This allows evaluation beyond discrete 18-action settings.\"},{\"question\":\"Which kinds of agents can be evaluated using CALE?\",\"answer\":\"CALE supports both continuous-control agents (e.g., PPO, SAC) and value-based agents (e.g., DQN, Rainbow) on the same benchmark suite.\"},{\"question\":\"What baseline results are provided in the paper?\",\"answer\":\"The paper provides initial baseline results using Soft Actor-Critic (SAC) to demonstrate CALE’s usefulness for continuous action evaluation.\"}]","Continuous Arcade Learning Environment (CALE) - NeurIPS 2024 Datasets and Benchmarks Track | PDF"]