[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82343-en":3,"doc-seo-82343-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82343,1374391974585,"Genevieve","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","A Sovereign, Open-Source Foundation Model for German and English","Soofi S 30B-A3B is a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba Transformer foundation model for German and English. Only 3B of 30B parameters are activated per token, while an inference cache remains near-constant as context length grows, delivering superior throughput for long-context, high-concurrency deployment. Trained on roughly 27T tokens with deliberately up-weighted German, it matches dense 14–27B aggregate benchmarks and leads code aggregates among 17 open base models, exceeding European sovereign baselines.","arXiv :2607 .09424v 1 [ cs .CL] 10 Jul 2026  \nA Sovereign, Open-Source Foundation Model for German and English  \nSoofi S Pretraining Report v1.0  \nThe Soofi-Team*  \nCore Team: Benedikt Droste 10 , David Fitzek3,9 , Ruben Härle5 , Lukas Helff2,5 , Maximilian Idahl 10 , Alex Jude3,9 , Abbas Goher Khan3 , Maurice Kraus5 , Timm Ruland3,9 , Richard Rutmann3,9 , Sebastian Sztwiertnia5  \nContributors: Markus Frey3,9 , Daniil Gurgurov2 , Jan Pfister6 , Tom Röhr7 , Sebastian von Rohrscheidt7  \nAdvisors: Jörg Bienert 1 , Nicolas Flores-Herr3 , Simon Gottschalk8 , Andreas Hotho6 , Kristian Kersting2,5,11 , Joachim Köhler3 , Alexander Löser7 , Wolfgang Nejdl8 , Simon Ostermann2 , Jan Plogsties4 , Patrick Putzky 12  \nTechnical Leads: Mehdi Ali3,9 , Michael Fromm3,9 , Max Lübbering3,9  \nAffiliations: 1 KI Bundesverband, 2 DFKI, 3 Fraunhofer IAIS, 4 Fraunhofer IIS, 5 Technische Universität Darmstadt, 6 Universität Würzburg, 7 Berliner Hochschule für Technik, 8 L3S Research Center, 9 Lamarr, 10 ellamind, 11 hessian.AI, 12 Merantix Momentum  \nCoordination & Funding: Consortium coordinated by the KI Bundesverband. Funded by the German Federal Ministry for Economic Affairs and Energy (BMWE) .  \n∗ Authors are listed alphabetically. Detailed Contributions in Appendix A.  \nAbstract  \nWe present Soofi S 30B-A3B, a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba Transformer foundation model for German and English. Its hybrid design activates only 3B of 30B parameters per token and keeps the inference cache near-constant as context grows, giving it a decisive throughput advantage over dense models for long-context, high-concurrency deployment. Pretrained on roughly 27 trillion tokens with deliberately up-weighted German, Soofi S matches dense 14 to 27B models on aggregate English and German benchmarks while achieving the best code aggregates in both languages among 17 open base models, and outperforms every European sovereign baseline in our comparison, including ones far larger in active parameters. Among fully open models, Soofi S obtains the highest English and German evaluation scores, ahead of Olmo 3 32Band Apertus 70B. Soofi S was built end-to-end on the German Industrial AI Cloud, a sovereign HPCscale AI infrastructure operated by Deutsche Telekom in Munich. Soofi S will be released under highly permissive, open-access terms: weights, selected intermediate checkpoints 1 , full per-source data accounting, hyperparameters, and training and evaluation code. Where source licenses permit, data-construction artifacts are released under permissive licenses; commercially licensed sources are documented with aggregate statistics and exact mixture accounting.  \n1 Introduction  \n100  \nCapability Index  \n90  \n80  \n70  \n60  \n50  \n40  \n25 50 100 200 400 800 1600 3200 6400 Aggregate decode TPS/GPU at 40K context  \n(a) Capability vs. measured aggregate decode TPS.  \n3200  \nAggregate decode TPS/GPU  \n1600  \n800  \n400  \n200  \n100  \n50  \n25  \n4K 8K 16K 32K 64K 128K 256K  \nContext length (tokens)  \n(b) Aggregate decode TPS scaling with context.  \nFigure 1: Long-context serving efficiency. Soofi S combines frontier-level capability with the highest measured aggregate long-context decode TPS, and unlike full-attention dense baselines maintains high throughput as context grows. Panel (a) plots Capability Index versus measured aggregate decode TPS/GPU at 40K context and batch 32 . The Capability Index averages five benchmark groups, i.e., Code, GSM8K, GPQA-Diamond, English aggregate, and German aggregate, after normalizing each group to the best plotted model. Aggregate decode TPS/GPU is measured with a TP=1, one-B200 vLLM latency-subtraction protocol. Panel (b) shows measured aggregate decode TPS/GPU as a function of input context length under the same batch-32 protocol. For both panels higher is better.  \nOpen language models have improved at remarkable speed, yet three gaps remain conspicuous for anyone deciding what to actually deploy.  \n1[https://hug","cbCaivKWRy0d4ZND","https://ap.wps.com/l/cbCaivKWRy0d4ZND","pdf",1578806,1,59,"English","en",105,"# Abstract\n# Introduction","[{\"question\":\"What makes Soofi S 30B-A3B different from dense foundation models for long context?\",\"answer\":\"Soofi S activates only 3B of 30B parameters per token and keeps the inference cache near-constant as context grows, preserving throughput when serving long sequences with high concurrency.\"},{\"question\":\"How was Soofi S trained and how is language coverage handled?\",\"answer\":\"The model was pretrained on roughly 27 trillion tokens with German deliberately up-weighted, aiming to match or exceed dense models on aggregated English and German benchmarks while improving language balance for German.\"},{\"question\":\"What openness and release artifacts are planned for Soofi S?\",\"answer\":\"Soofi S will be released under highly permissive, open-access terms, including model weights, selected intermediate checkpoints, full per-source data accounting, hyperparameters, and training and evaluation code, with reconstruction artifacts released when source licenses permit.\"}]",1784179777,149,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"a-sovereign-open-source-foundation-model-for-german-and-english","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/a-sovereign-open-source-foundation-model-for-german-and-english/82343/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What makes Soofi S 30B-A3B different from dense foundation models for long context?","Question",{"text":74,"@type":75},"Soofi S activates only 3B of 30B parameters per token and keeps the inference cache near-constant as context grows, preserving throughput when serving long sequences with high concurrency.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How was Soofi S trained and how is language coverage handled?",{"text":79,"@type":75},"The model was pretrained on roughly 27 trillion tokens with German deliberately up-weighted, aiming to match or exceed dense models on aggregated English and German benchmarks while improving language balance for German.",{"name":81,"@type":72,"acceptedAnswer":82},"What openness and release artifacts are planned for Soofi S?",{"text":83,"@type":75},"Soofi S will be released under highly permissive, open-access terms, including model weights, selected intermediate checkpoints, full per-source data accounting, hyperparameters, and training and evaluation code, with reconstruction artifacts released when source licenses permit.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":105,"slug":137},19,"General","general"]