Llama 2 - Open Foundation and Fine-Tuned Chat Models - arXiv 2307.09288v2

Sep 17, 2026

Uploaded by William

Llama 2 presents a release of pretrained and fine-tuned large language models spanning 7B to 70B parameters. The fine-tuned models, Llama 2-Chat, are optimized for dialogue and evaluated against open-source chat systems across most tested benchmarks. Human evaluations for helpfulness and safety indicate the models may serve as substitutes for closed-source alternatives. The work documents the fine-tuning procedure and the safety improvements used to support responsible development by the community.

LanguageEnglish

CategoryPresentations

FormatPDF

Pages77

Reading Time27 min

AccessFree to download

Introduction

Pretraining

Fine-tuning

Safety

Discussion

Related Work

Conclusion

Appendix

arXiv :2307 .09288v2 [ cs .CL] 19 Jul 2023 Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron∗ Louis Martin† Kevin Stone† Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale Dan Bikel Lukas Blecher Cristian Canton Ferrer Moya Chen Guillem Cucurull David Esiobu Jude Fernandes Jeremy Fu Wenyin Fu Brian Fuller Cynthia Gao Vedanuj Goswami Naman Goyal Anthony Hartshorn Saghar Hosseini Rui Hou Hakan Inan Marcin Kardas Viktor Kerkez Madian Khabsa Isabel Kloumann Artem Korenev Punit Singh Koura Marie-Anne Lachaux Thibaut Lavril Jenya Lee Diana Liskovich Yinghai Lu Yuning Mao Xavier Martinet Todor Mihaylov Pushkar Mishra Igor Molybog Yixin Nie Andrew Poulton Jeremy Reizenstein Rashi Rungta Kalyan Saladi Alan Schelten Ruan Silva Eric Michael Smith Ranjan Subramanian Xiaoqing Ellen Tan Binh Tang Ross Taylor Adina Williams Jian Xiang Kuan Puxin Xu Zheng Yan Iliyan Zarov Yuchen Zhang Angela Fan Melanie Kambadur Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom∗ GenAI, Meta Abstract In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our fine-tuned LLMs, called Llama 2-Chat, are optimized for dialogue use cases. Our models outperform open-source chat models on most benchmarks we tested, and based on our human evaluations for helpfulness and safety, may be a suitable substitute for closedsource models. We provide a detailed description of our approach to fine-tuning and safety improvements of Llama 2-Chat in order to enable the community to build on our work and contribute to the responsible development of LLMs. ∗ Equal contribution, corresponding authors: {tscialom, [htouvron}@meta.com](htouvron}@meta.com)[ ](htouvron}@meta.com)†Second author Contributions for all the authors can be found in Section A.1. Contents 1 Introduction 3 2 Pretraining 5 2.1 Pretraining Data ............................................. 5 2.2 Training Details ............................................. 5 2.3 Llama 2 Pretrained Model Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3 Fine-tuning 8 3.1 Supervised Fine-Tuning (SFT) ..................................... 9 3.2 Reinforcement Learning with Human Feedback (RLHF) ..................... 9 3.3 System Message for Multi-Turn Consistency ............................. 16 3.4 RLHF Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 4 Safety 20 4.1 Safety in Pretraining .......................................... 20 4.2 Safety Fine-Tuning ........................................... 23 4.3 Red Teaming ............................................... 28 4.4 Safety Evaluation of Llama 2-Chat .................................. 29 5 Discussion 32 5.1 Learnings and Observations ...................................... 32 5.2 Limitations and Ethical Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 5.3 Responsible Release Strategy ..................................... 35 6 Related Work 35 7 Conclusion 36 A Appendix 46 A. 1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 A.2 Additional Details for Pretraining ................................... 47 A.3 Additional Details for Fine-tuning .................................. 51 A.4 Additional Details for Safety ...................................... 58 A.5 Data Annotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 A.6 Dataset Contamination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 A.7 Model Card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 Figure 1: Helpfulness human evaluat

FAQs

What is Llama 2 and what model sizes are included?

Llama 2 is a collection of pretrained and fine-tuned large language models. It ranges from 7 billion to 70 billion parameters.

What is the purpose of Llama 2-Chat?

Llama 2-Chat is fine-tuned and optimized for dialogue use cases. It is designed for chat-oriented interaction.

How does the paper improve safety in Llama 2-Chat?

Safety is addressed through safety-focused fine-tuning and evaluation, including human feedback approaches and red teaming, followed by safety evaluation of Llama 2-Chat.

Sep 17, 2026

Uploaded by William