Introducing the Next Generation of Our Open Source Large Language Model

This latest release brings you the cutting-edge Llama language models, both pretrained and fine-tuned, ranging from 7B to 70B parameters, empowering a wide array of language-related tasks.

Llama 2 has been trained on 40% more data compared to Llama 1, and it boasts double the context length, enhancing its capabilities significantly.

Training Llama-2-chat involves a multi-step process. Initially, Llama 2 undergoes pretrained using publicly available online data. Subsequently, an initial version of Llama-2-chat is created through supervised fine-tuning. To refine it further, Llama-2-chat goes through iterative enhancements using Reinforcement Learning from Human Feedback (RLHF), which incorporates techniques like rejection sampling and proximal policy optimization (PPO).

Top Features:

• Llama 2 models trained on an extensive 2 trillion tokens with double the context length of Llama 1.
• Llama-2-chat models enriched with over 1 million new human annotations.
• Llama 2 surpasses other open source language models on multiple external benchmarks, demonstrating remarkable performance in reasoning, coding, proficiency, and knowledge tests.
• Llama-2-chat implements reinforcement learning from human feedback to ensure safety and helpfulness in its interactions.