Grok 2.5 (OSS Ver.) by xAI is the open-source iteration of their state-of-the-art large-scale model from 2024, now available for developers and researchers. With a size of approximately 500 GB, the model weights are distributed under the Grok 2 Community License Agreement. Designed for high-performance AI tasks, Grok 2.5 requires the SGLang inference engine (version 0.5.1 or later) for deployment and supports configurations with 8 GPUs (each with over 40GB of memory) using tensor parallelism (TP=8). The model utilizes advanced features such as FP8 quantization and Triton-based attention mechanisms to optimize performance. Users can interact with the model via predefined chat templates, enabling seamless post-trained inference.
Information shown may be outdated. Found an error? Report it here
Auto-fetched from GitHub today.
