GLM-4.5

About

GLM-4.5 is a large-scale language model developed with enhanced reasoning capacity, utilizing advanced architectural components like Grouped-Query Attention with partial RoPE and 2.5 times more attention heads. It employs the Muon optimizer for faster convergence and QK-Norm for stability. The model includes a Multi-Token Prediction (MTP) layer to support speculative decoding during inference. Its training involves multiple stages: 15T tokens of general pre-training, 7T tokens of a code & reasoning corpus, followed by domain-specific fine-tuning on instruction data. For efficient Reinforcement Learning (RL) training, GLM-4.5 leverages 'slime', an open-sourced RL infrastructure designed for flexibility, efficiency, and scalability, featuring a hybrid training architecture, decoupled agent-oriented design, and accelerated data generation with mixed precision. Post-training RL further enhances agentic capabilities (coding, deep search, general tool-using) and reasoning, using a difficulty-based curriculum and verifiable tasks. The model also incorporates an optimized user simulator for TAU-Bench.