AI-ML
PyTorch v1.3
RESUMEN
TL;DR We recently demonstrated a +30.2% training speedup for Llama4 Scout with equivalent convergence to bfloat16, by using MXFP8 MoE training primitives in TorchAO! This is ~81% of the theoretical...
Descripción Detallada
TL;DR We recently demonstrated a +30.2% training speedup for Llama4 Scout with equivalent convergence to bfloat16, by using MXFP8 MoE training primitives in TorchAO! This is ~81% of the theoretical...
Mejoras en la velocidad de entrenamiento para Llama4 Scout usando nuevas primitivas.
- Aumento del 30.2% en la velocidad de entrenamiento para Llama4 Scout.
- Convergencia equivalente a bfloat16.
- Uso de primitivas de entrenamiento MXFP8 MoE en TorchAO.
A quién le importa
Todos los que usan Llama4 Scout para entrenamiento.
Generado por IA · puede contener errores
Releases Relacionados
AI-ML
PyTorch v2.11
We are excited to announce the release of PyTorch® 2.11 (release notes)! The PyTorch 2.11 release features the following changes: Differentiable Collectives for Distributed Training FlexAttention now has a FlashAttention-4...
AI-ML
PyTorch v0.0.0
The world of AI is expanding beyond the cloud, reaching devices that fit in the palm of your hand. Running PyTorch models on these tiny systems, where memory is measured...
AI-ML