DeepSeek-V3 Technical Report

DeepSeek-AI, 2024, DeepSeek-V3 Technical Report (arXiv preprint arXiv:2412.19437)

Subscriber deck — free during early access

1 cards in “1 Introduction” · whole deck

1 Introduction

What is DeepSeek-V3, in numbers?
A Mixture-of-Experts LLM with 671B parameters, of which 37B are active per token, pretrained on 14.8T tokens. Its full training took 2.788M H800 GPU hours (about $5.6M at $2 per GPU hour, excluding earlier research and ablations), with no irrecoverable loss spikes or rollbacks.
- DeepSeek-AI, 2024, DeepSeek-V3 Technical Report (arXiv preprint arXiv:2412.19437), §1

Source

DeepSeek-AI. 2024. DeepSeek-V3 Technical Report. In arXiv preprint arXiv:2412.19437. https://arxiv.org/abs/2412.19437