DeepSeek-V2

Combined Multi-head Latent Attention with fine-grained MoE routing to reduce training and inference cost.

LegacyLanguageOpen weights

MODEL NUMBER016

Specifications

Only published specifications are shown.

Total parameters

236B

Active parameters

21B

Architecture

MoE

Context

128K

Input modalities

Text

Open weights

Yes

License

DeepSeek License

Status

Legacy

Combined Multi-head Latent Attention with fine-grained MoE routing to reduce training and inference cost.

DeepSeek-V2 showed that efficient sparse architecture could compete on both capability and cost in an open-weight model.

MLA

Long contextCoding

2024-05-06Open weights

DeepSeek-V2 released model weights and its technical report.

Select a card to view the related model.