DeepSeekMoE 16B

A 16.4B MoE using fine-grained expert segmentation and shared-expert isolation, released in Base and Chat variants.

LegacyLanguageCodingOpen weights

MODEL NUMBER023

Specifications

Only published specifications are shown.

Total parameters

16.4B

Architecture

MoE

Context

Input modalities

Text

Open weights

Yes

License

DeepSeek Model License

Status

Legacy

A 16.4B MoE using fine-grained expert segmentation and shared-expert isolation, released in Base and Chat variants.

DeepSeekMoE was the key intermediate step validating DeepSeek’s efficient sparse-expert path before V2.

moeEfficiencyMultilingual

2024-01-11Open weights

DeepSeekMoE 16B was released publicly, exploring efficient MoE through fine-grained and shared-expert routing.

Select a card to view the related model.

No related models recorded yet