DeepSeek-Lite

About

What DeepSeek-Lite is

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model 1. Introduction Last week, the release and buzz around DeepSeek-V2 have ignited widespread interest in MLA (Multi-head Latent Attention)! Many in the community suggested open-sourcing a smaller MoE model for in-depth research. And now DeepSeek-V2-Lite comes out: - 16B total params, 2.4B active params, scratch training with 5.7T tokens - Outperforms 7B dense and 16B MoE on many English & Chinese benchmarks - Deployable on single 40G GPU, fine-tunable on 8x80G GPUs DeepSeek-V2, a strong Mixture-of-Experts (MoE) language model characterized by economical training and efficient inference. DeepSeek-V2 adopts innovative architectures including Multi-head Latent Attention (MLA) and DeepSeekMoE. MLA guarantees efficient inference through significantly compressing the Key-Value (KV) cache into a latent vector, while DeepSeekMoE enables training strong models at an economical cost through sparse computation. 2. News - 2024.05.16: We released the DeepSeek-V2-Lite. - 2024.05.06: We released the DeepSeek-V2. 3. Model Downloads With DeepSeek-V2, we are open-sourcing base and chat models across two sizes: Due to the constraints of HuggingFace, the open-source code currently experiences slower performance than our internal codebase when running on GPUs with Huggingface. To facilitate the efficient execution of our model, we offer a dedicated vllm solution that optimizes performance for running our model effectively. 4. Evaluation Results Base Model Standard Benchmark For more evaluation details, such as few-shot settings and prompts, please check our paper. Chat Model Standard Benchmark 5. Model Architecture DeepSeek-V2 adopts innovative architectures to guarantee economical training and efficient inference： - For attention, we design MLA (Multi-head Latent Attention), which utilizes low-rank key-value union compression to eliminate the bottleneck of inference-time key-value cache, thus supporting efficient inference. - For Feed-Forward Networks (FFNs), we adopt DeepSeekMoE architecture, a high-performance MoE architecture that enables training stronger models at lower costs. DeepSeek-V2-Lite has

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys deepseek-lite for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (deepseek-lite below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"deepseek-lite","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Papers

Licence

Commercial licence needed

Sources

Weights & code

Explore

More language models