K-EXAONE-2.0-DSpark

Introduction

We introduce K-EXAONE 2.0, a frontier-scale multilingual language model developed by LG AI Research. K-EXAONE 2.0 was scaled to more than three times the size of its predecessor through upcycling, followed by continual pretraining, difficulty-focused mid-training, and post-training. K-EXAONE 2.0 is broadly competitive with leading open-weight models, demonstrating substantial improvements over its predecessor and achieving particularly strong results in long-context retrieval and safety.

Highlights

• Frontier-Class Scale To build a large-scale foundation model with frontier-level intelligence, we upcycled the K-EXAONE model by expanding both its depth and width, resulting in a more favorable scaling curve. During this process, we found that clamping after two SwiGLU branches effectively mitigates the exploding activations in deeper layers, improving both training and inference stability.

• Advanced Reasoning & Agentic Intelligence In response to the growth of agentic AI, we focused on expanding the model's capabilities in reasoning, agentic workflows, and long-context management. Through careful calibration of the training data and recipes, K-EXAONE 2.0 achieves consistent improvements in agentic coding and long-context understanding, with strong performance on long-context retrieval and safety.

• Production-Ready Inference We support two speculative decoding methods to accelerate inference: MTP (Multi-Token Prediction) and DSpark. Both methods can speed up model generation by approximately 3–5×, reducing latency for long-horizon workloads such as agentic tasks.

• Multilinguality & Openness We expanded multilingual coverage from six to ten languages: Korean, English, Spanish, German, Japanese, Vietnamese, French, Italian, Polish, and Portuguese. We also release K-EXAONE 2.0 under the Apache license 2.0 so that the broader AI ecosystem can inspect, deploy, and build upon it.

Model Configurations

5 DSpark layers 1 x 4096 SWA 19 x [3 x 128 SWA + 1 x Global] Blocks 256 Total Experts 8 Activated Experts

Evaluation Results

The following table shows the benchmark results for the K-EXAONE 2.0 BF16 model. Detailed evaluation results and configurations can be found in our technical report.

Quickstart

Serving K-EXAONE 2.0

SGLang

Install SGLang from our fork as follows:

uv venv
source .venv/bin/activate
uv pip install git+https://github.com/lkm2835/sglang@add-k-exaone2
uv pip install git+https://github.com/nuxlear/transformers@add-k-exaone2

The following script shows how to serve K-EXAONE 2.0 with SGLang on two nodes of 8 x NVIDIA H200 GPUs. Before starting the server, set $HEAD_ADDR to the IP address of the head (rank-0) node including the port number, and set $NODE_RANK to the number index of the node.

Run the one of the following scripts on every node, according to your purpose.

• Low latency

  sglang serve \
      --model-path LGAI-EXAONE/K-EXAONE-2.0-750B-A37B-DSpark \
      --served-model-name K-EXAONE-2.0-750B-A37B-DSpark \
      --tp 16 \
      --dist-init-addr $HEAD_ADDR \
      --nnodes 2 \
      --node-rank $NODE_RANK \
      --reasoning-parser qwen3 \
      --tool-call-parser qwen3_coder \
      --host 0.0.0.0 \
      --port 8000 \
      --max-running-requests 128 \
      --speculative-algorithm DSPARK \
      --speculative-num-draft-tokens 8 \
      --mem-fraction-static 0.85 \
      --swa-full-tokens-ratio 0.6
  

• High throughput

  sglang serve \
      --model-path LGAI-EXAONE/K-EXAONE-2.0-750B-A37B-DSpark \
      --served-model-name K-EXAONE-2.0-750B-A37B-DSpark \
      --tp 16 \
      --dp 2 \
      --enable-dp-attention \
      --enable-dp-lm-head \
      --dist-init-addr $HEAD_ADDR \
      --nnodes 2 \
      --node-rank $NODE_RANK \
      --reasoning-parser qwen3 \
      --tool-call-parser qwen3_coder \
      --host 0.0.0.0 \
      --port 8000 \
      --max-running-requests 192 \
      --cuda-graph-max-bs 96 \
      --speculative-algorithm DSPARK \
      --speculative-num-draft-tokens 8 \
      --mem-fraction-static 0.85 \
      --swa-full-tokens-ratio 0.6
  

[!NOTE] If you are using NVIDIA B200 GPUs, please add the --disable-prefill-cuda-graph option to prevent issues with model generation collapse. We will update this note once these issues have been resolved.

vLLM

You should install the vLLM library from our fork as below:

uv venv
source .venv/bin/activate
uv pip install git+https://github.com/lkm2835/vllm@add-k-exaone2 --torch-backend auto
uv pip install git+https://github.com/nuxlear/transformers@add-k-exaone2

The following script shows how to serve K-EXAONE 2.0 with vLLM on two nodes of 8 x NVIDIA H200 GPUs. Before starting the server, set $HEAD_IP to the IP address of the head (rank-0) node, and set $NODE_RANK to the number index of the node.

Run the below script on every server.

HEADLESS_ARG=''
if [ \"\$NODE_RANK\" -ne 0 ]; then
    HEADLESS_ARG='--headless'
fi

exec vllm serve LGAI-EXAONE/K-EXAONE-2.0-750B-A37B-DSpark \
    --served-model-name K-EXAONE-2.0-750B-A37B-DSpark \
    --trust-remote-code \
    --tensor-parallel-size 16 \
    --distributed-executor-backend mp \
    --nnodes 2 \
    --node-rank $NODE_RANK \
    --master-addr $HEAD_IP \
    --master-addr 30000 \
    --gpu-memory-utilization 0.9 \
    --max-num-seqs 256 \
    --reasoning-parser qwen3 \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_xml \
    --host 0.0.0.0 \
    --port 8000 \
    --speculative_config '{
        "method": "mtp",
        "num_speculative_toke