EXAONE-4.0

About

What EXAONE-4.0 is

🎉 License Updated! We are pleased to announce our more flexible licensing terms 🤗 EXAONE-4.0-32B Introduction We introduce EXAONE 4.0, which integrates a Non-reasoning mode and Reasoning mode to achieve both the excellent usability of EXAONE 3.5 and the advanced reasoning abilities of EXAONE Deep. To pave the way for the agentic AI era, EXAONE 4.0 incorporates essential features such as agentic tool use, and its multilingual capabilities are extended to support Spanish in addition to English and Korean. The EXAONE 4.0 model series consists of two sizes: a mid-size 32B model optimized for high performance, and a small-size 1.2B model designed for on-device applications. In the EXAONE 4.0 architecture, we apply new architectural changes compared to previous EXAONE models as below: 1. Hybrid Attention: For the 32B model, we adopt hybrid attention scheme, which combines Local attention (sliding window attention) with Global attention (full attention) in a 3:1 ratio. We do not use RoPE (Rotary Positional Embedding) for global attention for better global context understanding. 2. QK-Reorder-Norm: We reorder the LayerNorm position from the traditional Pre-LN scheme by applying LayerNorm directly to the attention and MLP outputs, and we add RMS normalization right after the Q and K projection. It helps yield better performance on downstream tasks despite consuming more computation. For more details, please refer to our technical report, HuggingFace paper, blog, and GitHub. Model Configuration - Number of Parameters (without embeddings): 30.95B - Number of Layers: 64 - Number of Attention Heads: GQA with 40-heads and 8-KV heads - Vocab Size: 102,400 - Context Length: 131,072 tokens Quickstart You should install the transformers library with version = 4.54.0. Non-reasoning mode For general use, you can use the EXAONE 4.0 models with the following example: Reasoning mode The EXAONE 4.0 models have reasoning capabilities for handling complex problems. You can activate reasoning mode by using the enablethinking=True argument with the tokenizer, which opens a reasoning block that starts with <think tag without closing it. [!IMPORTANT] The model generation with reasoning mode

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys exaone-4-0 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (exaone-4-0 below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"exaone-4-0","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Papers

Licence

Commercial licence needed

Sources

Weights & code

Explore

More language models