Model reference · open weights
EXAONE-3.5 is an open-weight language model from LGAI-EXAONE, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
EXAONE-3.5-7.8B-Instruct-AWQ Introduction We introduce EXAONE 3.5, a collection of instruction-tuned bilingual (English and Korean) generative models ranging from 2.4B to 32B parameters, developed and released by LG AI Research. EXAONE 3.5 language models include: 1) 2.4B model optimized for deployment on small or resource-constrained devices, 2) 7.8B model matching the size of its predecessor but offering improved performance, and 3) 32B model delivering powerful performance. All models support long-context processing of up to 32K tokens. Each model demonstrates state-of-the-art performance in real-world use cases and long-context understanding, while remaining competitive in general domains compared to recently released models of similar sizes. For more details, please refer to our technical report, blog and GitHub. This repository contains the AWQ-quantized weights of the instruction-tuned 7.8B language model with the following features: - Number of Parameters (without embeddings): 6.98B - Number of Layers: 32 - Number of Attention Heads: GQA with 32 Q-heads and 8 KV-heads - Vocab Size: 102,400 - Context Length: 32,768 tokens - Quantization: AWQ with 4-bit group-wise weight-only quantization (W4A16g128) Quickstart We recommend to use transformers=4.43 and autoawq=0.2.7.post3. Here is the code snippet to run conversational inference with the model: ### Note The EXAONE 3.5 instruction-tuned language models were trained to utilize the system prompt, so we highly recommend using the system prompts provided in the code snippet above. Deployment EXAONE 3.5 models can be inferred in the various frameworks, such as: - TensorRT-LLM - vLLM - SGLang - llama.cpp - Ollama Please refer to our EXAONE 3.5 GitHub for more details about the inference frameworks. Quantization We provide the pre-quantized EXAONE 3.5 models with AWQ and several quantization types in GGUF format. Please refer to our EXAONE 3.5 collection to find corresponding quantized models. Limitation The EXAONE language model has certain limitations and may occasionally generate inappropriate responses. The language model generates responses based on the output probability of tokens, and it is determined durin
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | LGAI-EXAONE |
|---|---|
| Type | Language models |
| Parameters (lead) | 7.8B |
| Context | 32k tokens |
| Variants | 5 |
| Runs with | transformers |
| Based on | EXAONE-3.5-7.8B-Instruct |
| Released | 2024-12-01 |
| Popularity | 399k downloads / month |
| Likes | 190 |
| Licence | Commercial licence needed |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
Using it via the API
Once AxForge deploys exaone-3-5 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (exaone-3-5 below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"exaone-3-5","messages":[{"role":"user","content":"Hello"}]}'
Licence
The weights are open but its licence needs a commercial agreement for business use. AxForge can arrange that licence and host the model for you — you pay AxForge, we settle with the model’s maker. Ask us for a quote. Read the licence ↗
Sources