Ouro

About

What Ouro is

Ouro-1.4B 📚 Paper • 🏠 Project Page Model Description ⚠️ IMPORTANT: This model is intended for research purposes only. It is provided as-is without warranties for production use. Ouro-1.4B is a 1.4 billion parameter Looped Language Model (LoopLM) that achieves exceptional parameter efficiency through iterative shared-weight computation. Key Features - Exceptional Parameter Efficiency: Matches 3-4B standard transformer performance with only 1.4B parameters - Iterative Latent Reasoning: Performs reasoning through recurrent computation in latent space - Adaptive Computation: Supports early exit mechanisms for dynamic compute allocation Configuration Recurrent Steps and Adaptive Exit The model's computational behavior can be configured through the config.json file: - totalutsteps: Controls the number of recurrent steps (default: 4). You can adjust this value to trade off between performance and computation time. - earlyexitthreshold: Controls the adaptive exit mechanism (default: 1.0). Lower values encourage earlier exit, while 1.0 means always use all steps. Example: Modify recurrent steps Note: vLLM does not currently support the adaptive exit feature due to its inference optimization characteristics. When using vLLM, the model will always execute the full number of totalutsteps. Model Architecture Ouro-1.4B is based on the decoder-only Transformer architecture with parameter sharing across recurrent steps: Training Details - Training Tokens: 7.7T tokens - Training Pipeline: - Stage 1: Pre-training (6T tokens) - Stage 2: CT Annealing (1.4T tokens) - Stage 3: Long Context Training (20B tokens) - Stage 4: Mid-training (300B tokens) - Data Composition: Web data, code, mathematics, long-context documents - Optimizer: AdamW (β₁=0.9, β₂=0.95) - Learning Rate Scheduler: Warmup-Stable-Decay (WSD) Quick Start ⚠️ IMPORTANT: Please use transformers<4.56.0 to avoid compatibility issues. We recommend transformers==4.54.1 or earlier versions. Acknowledgments We thank @Antizana for the KV cache fix merged from ouro-cache-fix, which resolved a critical compatibility issue with transformers=4.56.0. Citation

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys ouro for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (ouro below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"ouro","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Papers

Licence

Open weights

Sources

Weights & code

Explore

More language models