Connect → LiteLLM

Use AxForge with LiteLLM

LiteLLM is an open-source proxy. Put AxForge behind it to get one endpoint for many models, with budgets, keys and fallback — and to reach AxForge from tools that speak a different API shape.

config.yaml

model_list:
  - model_name: axforge-chat
    litellm_params:
      model: openai/chat                 # openai/ prefix = OpenAI-compatible upstream
      api_base: https://api.axforge.ai/v1  # MUST include /v1
      api_key: os.environ/AXFORGE_API_KEY
  - model_name: axforge-embed
    litellm_params:
      model: openai/embeddings
      api_base: https://api.axforge.ai/v1
      api_key: os.environ/AXFORGE_API_KEY

$ litellm --config config.yaml     # proxy on http://localhost:4000

Two easy mistakes. api_base must end in /v1 (omit it and you get a Not Found), and you should never append /chat/completions yourself — LiteLLM's OpenAI handler adds the path.

Three surfaces from one upstream

The proxy can re-expose AxForge on all three of the API shapes tools expect:

Surface	Reaches	Note
/v1/chat/completions	OpenAI clients	native, no extra config
/v1/messages	Anthropic / Claude Code	translated automatically
/v1/responses	Codex / Responses clients	add `use_chat_completions_api: true`

For the Responses bridge, add that flag to the model's litellm_params and use LiteLLM ≥ 1.63.8. You don't need this to reach AxForge from Codex or Claude Code — both are served natively — but it's handy if you already front everything with a gateway.

Pin a clean LiteLLM release. Avoid the PyPI builds 1.82.7 / 1.82.8, which were flagged upstream — install a known-good version.