Model reference · open weights
GPT-2-wikitext-chunks is an open-weight language model from EleutherAI, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
GPT-2 fine-tuned on bergson-wikitext-512-chunks GPT-2 (124M) fine-tuned on EleutherAI/bergson-wikitext-512-chunks (WikiText-2 pre-chunked to 512-token sequences, 4,608 train chunks) using the bergson MAGIC trainer, as the trained model for MAGIC attribution experiments. Training - 4 epochs, global batch size 64 (8x data parallel), 288 steps - AdamW, polynomial LR schedule: lr 8e-4 (start 1e-6, end 8e-5), 25% warmup, fp32 - Loss on held-out test[:4] chunks: 3.22 (base gpt2: 3.62) Files - Standard HF model + tokenizer files - bergsonconfig.yaml — the fully-resolved bergson run config (all fields incl. defaults) that produced this model; rerun with python -m bergson bergsonconfig.yaml - optimizer.pt — AdamW second moments (expavgsq) at the final training step, in bergson's optimizer.pt normalizer format ({"state": {idx: {"expavgsq": ...}}, "paramgroups": [...]} with idx indexing deduplicated model.namedparameters()), for gradient normalization in attribution runs.
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | EleutherAI |
|---|---|
| Type | Language models |
| Parameters (lead) | 124M |
| Context | 1k tokens |
| Variants | 1 |
| Runs with | transformers |
| Based on | gpt2 |
| Released | 2026-07-08 |
| Popularity | 292 downloads / month |
| Licence | Open weights |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| GPT-2-wikitext-chunks | 124M | BF16 | ~0.3 GB | ✓ | Weights ↗ |
Using it via the API
Once AxForge deploys gpt-2-wikitext-chunks for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (gpt-2-wikitext-chunks below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"gpt-2-wikitext-chunks","messages":[{"role":"user","content":"Hello"}]}'
Details
Trained / evaluated on
Tags
Licence
Open weights under mit — commercial use is permitted. Deploy it on AxForge EU hardware on request. Read the licence ↗