Haidass-Translate

FLORES-200 dev

*Drafter-143M: a control model with identical configuration, data and training recipe, except that it starts from random initialization instead of the pretrained base — used to quantify the contribution of base-model pretraining.

OPUS-MT models are single-directional — one independent 78M model per direction; "-" marks directions a model does not serve.

FLORES+ devtest

The same models re-evaluated on FLORES+ devtest (released 2026; zero overlap with dev):

devtest sentences do not overlap with dev. This model's scores on the new split are essentially unchanged (en→zh 27.56→28.48, zh→en 17.17→17.01), indicating that the results reflect translation ability rather than memorization of a specific benchmark.

Decontamination

To verify that the scores contain no test-set leakage, we audited all 15.83M training samples: every sentence is cut into consecutive fragments (8 words for English, 10 characters for Chinese), and any training sample sharing any fragment with any test sentence is counted as a hit. Results: 1,147 hits (0.0072%) against FLORES-200 dev, 1,788 (0.0113%) against FLORES+ devtest. Manual inspection shows the hits are common-phrase-level fragment overlaps rather than full-sentence leakage — i.e., the reported scores are not inflated by leakage. Audit report (top-50 overlapping samples included for inspection): audit_report.json (devtest audit: audit_floresplus_devtest.json in the same repo).

Training recipe

• Base: Haidass1.5-143M (Qwen3 architecture: 30 layers, hidden 576, GQA 9/3, vocab 64,000)
• Data: 7.837M cleaned zh↔en parallel sentence pairs (15.67M samples bidirectional, translation-only, no general-domain data)
• Packing: official MindSpeed-LLM --pack --neat-pack (607,622 full 2048-token sequences with inter-document attention-mask isolation)
• Training: 16×Ascend 910C, GBS=256, lr 3e-5 cosine over a 5-epoch schedule; released checkpoint at epoch 4 (iteration 9,496, ~5.0B tokens, loss 1.671) — epoch-wise ablation showed epoch 4 as the sweet spot (epoch 5 added no gain)
• Framework: MindSpeed-LLM v2.3.0 + Megatron-LM core_v0.12.1 (NPU)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("DALabCommunity/Haidass-Translate-143M", torch_dtype="bfloat16", device_map="auto")
tok = AutoTokenizer.from_pretrained("DALabCommunity/Haidass-Translate-143M")

msgs = [{"role": "user", "content": "将以下文本翻译为英文：光子甚至比构成原子的物质还要小！"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
# Even photons are smaller than the stuff that makes up atoms!

Note: the training data follows the qwen3 chat template (an empty `` block precedes the assistant turn). Always use the model's built-in chat_template at inference; do not hand-craft prompts.

Translation samples (spot check on FLORES-200 dev)

en→zh:

Src: Water is another example. The compound water is made up of two hydrogen atoms and one oxygen atom. Out: 水是另一个例子。化合物水是由两个氢原子和一个氧原子组成的。(sentence chrF++ 71.7)

Src: They are listed on the UNESCO World Heritage List. Out: 它们被列入联合国教科文组织世界遗产名录。(sentence chrF++ 69.4)

zh→en:

Src: 它们被列入了联合国教科文组织世界遗产名录。 Out: They are listed in the UNESCO World Heritage List. (sentence chrF++ 89.2)

Src: 光子甚至比构成原子的物质还要小！ Out: Even photons are smaller than the stuff that makes up atoms! (sentence chrF++ 84.1)

Known limitations

• zh→en gains come mainly from longer training (2→4 epochs: 14.65→17.17); at 2 epochs, more same-distribution parallel data (1M/4M/8M) plateaued at ~14.4
• Typical residual errors: entity mix-ups (e.g., "斯洛伐克" → Slovenia), occasional omission of numeric details
• Optimized for zh⇄en translation only; not a general chat model

Evaluation

• Metrics: sacreBLEU corpus BLEU (tokenize=zh for Chinese targets, tokenize=13a for English) + chrF++ (word_order=2); prompts byte-identical to the training chat te