Blaze-SFT

Architecture Specifications

Bug Fixes in modeling_blaze.py

We fixed several issues in the original modeling code to make it ready for real use:

• Fixed KV-Cache Decode Bug: During single-token generation (q_len = 1, kv_len > 1), is_causal was previously set to True in PyTorch SDPA. This caused the model to only attend to token 0 and masked out all previous tokens, completely breaking text generation. We set is_causal = False for decode steps so it correctly attends to the full cached history.
• Fixed Transformers 5.x DynamicCache Crash: Calling past_kv.update() on recurrent passes threw an IndexError: list index out of range because slots 15 to 26 were never allocated. We now pre-allocate missing slots dynamically and track recurrent passes cleanly via _current_pass.
• Generation API & NoneType Crash: Fixed prepare_inputs_for_generation to slice position_ids, cache_position, and input_ids cleanly. Handled num_logits_to_keep = None to stop generate() from crashing with a TypeError.
• Eliminated PCIe GPU-to-CPU Stalls: Removed a synchronous torch.all(attention_mask == 1) check inside the attention layer that was stalling the GPU 26 times per token. Replaced memory-allocating repeat_interleave() with zero-copy view expansions, and added native FlashAttention support.
• Zero Weight Degradation: Weights were not modified or retrained during these code fixes. Logit parity check confirms exactly 0.00000000 difference against original checkpoints.

Benchmark Results

Evaluated 0-shot using normalized accuracy (acc_norm):

Intelligence Index Calculation

The Open SLM Intelligence Index adjusts each raw score above the random-chance floor:

Normalized Score = 100 * (score - chance) / (100 - chance)

• HellaSwag (score: 31.86%, chance: 25, weight: 1.00) -> 9.14
• Combined ARC (score: 32.76%, chance: 25, weight: 1.00) -> 10.35
• PIQA (score: 61.75%, chance: 50, weight: 1.00) -> 23.50
• ArithMark-3.0 (score: 37.90%, chance: 25, weight: 0.65) -> 17.20

Intelligence Index = (9.14 + 10.35 + 23.50 + 0.65 * 17.20) / 3.65 = 14.84

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "SurjoLabs/Blaze-SFT"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "user", "content": "Explain why the sky is blue in two simple sentences."}
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=64,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    eos_token_id=tokenizer.eos_token_id,
)

response = outputs[0][inputs.input_ids.shape[1]:]
print(tokenizer.decode(response, skip_special_tokens=True))

Limitations

• Parameter Size: At 48.3M parameters, Blaze-SFT is built for ultra-lightweight, edge-device tasks. It is not meant for complex multi-step reasoning, advanced math, or heavy factual knowledge. We aim to finetune this model in order to test downstream capability of the architecture.
• Hallucinations: Like any model of this size, it can generate confident-sounding incorrect statements.

License

MIT