PIXIE-Spell

Model Description

• Model Type: Sentence Transformer

• Maximum Sequence Length: 8192 tokens

• Output Dimensionality: 1024 dimensions

• Similarity Function: Cosine Similarity

• Language: Multilingual — optimized for high performance in Korean and English

• Domain Specialization: Aerospace semantic search

• License: apache-2.0

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'Qwen3Model'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Quality Benchmarks

PIXIE-Spell-Preview-0.6B is a multilingual embedding model specialized for Korean and English retrieval tasks. It delivers consistently strong performance across a diverse set of domain-specific and open-domain benchmarks in both languages, demonstrating its effectiveness in real-world semantic search applications. The table below presents the retrieval performance of several embedding models evaluated on a variety of Korean and English benchmarks. We report Normalized Discounted Cumulative Gain (NDCG) scores, which measure how well a ranked list of documents aligns with ground truth relevance. Higher values indicate better retrieval quality.

• Avg. NDCG: Average of NDCG@1, @3, @5, and @10 across all benchmark datasets.
• NDCG@k: Relevance quality of the top-k retrieved results.

All evaluations were conducted using the open-source Korean-MTEB-Retrieval-Evaluators codebase to ensure consistent dataset handling, indexing, retrieval, and NDCG@k computation across models.

6 Datasets of MTEB (Korean)

Our model, telepix/PIXIE-Spell-Preview-0.6B, achieves strong performance across most metrics and benchmarks, demonstrating strong generalization across domains such as multi-hop QA, long-document retrieval, public health, and e-commerce.

Descriptions of the benchmark datasets used for evaluation are as follows:

• Ko-StrategyQA A Korean multi-hop open-domain question answering dataset designed for complex reasoning over multiple documents.
• AutoRAGRetrieval A domain-diverse retrieval dataset covering finance, government, healthcare, legal, and e-commerce sectors.
• MIRACLRetrieval A document retrieval benchmark built on Korean Wikipedia articles.
• PublicHealthQA A retrieval dataset focused on medical and public health topics.
• BelebeleRetrieval A dataset for retrieving relevant content from web and news articles in Korean.
• MultiLongDocRetrieval A long-document retrieval benchmark based on Korean Wikipedia and mC4 corpus.

Tip: While many benchmark datasets are available for evaluation, in this project we chose to use only those that contain clean positive documents for each query. Keep in mind that a benchmark dataset is just that a benchmark. For real-world applications, it is best to construct an evaluation dataset tailored to your specific domain and evaluate embedding models, such as PIXIE, in that environment to determine the most suitable one.

7 Datasets of BEIR (English)

Our model, telepix/PIXIE-Spell-Preview-0.6B, achieves strong performance on a wide range of tasks, including fact verification, multi-hop question answering, financial QA, and scientific document retrieval, demonstrating competitive generalization across diverse domains.