SingGuard-NSFA

Model Details

Model Description

SingGuard-NSFA is built on the NSFA (Not-Secure-For-Agents) taxonomy, a CIA-triad-grounded hierarchical classification of 185 risk variants cross-validated against three OWASP guidelines. The framework operates as a single-turn, text-based guardrail, inspecting user queries (input guardrail) and agent responses (output guardrail) to block operational threats before agent execution.

• Developed by: SingGuard Team, AI Security Lab, Ant Group
• Model type: Dual-mode guardrail (generative reasoning + discriminative classification heads) for agentic AI security
• Language(s) (NLP): 133 languages
• License: Apache 2.0
• Finetuned from model: Qwen3.5 (Base variants, 0.8B / 2B / 4B / 9B)

Model Sources

• Repository: https://github.com/inclusionAI/SingGuard-NSFA
• Paper: SingGuard-NSFA: Extensible Guardrails for Agentic AI via Generative Reasoning and Real-Time Classification (https://arxiv.org/abs/2607.13081)

Uses

Direct Use

SingGuard-NSFA is intended to be deployed as a guardrail module in agentic AI systems to detect operational security threats in real time. It supports two complementary inference modes:

• Real-time classification (online interception): Lightweight per-domain MLP classification heads on the frozen SFT backbone output risk probability scores in a single forward pass (~45--57 ms per sample on a single NVIDIA A100 GPU). This mode is suitable for high-throughput online traffic where rapid risk screening is the primary requirement. Operators can set per-domain confidence thresholds based on their risk tolerance.
• Generative reasoning (offline auditing): The SFT model autoregressively generates a free-form chain-of-thought risk analysis followed by a structured risk-type judgment, providing full interpretability for compliance auditing, incident investigation, and human-in-the-loop decision workflows.

The guardrail inspects two detection sides:

• Query-side (input guardrail): 5 Level-1 risk domains -- Prompt Injection & Jailbreak, Malicious Code & Cyberattack, Sensitive Information Stealing, Dangerous Operations & Tool Abuse, Resource Abuse.
• Response-side (output guardrail): 2 Level-1 risk domains -- Hazardous Action Generation, Sensitive Information Leakage.

Downstream Use

• Plug-in enhancement for other guardrails: The classification-head architecture can be trained on top of any frozen guardrail backbone (e.g., Llama Guard 3) to extend its detection capabilities to NSFA risk domains. Experiments show that augmenting Llama Guard 3 with NSFA classification heads improves F1 by 17.6 points on query detection and elevates it to the top rank among all external guardrails.
• Extensibility to new risk types: New risk domains can be added by training only an additional lightweight classification head on the frozen backbone's embeddings, without retraining the backbone or disrupting existing detection capabilities. For example, a content safety head trained on the SingGuard-NSFA 9B backbone achieves near state-of-the-art performance on content moderation benchmarks.
• Edge deployment: The 0.8B model variant is suitable for resource-constrained edge devices while maintaining >94% F1.

Out-of-Scope Use

• Multi-turn or trajectory-level analysis: SingGuard-NSFA processes single-turn, text-only inputs. It cannot detect threats that emerge across multi-turn interaction trajectories, including gradual goal hijacking and cascading tool-call failures.
• Multimodal threats: Image, audio, or video-based threats are outside the current scope.
• Inter-agent communication poisoning: Multi-agent system-level threats such as cascading failures and inter-agent communication poisoning are not covered.
• Content safety moderation: The NSFA taxonomy focuses on operational agent security (what an agent does), not textual compliance (what a model says). Risks such as pornography, violence, and drug-related content are excluded from the NSFA taxonomy. (However, the classification-head architecture can be extended to content safety as a downstream use.)
• Malicious use: The model should not be used to generate, optimize, or evade detection of harmful agent inputs. It is a defensive tool only.

Recommendations

Users (both direct and downstream) should be made aware of the following:

• SingGuard-NSFA is a single-turn guardrail and should be complemented by multi-turn trajectory analysis tools for comprehensive agent security.
• Per-domain confidence thresholds should be tuned based on deployment-specific risk tolerance and traffic characteristics.
• For low-resource language deployments, additional evaluation on local language data is recommended.
• The classification-head architecture is natively extensible; operators are encouraged to train custom heads for domain-specific risks not covered by the NSFA taxonomy.

How to Get Started with the Model

SingGuard-NSFA supports two inference modes. Below are usage examples.

Generative Reasoning Mode

The generative reasoning mode uses vLLM for efficient inference. The model accepts user queries or agent responses wrapped in boundary tags (for queries, for responses) and outputs a chain-of-thought risk analysis followed by a structured risk-domain