Guardrails and Safety Classifiers
Input and output moderation models.
Tier Grades
Tier A: Frontier AI labs
feasible
medium confidence
Meta and NVIDIA are observed shipping guard models.
Tier B: Indonesian AI labs
feasible
medium confidence
A 7B model tuned on a low-volume dataset is within reach for a lab that trained a 70B model.
Tier C: Institution units such as ITB
feasible
medium confidence
Tuning a 7B model on a small dataset fits academic GPUs when combined with QLoRA-type methods (topic 7).
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
feasible
medium confidence
A weeks-long 7B fine-tune on a small dataset fits proof-of-concept scale; the notes give no direct proof-of-concept anchor.
Where this comes from
Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.
Anchors
- ComputeLlama Guard: Llama2-7B instruction-tuned on a low-volume dataset; compute not given.
- DataLow-volume dataset labelled against a risk taxonomy.
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): not sourced in the notes.
- Time horizonWeeks.
- Shelf-life signalsnot in the notes
- Pipeline roleTier A releases guard models; Tier B and Tier C tune them to local languages and norms; Tier D puts them in front of client deployments.
Where this comes from
Team size is a proxy (author counts of exemplar papers), not a headcount.
Who and What
Exemplars
Observed actors
Linked models
Llama Guardguardexemplar
Nemotron Safety modelsguardexemplar
Shieldstral 1.0 3Bguardproduced byNemotron 3.5 Content Safetyguardproduced by
OpenAI privacy-filterguardproduced by
Granite Guardian 4.1-8Bguardproduced byNemotron 3 Content Safety 4Bguardproduced by
Gemma-SEA-Guard-12B-2602guardproduced byLlama-SEA-Guard-8B-2602guardproduced byQwen-SEA-Guard-8B-2602guardproduced bySEA-Guardguardproduced bygpt-oss-safeguard-20bguardproduced by
Qwen3Guardguardproduced byQwen3Guard-Gen-0.6Bguardproduced byQwen3Guard-Gen-4Bguardproduced byQwen3Guard-Gen-8Bguardproduced byQwen3Guard-Stream-0.6Bguardproduced byQwen3Guard-Stream-4Bguardproduced byQwen3Guard-Stream-8Bguardproduced bygpt-oss-safeguard-120bguardproduced by
ShieldGemma 2 4Bguardproduced byLlama Guard 4 12Bguardproduced byLlama Prompt Guard 2 22Mguardproduced byLlama Prompt Guard 2 86Mguardproduced by+19
Prompt-Guard-86Mguardproduced by
Bespoke-MiniCheck-7Bguardproduced by
Roblox Voice Safety Classifierguardproduced by
WildGuard 7Bguardproduced byShieldGemma 27Bguardproduced byShieldGemma 2Bguardproduced byShieldGemma 9Bguardproduced by
HHEM-2.1-Openguardproduced by
StarPIIguardproduced byFalconsai NSFW image detectionvisionproduced byGoogle Model Armor and Vertex safety filtersguardproduced byGranite Guardian 3.3-8Bguardproduced byGranite Guardian 4.0-3B toxicity-jaguardproduced byGranite Guardian HAP 125Mguardproduced byGranite Guardian HAP 38Mguardproduced by
Azure AI Content Safetyguardproduced byMistral Moderation 2guardproduced byOpenAI Moderation (omni-moderation-latest)guardproduced by
Detoxify unbiased-toxic-robertaguardproduced byLinks
R&D TopicGuardrails and Safety Classifiers
ModelsLlama GuardMeta, guardNemotron Safety modelsNVIDIA, guardFalconsai NSFW image detectionvisionGemma-SEA-Guard-12B-2602AI Singapore, guard+39 more
Llama GuardMeta, guardNemotron Safety modelsNVIDIA, guardFalconsai NSFW image detectionvisionGemma-SEA-Guard-12B-2602AI Singapore, guard+39 moreUse Casesnone recorded
Who Runs Itnone recorded
Models 43
Qwen3GuardAlibaba (Qwen), guardproduced bySEA-GuardAI Singapore, guardproduced byQwen-SEA-Guard-8B-2602AI Singapore, guardproduced byGemma-SEA-Guard-12B-2602AI Singapore, guardproduced bygpt-oss-safeguard-120bOpenAI, guardproduced bygpt-oss-safeguard-20bOpenAI, guardproduced byGranite Guardian 4.1-8BIBM, guardproduced byHHEM-2.1-OpenVectara, guardproduced byLlama-SEA-Guard-8B-2602AI Singapore, guardproduced byQwen3Guard-Stream-4BAlibaba (Qwen), guardproduced byBespoke-MiniCheck-7BBespoke Labs, guardproduced byLlama GuardMeta, guardexemplar+31
Llama Guard 4 12BMeta, guardproduced byNemotron 3.5 Content SafetyNVIDIA, guardproduced byQwen3Guard-Gen-0.6BAlibaba (Qwen), guardproduced byQwen3Guard-Gen-4BAlibaba (Qwen), guardproduced byQwen3Guard-Gen-8BAlibaba (Qwen), guardproduced byQwen3Guard-Stream-0.6BAlibaba (Qwen), guardproduced byQwen3Guard-Stream-8BAlibaba (Qwen), guardproduced byShieldGemma 2 4BGoogle, guardproduced byShieldGemma 27BGoogle, guardproduced byShieldGemma 2BGoogle, guardproduced byShieldGemma 9BGoogle, guardproduced byShieldstral 1.0 3BMistral AI, guardproduced byWildGuard 7BAllen AI, guardproduced byAzure AI Content SafetyMicrosoft AI, guardproduced byDetoxify unbiased-toxic-robertaUnitary, guardproduced byGoogle Model Armor and Vertex safety filtersGoogle, guardproduced byGranite Guardian 3.3-8BIBM, guardproduced byGranite Guardian 4.0-3B toxicity-jaIBM, guardproduced byGranite Guardian HAP 125MIBM, guardproduced byGranite Guardian HAP 38MIBM, guardproduced byLlama Prompt Guard 2 22MMeta, guardproduced byLlama Prompt Guard 2 86MMeta, guardproduced byMistral Moderation 2Mistral AI, guardproduced byNemotron 3 Content Safety 4BNVIDIA, guardproduced byNemotron Safety modelsNVIDIA, guardexemplarOpenAI Moderation (omni-moderation-latest)OpenAI, guardproduced byOpenAI privacy-filterOpenAI, guardproduced byPrompt-Guard-86MMeta, guardproduced byRoblox Voice Safety ClassifierRoblox, guardproduced byStarPIIBigCode (Hugging Face and ServiceNow), guardproduced byFalconsai NSFW image detectionvisionproduced byPapers
- Llama Guard: LLM-based Input-Output Safeguard for Human-AI ConversationsHakan Inan et al., arXiv preprint, 2023 founding
- Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red TeamingMrinank Sharma et al., arXiv preprint, 2025 state of the art
- WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMsSeungju Han et al., NeurIPS 2024 (per arXiv comment), 2024 state of the art
- ShieldGemma: Generative AI Content Moderation Based on GemmaWenjun Zeng et al., arXiv preprint, 2024 state of the art
- NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable RailsTraian Rebedea et al., EMNLP 2023 (per arXiv comment), 2023 state of the art
Where this comes from
The papers to read first: where the direction began, its current best, and a survey when one exists.
Sources
- NVIDIA Nemotron nvidia.com https://www.nvidia.com/en-us/ai-data-science/foundation-models/nemotron/
- Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations arxiv.org https://arxiv.org/abs/2312.06674
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Guardrails and Safety Classifiers", https://aurion.semai.id/topics/guardrails-and-safety-classifiers/, records of October 2026.