Project AURIONAI Use-Cases Research & Innovation

IFEval

Instruction following (SmolLM3 card)

knowledge and reasoning

Score in %

  1. 1Liquid AILFM2.5-8B-A1B91.84 %vendor-reported
  2. 2Allen AIOlmo 3 32B Think89 %vendor-reported
  3. 3NVIDIANemotron-4 340B79.9 %vendor-reported
10scores
10models
0independent
0maintainer-run
10vendor-reported

Definition

Version
not in the notes
Category
knowledge and reasoning
Maintainer
not in the notes
Status
not in the notes
Home page
not in the notes

Sources

  1. allenai/Olmo-3-32B-Think , Hugging Face huggingface.co https://huggingface.co/allenai/Olmo-3-32B-Think
  2. HuggingFaceTB/SmolLM3-3B , Hugging Face huggingface.co https://huggingface.co/HuggingFaceTB/SmolLM3-3B

Scores at a Glance

Score in %

Higher is better for this unit.

1Liquid AILFM2.5-8B-A1BLiquid AI
91.84 %
vendor-reported2Allen AIOlmo 3 32B ThinkAllen AI
89 %
vendor-reported3NVIDIANemotron-4 340BNVIDIAInstruct model; prompt-strict accuracy
79.9 %
vendor-reported4Z.ai (Zhipu)GLM-4 (0520)Z.ai (Zhipu)English, prompt-level strict
79.1 %
vendor-reported5Hugging FaceSmolLM3-3BHugging FaceCard 5-model comparison table; mode label not extracted
76.7 %
vendor-reported
0255075100
Where this comes from
Each bar is one model, with its most independent recorded score. Hatched bars are vendor-reported; the word beside each bar says who ran it. Settings and dates differ between rows; the Scores table below has every result.

Scores

10 of 10 scores shown ifeval-scores.csv
ModelScoreWho ran itSettingView or populationDateSource
Liquid AILFM2.5-8B-A1B91.84 %vendor-reported–––huggingface.co
NVIDIALlama 3.1 Nemotron Ultra 253B v189.45 %vendor-reportedReasoning on, strict instructionNemotron Ultra model cardApr 2025huggingface.co
NVIDIALlama 3.3 Nemotron Super 49B v189.21 %vendor-reportedReasoning off, strict instructionNemotron Super model cardMar 2025huggingface.co
Allen AIOlmo 3 32B Think89 %vendor-reported–––huggingface.co
Allen AIOlmo 3.1 32B Instruct88.8 %vendor-reported–Olmo 3.1 32B Instruct model cardDec 2025huggingface.co
Allen AIOLMo 2 32B (OLMo-2-0325-32B-Instruct)85.6 %vendor-reported–OLMo-2-0325-32B-Instruct model cardMar 2025huggingface.co
IBMGranite-4.0-H-Micro84.32 %vendor-reportedAverage of instruct and prompt strictGranite-4.0-H-Micro model card2 Oct 2025huggingface.co
NVIDIANemotron-4 340B79.9 %vendor-reportedInstruct model; prompt-strict accuracy–Jun 2024huggingface.co
Z.ai (Zhipu)GLM-4 (0520)79.1 %vendor-reportedEnglish, prompt-level strict–18 Jun 2024arxiv.org
Hugging FaceSmolLM3-3B76.7 %vendor-reportedCard 5-model comparison table; mode label not extracted––huggingface.co
Where this comes from
Scores from different settings, harnesses or dates are not directly comparable; the row says which. Sorted by score, highest first, as a reading aid, not a ranking.

To cite this page: Project AURION, AI Use-Cases Research & Innovation, "IFEval", https://aurion.semai.id/benchmarks/ifeval/, records of October 2026.