Runtimes
Inference runtimes and formats that decide where a model can run.
- 30runtimes
Models per Runtime
Anemoi
- Owner
- ECMWF
- Licence
- not in the notes
- Latest release
- not in the notes
- Backends
- not in the notes
- Notes
- Open-source framework used to train AIFS
- Models
- Sources
- ECMWF’s ensemble AI forecasts become operational ecmwf.int https://www.ecmwf.int/en/about/media-centre/news/2025/ecmwfs-ensemble-ai-forecasts-become-operational
ComfyUI
- Owner
- not in the notes
- Licence
- not in the notes
- Latest release
- not in the notes
- Backends
- not in the notes
- Notes
- Claims MiniMax H3 runs in 12 GB with offload
- Models
- Sources
- MiniMax Ships H3 Weights , With the US and EU Excluded rits.shanghai.nyu.edu https://rits.shanghai.nyu.edu/ai/minimax-ships-h3-weights-with-the-us-and-eu-excluded/
Core ML (via Core ML Tools)
- Owner
- Apple
- Licence
- coremltools BSD-style licence (Apple 2020 to 2023)
- Latest release
- coremltools 9.0 (10 Nov 2025)
- Backends
- not in the notes
- Notes
- Converts TensorFlow 1.x/2.x, PyTorch, scikit-learn, XGBoost and LibSVM to Core ML format for deployment through Xcode; Core ML runtime ships with Apple OSes (runtime licence not checked)
- Models
- not in the notes
- Sources
- GitHub - apple/coremltools: Core ML tools contain supporting tools for Core ML model conversion, editing, and validation. github.com https://github.com/apple/coremltools
- Client Challenge pypi.org https://pypi.org/project/coremltools/
DGL
- Owner
- DMLC community
- Licence
- Apache License 2.0
- Latest release
- 2.2.1 (13 May 2024)
- Backends
- not in the notes
- Notes
- No PyPI release since May 2024
- Models
- not in the notes
- Sources
- pypi.org/pypi/dgl/json https://pypi.org/pypi/dgl/json
- GitHub - dmlc/dgl: Python package built to ease deep learning on graph, on top of existing DL frameworks. github.com https://github.com/dmlc/dgl
ExecuTorch
- Owner
- Meta
- Licence
- BSD 3-Clause
- Latest release
- 1.5.1 (22 Sep 2026)
- Backends
- XNNPACK, Core ML (ANE), MPS (deprecated), Qualcomm, MediaTek, Samsung Exynos, Arm Ethos-U, Vulkan, OpenVINO, NXP, Cadence DSP, CUDA/Metal (experimental)
- Notes
- AOT export (torch.export) to .pte, C++ runtime, 50 KB base footprint. Used in Instagram, WhatsApp, Quest 3, Ray-Ban Meta glasses
- Models
- Sources
- GitHub - pytorch/executorch: On-device AI across mobile, embedded and edge for PyTorch github.com https://github.com/pytorch/executorch
- executorch pypi.org https://pypi.org/project/executorch/
FunASR toolkit
- Owner
- Alibaba (Qwen)
- Licence
- MIT License
- Latest release
- not in the notes
- Backends
- not in the notes
- Notes
- Model zoo includes Fun-ASR, SenseVoice, Paraformer, CAM++, Whisper
- Models
- not in the notes
- Sources
- GitHub - modelscope/FunASR: Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diariza github.com/modelscope/FunASR https://github.com/modelscope/FunASR
- GitHub - modelscope/FunASR: Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diariza github.com/modelscope/FunASR https://github.com/modelscope/FunASR
Genesis / Genesis World
- Owner
- Genesis AI
- Licence
- Apache License 2.0
- Latest release
- Genesis World 1.0 (May 2026)
- Backends
- not in the notes
- Notes
- Multi-physics engine, Nyx renderer, Quadrants compiler (forked from Taichi, Jun 2025); about 30k GitHub stars
- Models
- not in the notes
- Sources
- GitHub - Genesis-Embodied-AI/genesis-world: Simulation platform for general-purpose robotics & embodied AI learning. github.com https://github.com/Genesis-Embodied-AI/Genesis
- Genesis AI launches with $105M seed funding from Eclipse, Khosla to build AI models for robots | TechCrunch techcrunch.com https://www.techcrunch.com/2025/07/01/genesis-ai-launches-with-105m-seed-funding-from-eclipse-khosla-to-build-ai-models-for-robots
Genie Sim 3.0
- Owner
- AgiBot (Zhiyuan)
- Licence
- not in the notes
- Latest release
- 3.0
- Backends
- not in the notes
- Notes
- Simulation platform used for GO-2 sim-to-real
- Models
- not in the notes
- Sources
- AGIBOT releases GO-2 foundation model for embodied AI - The Robot Report therobotreport.com https://www.therobotreport.com/agibot-releases-go-2-foundation-model-embodied-ai/
Isaac Lab (on Isaac Sim)
- Owner
- NVIDIA
- Licence
- BSD 3-Clause
- Latest release
- Isaac Lab 3.0 (referenced Mar 2026) (6 Nov 2025)
- Backends
- GPU-parallel physics
- Notes
- GPU-native successor to Isaac Gym; photoreal rendering, sensor sims, domain randomisation; Newton integration planned. Licence BSD-3 unverified
- Models
- not in the notes
- Sources
- Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning arxiv.org https://arxiv.org/abs/2511.04831
- NVIDIA GR00T N1.7 Hits Early Access for Humanoids | Origin of Bots originofbots.com https://www.originofbots.com/news/nvidia-gr00t-n17-hits-early-access-for-humanoids
LeRobot
- Owner
- not in the notes
- Licence
- not in the notes
- Latest release
- not in the notes
- Backends
- not in the notes
- Notes
- GR00T N1.7 integration
- Models
- Sources
- NVIDIA GR00T N1.7 Hits Early Access for Humanoids | Origin of Bots originofbots.com https://www.originofbots.com/news/nvidia-gr00t-n17-hits-early-access-for-humanoids
LiteRT (formerly TensorFlow Lite) and LiteRT-LM
- Owner
- Licence
- Apache License 2.0
- Latest release
- ai-edge-litert 2.3.0 (6 Oct 2026)
- Backends
- On-device; GPU and NPU details unverified
- Models
- Sources
- LiteRT: High-Performance On-Device Machine Learning Framework | Google AI Edge | Google for Developers ai.google.dev https://ai.google.dev/edge/litert
- GitHub - google-ai-edge/LiteRT: LiteRT, successor to TensorFlow Lite. is Google's On-device framework for high-performance ML & GenAI deploy github.com https://github.com/google-ai-edge/LiteRT
- ai-edge-litert pypi.org https://pypi.org/project/ai-edge-litert/
llama.cpp
- Owner
- ggml-org
- Licence
- MIT License
- Latest release
- not in the notes
- Backends
- Metal, CUDA, Vulkan, SYCL, CANN (Ascend), HIP, Hexagon, OpenCL (Adreno), WebGPU, ZenDNN, MUSA, OpenVINO (in progress)
- Notes
- 1.5-bit to 8-bit integer quantization
- Models
- Sources
- GitHub - modelscope/FunASR: Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diariza github.com https://github.com/modelscope/FunASR
- GitHub - ggml-org/llama.cpp: LLM inference in C/C++ github.com https://github.com/ggml-org/llama.cpp
MLC LLM
- Owner
- MLC (mlc-ai project, CMU/OctoAI origin)
- Licence
- Apache License 2.0
- Latest release
- not in the notes
- Backends
- Vulkan, ROCm, CUDA, Metal (macOS, iOS/iPadOS A-series), Intel GPU via Vulkan/Metal, WebGPU and WASM (browser), OpenCL on Adreno and Mali (Android)
- Notes
- MLCEngine OpenAI-compatible REST, Python and JS API; builds on Apache TVM; companion WebLLM
- Models
- not in the notes
- Sources
- GitHub - mlc-ai/mlc-llm: Universal LLM Deployment Engine with ML Compilation github.com https://github.com/mlc-ai/mlc-llm
MONAI
- Owner
- not in the notes
- Licence
- Apache License 2.0
- Latest release
- not in the notes
- Backends
- not in the notes
- Notes
- Core, Label and Deploy components for medical imaging AI; NVIDIA and King's College London origin unverified
- Models
- not in the notes
- Sources
- GitHub - Project-MONAI/MONAI: AI Toolkit for Healthcare Imaging github.com https://github.com/Project-MONAI/MONAI
- monai.io https://monai.io/
MuJoCo (with MJX and MuJoCo Warp)
- Owner
- Google DeepMind
- Licence
- Apache License 2.0
- Latest release
- not in the notes
- Backends
- not in the notes
- Notes
- Monthly release cadence; MJX is a JAX branch; Gemini Robotics On-Device SDK ships a MuJoCo simulator
- Models
- Sources
- GitHub - google-deepmind/mujoco: Multi-Joint dynamics with Contact. A general purpose physics simulator. github.com https://github.com/google-deepmind/mujoco
- aiwiki.ai/wiki/gemini_robotics_2 https://aiwiki.ai/wiki/gemini_robotics_2
Newton
- Owner
- Linux Foundation
- Licence
- Apache License 2.0
- Latest release
- not in the notes
- Backends
- GPU physics on NVIDIA Warp (requires Warp 1.18); MuJoCo Warp as primary backend
- Notes
- OpenUSD; differentiable. Initiated by Disney Research, Google DeepMind and NVIDIA; docs CC-BY-4.0
- Models
- not in the notes
- Sources
- GitHub - newton-physics/newton: An open-source, GPU-accelerated physics simulation engine built upon NVIDIA Warp, specifically targeting rob github.com https://github.com/newton-physics/newton
NVIDIA cuOpt
- Owner
- NVIDIA
- Licence
- Apache License 2.0
- Latest release
- 26.10
- Backends
- GPU-accelerated
- Notes
- Optimisation solver (not a neural model); Python API and server; TSP, VRP, PDP, LP, QP, MIP (beta)
- Models
- not in the notes
- Sources
- GitHub - NVIDIA/cuopt: GPU accelerated decision optimization github.com https://github.com/NVIDIA/cuopt
NVIDIA NIM
- Owner
- NVIDIA
- Licence
- not in the notes
- Latest release
- not in the notes
- Backends
- not in the notes
- Notes
- Serves Cosmos 3 and Alpamayo 1.5
- Models
- Sources
- investor.nvidia.com/news/press-release-details/2026/NVIDIA-Launch... https://investor.nvidia.com/news/press-release-details/2026/NVIDIA-Launches-Cosmos-3-the-Open-Frontier-Foundation-Model-for-Physical-AI/default.aspx
- Alpamayo-1.5-10B | NVIDIA NGC catalog.ngc.nvidia.com https://catalog.ngc.nvidia.com/orgs/nim/nvidia/models/alpamayo-1-5-10b/-
Ollama
- Owner
- Ollama
- Licence
- MIT License
- Latest release
- not in the notes
- Backends
- CPU/GPU details not fetched
- Notes
- Local LLM runner, GGUF-based
- Models
- Sources
- library ollama.com https://ollama.com/library
- ollama/LICENSE at main , ollama/ollama github.com https://github.com/ollama/ollama/blob/main/LICENSE
ONNX Runtime
- Owner
- Microsoft AI
- Licence
- MIT License
- Latest release
- 1.31.0 (9 Oct 2026)
- Backends
- CPU: default CPU, Intel DNNL, Intel OpenVINO, XNNPACK. GPU: NVIDIA CUDA, NVIDIA TensorRT, DirectML, AMD MIGraphX, AMD ROCm (deprecated), WebGPU. IoT/edge/mobile: Intel OpenVINO, Rockchip NPU (preview), Arm Compute Library (preview), Android NNAPI, Arm NN (preview), CoreML (preview), Qualcomm QNN, XNNPACK. Other: Xilinx Vitis-AI, TVM, Huawei CANN, Azure (all preview)
- Models
- Sources
- GitHub - microsoft/onnxruntime: ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator github.com https://github.com/microsoft/onnxruntime
- Execution Providers onnxruntime.ai https://onnxruntime.ai/docs/execution-providers/
- onnxruntime pypi.org https://pypi.org/project/onnxruntime/
OpenVINO
- Owner
- Intel
- Licence
- Apache License 2.0
- Latest release
- 2026.4.1 (1 Oct 2026)
- Backends
- CPU (x86, ARM), Intel integrated and discrete GPU, Intel NPU
- Notes
- Imports PyTorch, TensorFlow, ONNX, TensorFlow Lite, PaddlePaddle, JAX/Flax. Companion OpenVINO Model Server
- Models
- Sources
- aisingapore/Qwen-SEA-LION-v4.5-27B-IT , Hugging Face huggingface.co https://huggingface.co/aisingapore/Qwen-SEA-LION-v4.5-27B-IT
- GitHub - openvinotoolkit/openvino: OpenVINO™ is an open source toolkit for optimizing and deploying AI inference github.com https://github.com/openvinotoolkit/openvino
- Client Challenge pypi.org https://pypi.org/project/openvino/
PyTorch Geometric (PyG)
- Owner
- PyG team
- Licence
- MIT License
- Latest release
- 2.8.0.post1 (20 Jul 2026)
- Backends
- PyTorch 2.9 to 2.12
- Notes
- 2.8.0 adds RelBench-to-PyG converter, GRetriever example, LLM txt2qa; 2.7.0 added LPFormer, SGFormer, Polynormer
- Models
- not in the notes
- Sources
- Releases , pyg-team/pytorch_geometric github.com https://github.com/pyg-team/pytorch_geometric/releases
- pypi.org/pypi/torch-geometric/json https://pypi.org/pypi/torch-geometric/json
- GitHub - pyg-team/pytorch_geometric: Graph Neural Network Library for PyTorch github.com https://github.com/pyg-team/pytorch_geometric
Qualcomm AI Runtime (QAIRT) / GenieX
- Owner
- Qualcomm
- Licence
- not in the notes
- Latest release
- not in the notes
- Backends
- Snapdragon devices
- Notes
- Runtime listed on Qualcomm AI Hub; GenieX (QAIRT, llama.cpp)
- Models
- not in the notes
- Sources
- Qualcomm AI Hub aihub.qualcomm.com https://aihub.qualcomm.com/models
SGLang
- Owner
- not in the notes
- Licence
- not in the notes
- Latest release
- not in the notes
- Backends
- Multi-GPU
- Notes
- Recommended multi-GPU (4 GPUs) serving for MiniMax H3
- Models
- Sources
- MiniMax Ships H3 Weights , With the US and EU Excluded rits.shanghai.nyu.edu https://rits.shanghai.nyu.edu/ai/minimax-ships-h3-weights-with-the-us-and-eu-excluded/
TensorFlow Lite for Microcontrollers (TFLM)
- Owner
- Licence
- Apache License 2.0
- Latest release
- not in the notes
- Backends
- DSPs, MCUs: Cortex-M, RISC-V, Hexagon, Xtensa; ports for Arduino, Espressif, Renesas, Silicon Labs, TI, Coral Dev Board Micro
- Notes
- Rename to LiteRT for Microcontrollers not confirmed
- Models
- Sources
- GitHub - tensorflow/tflite-micro: Infrastructure to enable deployment of ML models to low-power resource-constrained embedded targets (inclu github.com https://github.com/tensorflow/tflite-micro
TensorRT
- Owner
- NVIDIA
- Licence
- Proprietary
- Latest release
- 11.4.0.106 (8 Oct 2026)
- Backends
- not in the notes
- Notes
- Per-model support matrix (LLM, encoder NLP, vision, audio, diffusion, multimodal). Open-source components on GitHub are Apache 2.0; PyPI package declares Proprietary. Jetson not checked
- Models
- not in the notes
- Sources
- GitHub - NVIDIA/TensorRT: NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains github.com https://github.com/NVIDIA/TensorRT
- Client Challenge pypi.org https://pypi.org/project/tensorrt/
TensorRT-LLM
- Owner
- NVIDIA
- Licence
- Apache License 2.0
- Latest release
- 1.2.1 (20 Apr 2026)
- Backends
- Blackwell data-centre GPUs; Jetson and Thor edge support unverified
- Notes
- GitHub README news to 17 Jul (PyPI may lag)
- Models
- not in the notes
- Sources
- GitHub - NVIDIA/TensorRT-LLM: TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports github.com https://github.com/NVIDIA/TensorRT-LLM
- tensorrt-llm pypi.org https://pypi.org/project/tensorrt-llm/
TerraTorch
- Owner
- not in the notes
- Licence
- not in the notes
- Latest release
- not in the notes
- Backends
- not in the notes
- Notes
- Tooling for Prithvi-EO-2.0 geospatial models
- Models
- Sources
- ibm-nasa-geospatial/Prithvi-EO-2.0-600M , Hugging Face huggingface.co https://huggingface.co/ibm-nasa-geospatial/Prithvi-EO-2.0-600M
TinyEngine (MCUNet)
- Owner
- MIT Han Lab
- Licence
- MIT License
- Latest release
- not in the notes
- Backends
- Microcontrollers (STM32F746 profiled)
- Notes
- 1.5 to 3x faster and 2.7 to 4.8x lower peak memory than TFLM, CMSIS-NN, MicroTVM
- Models
- Sources
- GitHub - mit-han-lab/mcunet: [NeurIPS 2020] MCUNet: Tiny Deep Learning on IoT Devices; [NeurIPS 2021] MCUNetV2: Memory-Efficient Patch-based github.com https://github.com/mit-han-lab/mcunet
vLLM
- Owner
- not in the notes
- Licence
- not in the notes
- Latest release
- not in the notes
- Backends
- not in the notes
- Models
- Sources
- huggingface.co/tiiuae/Falcon-H1R-7B https://huggingface.co/tiiuae/Falcon-H1R-7B
