Contrastive Image-Text Encoders (CLIP and SigLIP Lines)
Image and text encoders trained together so that pictures and captions share one embedding space for zero-shot search and classification.
Tier Grades
Tier A: Frontier AI labs
feasible
Web-scale image-text pretraining is led by big tech.
Tier B: Indonesian AI labs
feasible
Four TPUv4 chips for two days, as in the SigLIP anchor, is within an Indonesian lab budget for locked-image tuning.
Tier C: Institution units such as ITB
feasible
A 4-author paper shows a university unit can run this research.
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
feasible
Zero-shot image search and tagging from a released encoder fits a proof of concept, for example product search.
Where this comes from
Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.
Anchors
- ComputeA SigLiT model reaches 84.5% ImageNet zero-shot accuracy with four TPUv4 chips in two days.
- DataBatch sizes up to one million were tested; 32k was found sufficient.
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): 4 authors on the SigLIP paper (arXiv 2303.15343).
- Time horizonYear.
- Shelf-life signalsEncoders are absorbed into multimodal LLM vision towers, Domain encoders (fashion, medicine) outperform general ones
- Pipeline roleTier A trains web-scale encoders; Tier B and Tier C tune them on Indonesian captions and domains; Tier D uses them for search and tagging pilots.
Where this comes from
Team size is a proxy (author counts of exemplar papers), not a headcount.
Who and What
Exemplars
Observed actors
Linked models
TIPSv2visionexemplarSigLIP 2embeddingexemplar
MobileCLIPembeddingexemplarFashionCLIPvisionexemplar
OpenCLIP (LAION)embeddingexemplar
CLIPembeddingexemplarALIGNvisionexemplarSigLIP-400MvisionexemplarMobileCLIP2-S0embeddingproduced by
MetaCLIP 2 (worldwide, ViT-H)embeddingproduced byPE-Core-G14-448embeddingproduced byPE-Core-L14-336embeddingproduced byPerception Encoderembeddingproduced bySigLIP 2 base patch16 224embeddingproduced bySigLIP 2 giant patch16 384embeddingproduced bySigLIP 2 So400m patch14 384embeddingproduced byAIMv2visionproduced by
LLM2CLIPvisionproduced by
nomic-embed-vision-v1.5embeddingproduced by
ViTaminvisionproduced byMobileCLIP-S0embeddingproduced by
CONCHvisionproduced by
InternViTvisionproduced byDFN5B-CLIP ViT-H-14-378embeddingproduced by+18
ImageBindembeddingproduced by
EVA-CLIPembeddingproduced byBiomedCLIPembeddingproduced byOpenCLIP ViT-bigG-14 LAION-2Bembeddingproduced by
Chinese-CLIP ViT-B/16embeddingproduced byOpenCLIP ViT-B-32 LAION-2Bembeddingproduced byOpenCLIP ViT-H-14 LAION-2Bembeddingproduced byBEiT-3visionproduced byX-CLIPvisionproduced byX-CLIP base patch32embeddingproduced byCLIP ViT-L/14 336pxembeddingproduced by
oCLIPOCR and documentproduced by
ActionCLIPvisionproduced byMedSigLIPvisionproduced byCLIP ViT-B/32embeddingproduced byCLIP ViT-L/14embeddingproduced by
TwelveLabs Marengoembeddingproduced byMarengo 3.0embeddingproduced byLinks
R&D TopicContrastive Image-Text Encoders (CLIP and SigLIP Lines)
Who Runs Itnone recorded
Models 42
BEiT-3Microsoft AI, visionproduced byCLIPOpenAI, embeddingexemplarMedSigLIPGoogle, visionproduced byMobileCLIPApple, embeddingexemplarOpenCLIP (LAION)LAION, embeddingexemplarSigLIP 2Google, embeddingexemplarAIMv2Apple, visionproduced byCONCHMahmood Lab, Harvard, visionproduced byMarengo 3.0TwelveLabs, embeddingproduced byMobileCLIP-S0Apple, embeddingproduced byMobileCLIP2-S0Apple, embeddingproduced byoCLIPNanyang Technological University, ByteDance, OCR and documentproduced by+30
Perception EncoderMeta, embeddingproduced byTwelveLabs MarengoTwelveLabs, embeddingproduced byActionCLIPZhejiang University, visionproduced byBiomedCLIPMicrosoft AI, embeddingproduced byCLIP ViT-B/32OpenAI, embeddingproduced byCLIP ViT-L/14OpenAI, embeddingproduced byCLIP ViT-L/14 336pxOpenAI, embeddingproduced byImageBindMeta, embeddingproduced byInternViTShanghai AI Lab, Nanjing University, visionproduced byOpenCLIP ViT-B-32 LAION-2BLAION, embeddingproduced byOpenCLIP ViT-bigG-14 LAION-2BLAION, embeddingproduced byOpenCLIP ViT-H-14 LAION-2BLAION, embeddingproduced byPE-Core-G14-448Meta, embeddingproduced byPE-Core-L14-336Meta, embeddingproduced bySigLIP 2 base patch16 224Google, embeddingproduced bySigLIP 2 giant patch16 384Google, embeddingproduced bySigLIP 2 So400m patch14 384Google, embeddingproduced bySigLIP-400MvisionexemplarX-CLIPMicrosoft AI, visionproduced byX-CLIP base patch32Microsoft AI, embeddingproduced byALIGNGoogle, visionexemplarChinese-CLIP ViT-B/16Alibaba DAMO, embeddingproduced byDFN5B-CLIP ViT-H-14-378Apple, embeddingproduced byEVA-CLIPBAAI, embeddingproduced byLLM2CLIPMicrosoft AI, visionproduced byMetaCLIP 2 (worldwide, ViT-H)Meta, embeddingproduced bynomic-embed-vision-v1.5Nomic AI, embeddingproduced byTIPSv2Google, visionexemplarViTaminJohns Hopkins University, visionproduced byFashionCLIPvisionexemplarPapers
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford et al., arXiv preprint, 2021 founding
- Scaling Up Visual and Vision-Language Representation Learning With Noisy Text SupervisionChao Jia et al., International Conference on Machine Learning 2021, 2021 founding
- SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense FeaturesMichael Tschannen et al., arXiv preprint, 2025 state of the art
- Sigmoid Loss for Language Image Pre-TrainingXiaohua Zhai et al., ICCV'23 (per arXiv comment), 2023 state of the art
- Reproducible scaling laws for contrastive language-image learningMehdi Cherti et al., Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio, 2022 state of the art
Where this comes from
The papers to read first: where the direction began, its current best, and a survey when one exists.
Sources
- arxiv.org/abs/2303.15343 https://arxiv.org/abs/2303.15343
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Contrastive Image-Text Encoders (CLIP and SigLIP Lines)", https://aurion.semai.id/topics/contrastive-image-text-encoders/, records of October 2026.