Self-Supervised Visual Representation Learning
Pretraining image encoders without labels so their features transfer to many vision tasks.
Tier Grades
Tier A: Frontier AI labs
feasible
A 7B teacher on a dataset named LVD-1689M, built by a 26-author team, is frontier-lab work.
Tier B: Indonesian AI labs
not feasible
Pretraining at that scale is not feasible for an Indonesian lab; using the released distilled backbones is.
Tier C: Institution units such as ITB
stretch
Small-scale self-supervised studies on local data fit a university unit; frontier scale does not.
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
feasible
Released DINOv3 backbones give strong features for few-label client tasks.
Where this comes from
Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.
Anchors
- ComputeDINOv3 scales both dataset and model size; the 7B teacher is named in the Hugging Face checkpoint dinov3-vit7b16-pretrain-lvd1689m. Training compute is not in the sources read.
- DataThe checkpoint name refers to the LVD-1689M dataset; the abstract states data from natural to aerial images.
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): 26 authors on the DINOv3 report (arXiv 2508.10104).
- Time horizonYear.
- Shelf-life signalsA new DINO or JEPA generation becomes the default backbone, Vision-language encoders overtake label-free ones on dense tasks
- Pipeline roleTier A pretrains and releases backbones; Tier C studies them on local imagery such as aerial data; Tier D uses frozen features to build classifiers with few labels.
Where this comes from
Team size is a proxy (author counts of exemplar papers), not a headcount.
Who and What
Exemplars
Observed actors
Linked models
DINOv3visionexemplarDINOv2visionexemplarI-JEPAvisionexemplarMAE (Masked Autoencoder)visionexemplarLingBot-Visionvisionproduced bySapiens2visionproduced byEUPE (Efficient Universal Perception Encoder)visionproduced byDINOv3 ConvNeXt-Bvisionproduced byDINOv3 ConvNeXt-Lvisionproduced byDINOv3 ConvNeXt-Svisionproduced byDINOv3 ConvNeXt-Tvisionproduced byDINOv3 ViT-7Bvisionproduced byDINOv3 ViT-Bvisionproduced byDINOv3 ViT-H+visionproduced byDINOv3 ViT-Lvisionproduced byDINOv3 ViT-Svisionproduced byDINOv3 ViT-S+visionproduced by
AIMv2visionproduced bySapiensvisionproduced by
RAD-DINOvisionproduced by
VideoPrismvisionproduced byAIMvisionproduced byHieravisionproduced byVideoMAE V2visionproduced by+22
EVA-02visionproduced byEVAvisionproduced byBEiT-3visionproduced byBEiT v2visionproduced by
VideoMAEvisionproduced byDiT (Document Image Transformer)OCR and documentproduced bydata2vec visionvisionproduced by
SimMIMvisionproduced byBEiTvisionproduced byBEiT-Bvisionproduced byDINO (v1)visionproduced byBarlow Twinsvisionproduced by
SwAVvisionproduced by
BYOLvisionproduced bySimCLRvisionproduced byMoCovisionproduced byBigBiGANimage generationproduced by
SuperPointvisionproduced byCAEv2visionproduced by
C-RADIOv2visionproduced byNV-DINOv2visionproduced byUnsupervised Latent Landmarksvisionproduced byLinks
R&D TopicSelf-Supervised Visual Representation Learning
ModelsDINOv2Meta, visionDINOv3Meta, visionMAE (Masked Autoencoder)Meta, visionI-JEPAMeta, vision+42 more
DINOv2Meta, visionDINOv3Meta, visionMAE (Masked Autoencoder)Meta, visionI-JEPAMeta, vision+42 moreUse Casesnone recorded
Who Runs Itnone recorded
Models 46
DINOv3Meta, visionexemplarDINOv3 ViT-7BMeta, visionproduced byDINOv2Meta, visionexemplarBEiTMicrosoft AI, visionproduced byBEiT v2Microsoft AI, visionproduced byBEiT-3Microsoft AI, visionproduced byDINOv3 ConvNeXt-BMeta, visionproduced byDINOv3 ConvNeXt-LMeta, visionproduced byDINOv3 ConvNeXt-SMeta, visionproduced byDINOv3 ConvNeXt-TMeta, visionproduced bySapiensMeta, visionproduced bySapiens2Meta, visionproduced by+34
VideoMAENanjing University, Tencent, visionproduced byAIMv2Apple, visionproduced byBEiT-BMicrosoft AI, visionproduced byDINOv3 ViT-BMeta, visionproduced byDINOv3 ViT-H+Meta, visionproduced byDINOv3 ViT-LMeta, visionproduced byDINOv3 ViT-SMeta, visionproduced byDINOv3 ViT-S+Meta, visionproduced byEVABAAI, Huazhong University of Science and Technology, visionproduced byEVA-02BAAI, Huazhong University of Science and Technology, visionproduced byAIMApple, visionproduced byBarlow TwinsMeta, New York University, visionproduced byBigBiGANGoogle DeepMind, image generationproduced byDINO (v1)Meta, visionproduced byDiT (Document Image Transformer)Microsoft AI, OCR and documentproduced byRAD-DINOMicrosoft AI, visionproduced bySimMIMTsinghua University, Microsoft Research, visionproduced bySuperPointMagic Leap, visionproduced bySwAVINRIA, Meta, visionproduced byVideoMAE V2visionproduced byVideoPrismGoogle, visionproduced byBYOLGoogle DeepMind, visionproduced byC-RADIOv2NVIDIA, visionproduced byCAEv2visionproduced bydata2vec visionMeta, visionproduced byEUPE (Efficient Universal Perception Encoder)Meta, visionproduced byHieravisionproduced byI-JEPAMeta, visionexemplarMAE (Masked Autoencoder)Meta, visionexemplarMoCoMeta, visionproduced byNV-DINOv2NVIDIA, visionproduced bySimCLRGoogle, visionproduced byUnsupervised Latent LandmarksNVIDIA, visionproduced byLingBot-Visionvisionproduced byPapers
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen et al., ICML'2020 (per arXiv comment), 2020 founding
- Momentum Contrast for Unsupervised Visual Representation LearningKaiming He et al., CVPR 2020 (per arXiv comment), 2019 founding
- DINOv3Oriane Siméoni et al., arXiv preprint, 2025 state of the art
- DINOv2: Learning Robust Visual Features without SupervisionMaxime Oquab et al., arXiv preprint, 2023 state of the art
- Emerging Properties in Self-Supervised Vision TransformersMathilde Caron et al., arXiv preprint, 2021 state of the art
Where this comes from
The papers to read first: where the direction began, its current best, and a survey when one exists.
Sources
- arxiv.org/abs/2508.10104 https://arxiv.org/abs/2508.10104
- huggingface.co/facebook/dinov3-vit7b16-pretrain-lvd1689m https://huggingface.co/facebook/dinov3-vit7b16-pretrain-lvd1689m
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Self-Supervised Visual Representation Learning", https://aurion.semai.id/topics/self-supervised-visual-representation-learning/, records of October 2026.