Domain Adaptation (Vertical Models)

Training or tuning models for a sector.

Tier Grades

Tier A: Frontier AI labs

feasible

high confidence

Bloomberg is observed training a 50B domain model, and frontier general models beat it on most tasks compared.

Tier B: Indonesian AI labs

feasible

medium confidence

Domain tuning on open bases fits b; from-scratch 50B training is beyond the evidence, and GPT-4 few-shot beat BloombergGPT on 3 tasks.

Tier C: Institution units such as ITB

stretch

medium confidence

From-scratch training on 708B tokens exceeds academic GPUs, while PEFT-based tuning (topic 7) fits.

Tier D: Organisations such as SEMAI (PoT, PoC, PoV)

feasible

medium confidence

The shelf-life signal favours prompting or tuning general models, which suits proof-of-value work; d would not pretrain from scratch.

Where this comes from

Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.

Anchors

ComputeBloombergGPT 50B trained from scratch; compute not given.
DataBloombergGPT: 363B finance tokens plus 345B general tokens.
Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): not sourced in the notes.
Time horizonMonths.
Shelf-life signalsGPT-4 few-shot beat BloombergGPT on FiQA sentiment (88.11 vs 75.07) and ConvFinQA (76.48 vs 43.41); BloombergGPT led on NER (60.82 vs 56.71), Domain pretraining overtaken by general models
Pipeline roleTier A's general models set the bar; Tier B and Tier C tune for sectors; Tier D carries tuned or prompted models into sector proofs of value.

Where this comes from

Team size is a proxy (author counts of exemplar papers), not a headcount.

Who and What

Exemplars

BloombergGPTGPT-4 finance study

Observed actors

BloombergAcademic evaluators

Linked models

+38

MAIRA-2multimodal LLMproduced by

LLaVA-Med v1.5 (Mistral 7B)multimodal LLMproduced by

BioMistral 7BLLMproduced by

NYUTronLLMproduced by

Med-PaLM 2LLMproduced by

BioMedLMLLMproduced by

BioGPTLLMproduced by

Stanford De-identifier baselanguage encoderproduced by

Twitter-RoBERTa base sentiment (latest)language encoderproduced by LEGAL-BERTlanguage encoderproduced by

BiomedBERT (PubMedBERT)language encoderproduced by

FinBERT (ProsusAI)language encoderproduced by Bio_ClinicalBERTlanguage encoderproduced by

BioBERTlanguage encoderproduced by

GPT-4LLMused by

C2S-Scale (Cell2Sentence)scientificproduced by

Claude Mythos 5LLMused by

Claude Mythos 5.1LLMused by

Claude Opus 4.5LLMused by

Claude Sonnet 4.5LLMused by

AMIEmultimodal LLMproduced by

MedGemmamultimodal LLMproduced by

MedGemma 27B multimodalmultimodal LLMproduced by

MedGemma 4Bmultimodal LLMproduced by

Medical-SwallowLLMproduced by

Llama3-70B InstructLLMused by

Llama 3 8BLLMused by

MAI-Cyber-1-FlashLLMproduced by

TrialGPTLLMproduced by

NemotronLLMused by

GPT-5.2LLMused by

GPT-5.6 CyberLLMproduced by

Gemini 3.8 Flash CyberLLMproduced by

Siemens Industrial Foundation Model (IFM)multimodal LLMproduced by

CheXagentmultimodal LLMproduced by

Palmyra XLLMproduced by

Palmyra X4LLMproduced by

Palmyra X5LLMproduced by

Links

R&D TopicDomain Adaptation (Vertical Models)

Models

BloombergGPTBloomberg, LLM

GPT-4OpenAI, LLM

BiomedBERT (PubMedBERT)Microsoft AI, language encoder Bio_ClinicalBERTEmily Alsentzer et al. (Bio_ClinicalBERT authors), language encoder+58 more

Use CasesClinical LLM (Open Weights)Healthcare Clinical LLM (Frontier Proprietary)Healthcare Radiology Report GenerationHealthcare Pharma Documents, Regulatory and Life-Science R&DHealthcare+10 more

Who Runs It

Kementerian Kesehatan (Kemenkes), PusdatinPlanned, 19 Aug 2026

PT Pertamina Geothermal Energy Tbk (PGE)

PT Pertamina Geothermal EnergyIn production, 14 Jul 2026

Banner HealthIn production, 11 Jan 2026 CommureIn production, 11 Jan 2026+47 more

Graph

Models 62

GPT-5.2OpenAI, LLMused by

Claude Opus 4.5Anthropic, LLMused by

Claude Sonnet 4.5Anthropic, LLMused by

MedGemma 4BGoogle, multimodal LLMproduced by

NemotronNVIDIA, LLMused by

AMIEGoogle, Google DeepMind, multimodal LLMproduced by

Llama 3 8BMeta, LLMused by

MedGemmaGoogle, multimodal LLMproduced by

C2S-Scale (Cell2Sentence)Yale University, Google, Google DeepMind, scientificproduced by

C2S-Scale-Gemma-2-2BYale University, Google, Google DeepMind, scientificproduced by

Claude Mythos 5Anthropic, LLMused by

MedGemma 1.5 4BGoogle, multimodal LLMproduced by

+50

C2S-Scale-Gemma-2-27BYale University, Google, Google DeepMind, scientificproduced by

DataGemmaGoogle, LLMproduced by

Ling 3.0 Flash Fin (Ling-3.0-flash-Fin)inclusionAI, LLMproduced by

MedGemma 27B textGoogle, LLMproduced by

Claude Mythos 5.1Anthropic, LLMused by

DataGemma RAG 27B ITGoogle, LLMproduced by

DataGemma RIG 27B ITGoogle, LLMproduced by

HuatuoGPT-o1-72BFreedomIntelligence, reasoningproduced by LLama-3-8B-Tele-itLLMproduced by

Llama3-Med42-70BM42 Health, LLMproduced by

MAIRA-2Microsoft AI, multimodal LLMproduced by

MedGemma 27B multimodalGoogle, multimodal LLMproduced by

PrismPaige, multimodal LLMproduced by

TrialGPTNIH National Library of Medicine, National Cancer Institute, LLMproduced by

TxGemmaGoogle, scientificproduced by

Baichuan-M2-32BBaichuan AI, LLMproduced by

CheXagentStanford University, multimodal LLMproduced by

Gemini 3.8 Flash CyberGoogle DeepMind, LLMproduced by

GPT-5.6 CyberOpenAI, LLMproduced by

Llama3-70B InstructMeta, LLMused by

Llama3-Med42-8BM42 Health, LLMproduced by

LLaVA-Med v1.5 (Mistral 7B)Microsoft AI, multimodal LLMproduced by

Med42 Llama3M42 Health, LLMproduced by

NYUTronNYU Langone, LLMproduced by

Palmyra XWriter, LLMproduced by

Palmyra X5Writer, LLMproduced by

Siemens Industrial Foundation Model (IFM)Siemens, Microsoft AI, multimodal LLMproduced by

TxAgent T1 Llama 3.1 8BHarvard University, reasoningproduced by

BioMistral 7BBioMistral project, LLMproduced by

BloombergGPTBloomberg, LLMexemplar

GPT-4OpenAI, LLMused by

Med-PaLM 2Google, LLMproduced by

Medical-SwallowInstitute of Science Tokyo, LLMproduced by

Palmyra X4Writer, LLMproduced by

PLaMo-3-610m-fin-instructPreferred Networks, edge LLMproduced by

TxGemma 27B chatGoogle, scientificproduced by

TxGemma 2B predictGoogle, scientificproduced by

TxGemma 9B predictGoogle, scientificproduced by

Aloe Beta 8BBarcelona Supercomputing Center (HPAI), LLMproduced by

Baichuan-M3-235BBaichuan, LLMproduced by Bio_ClinicalBERTEmily Alsentzer et al. (Bio_ClinicalBERT authors), language encoderproduced by

BioBERTKorea University, language encoderproduced by

BioGPTMicrosoft AI, LLMproduced by

BiomedBERT (PubMedBERT)Microsoft AI, language encoderproduced by

BioMedLMStanford University, LLMproduced by

FinBERT (ProsusAI)Prosus AI, language encoderproduced by LEGAL-BERTAthens University of Economics and Business (AUEB NLP Group), language encoderproduced by

MAI-Cyber-1-FlashMicrosoft AI, LLMproduced by

Stanford De-identifier baseStanford University, language encoderproduced by

Twitter-RoBERTa base sentiment (latest)Cardiff NLP (Cardiff University), language encoderproduced by

Papers

Don't Stop Pretraining: Adapt Language Models to Domains and TasksSuchin Gururangan et al., ACL 2020 (per arXiv comment), 2020 founding
BloombergGPT: A Large Language Model for FinanceShijie Wu et al., arXiv preprint, 2023 state of the art
Large Language Models Encode Clinical KnowledgeKaran Singhal et al., arXiv preprint, 2022 state of the art
Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive SurveyChen Ling et al., arXiv preprint, 2023 survey

Where this comes from

The papers to read first: where the direction began, its current best, and a survey when one exists.

Sources

BloombergGPT: A Large Language Model for Finance arxiv.org https://arxiv.org/abs/2303.17564
Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? A Study on Several Typical Tasks arxiv.org https://arxiv.org/html/2305.05862v2

To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Domain Adaptation (Vertical Models)", https://aurion.semai.id/topics/domain-adaptation-vertical-models/, records of October 2026.