Deep Learning and Gradient Boosting for Tabular Data
Comparing and improving tree ensembles and neural networks for prediction on rows and columns of business data.
Tier Grades
Tier A: Frontier AI labs
feasible
Microsoft, Yandex and Amazon release the libraries and AutoML systems.
Tier B: Indonesian AI labs
feasible
Benchmarks over 45 datasets need modest compute.
Tier C: Institution units such as ITB
feasible
The benchmark paper came from a 3-author team.
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
feasible
Tree ensembles remain state of the art on about 10K-sample tables, the usual size of a client dataset in a proof of value.
Where this comes from
Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.
Anchors
- ComputeNo compute figure; the benchmark accounts for hyperparameter search budgets across 45 datasets.
- DataStandard set of 45 tabular datasets from varied domains; tree-based models remain state of the art on medium-sized data of about 10K samples.
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): 3 authors on the tabular benchmark paper.
- Time horizonYears (mature direction).
- Shelf-life signalsTabular foundation models (TabPFN, TabICL) ranking near tuned boosting on TabArena, Interpretable boosted models (EBM) for regulated decisions
- Pipeline roleTier A and libraries supply tools; Tier C benchmarks them on Indonesian data; Tier D delivers most tabular proofs of value with them.
Where this comes from
Team size is a proxy (author counts of exemplar papers), not a headcount.
Who and What
Exemplars
Observed actors
Linked models
XGBoosttabularexemplar
LightGBMtabularexemplar
CatBoosttabularexemplarFT-TransformertabularexemplarRouteE-Powertrain model librarytabularproduced by
AutoGluon 1.5tabularproduced byExtra Trees oil palm yield model (JOPR 2025)tabularproduced byTabMtabularproduced by
RealMLPtabularproduced byGANDALFtabularproduced byDANets (Deep Abstract Networks)tabularproduced byTabTransformertabularproduced byNODE (Neural Oblivious Decision Ensembles)tabularproduced byTabNettabularproduced by
AutoInttabularproduced byExplainable Boosting Machine (EBM)tabularproduced byPerpetualBoostertabularproduced byRandomForesttabularused byLinks
R&D TopicDeep Learning and Gradient Boosting for Tabular Data
Models 18
AutoGluon 1.5Amazon, tabularproduced byAutoIntPeking University, tabularproduced byCatBoostYandex, tabularexemplarExplainable Boosting Machine (EBM)Microsoft AI, tabularproduced byExtra Trees oil palm yield model (JOPR 2025)tabularproduced byFT-TransformerYandex, tabularexemplarLightGBMMicrosoft AI, tabularexemplarNODE (Neural Oblivious Decision Ensembles)Yandex, tabularproduced byRealMLPINRIA, tabularproduced byRouteE-Powertrain model librarytabularproduced byTabMYandex, tabularproduced byXGBoostDMLC community, tabularexemplarPapers
- XGBoost: A Scalable Tree Boosting SystemTianqi Chen et al., KDD'16 (per arXiv comment), 2016 founding
- TabM: Advancing Tabular Deep Learning with Parameter-Efficient EnsemblingYury Gorishniy et al., ICLR 2025 (per arXiv comment), 2024 state of the art
- TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023Yury Gorishniy et al., arXiv preprint, 2023 state of the art
- Why do tree-based models still outperform deep learning on tabular data?Léo Grinsztajn et al., arXiv preprint, 2022 state of the art
- Tabular Data: Deep Learning is Not All You NeedRavid Shwartz-Ziv et al., arXiv preprint, 2021 survey
Where this comes from
The papers to read first: where the direction began, its current best, and a survey when one exists.
Sources
- arxiv.org/abs/2207.08815 https://arxiv.org/abs/2207.08815
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Deep Learning and Gradient Boosting for Tabular Data", https://aurion.semai.id/topics/deep-learning-and-gradient-boosting-for-tabular-data/, records of October 2026.