- Published on
A Tabular Foundation Model That Tops TabArena Among Single Models
- Authors
- Name
- DeepHorus
Nums AI has released Causilo, a pretrained tabular foundation model for classification and regression. The model comes with a scikit-learn interface, Apache-2.0 licensed code, and pretrained weights available on Hugging Face. On TabArena, Causilo has achieved the highest Elo among single models for both classification and regression tasks. This marks a significant milestone in the emerging field of tabular foundation models.
Tabular data remains the most common data type in enterprise and research settings, yet foundation models for tabular data have lagged behind those for text and images. Competitors like Google Research's TabFM, LG AI Research's EXAONE Tabular, and Prior Labs' TabPFN-3 have pushed the boundaries. Causilo, pretrained exclusively on synthetic data, now claims the top spot among single models on TabArena, a comprehensive benchmark for tabular learning.
Causilo operates as an in-context learning model. Calling fit does not update the pretrained weights; instead, it stores training rows as context and predicts query rows in a single forward pass. Inputs can be NumPy arrays or pandas DataFrames, including categorical features and missing values. Classification supports up to 10 classes, while regression returns mean predictions by default, with version 1.0.1 adding median and quantile outputs based on 999 native quantiles.
The architecture comprises three phases: refinement, compression, and in-context learning. Features are grouped in sets of three, with each value embedded using 16 learned sine and cosine frequencies. Missing values receive their own learned vector. Two column stages summarize each feature group, with 128 latent slots reading only training rows and passing summaries to every row. Between column stages, a row stage allows feature groups to interact through four latent tokens using cross-attention, keeping cost linear in feature count. A pooling block compresses each row into a fixed 512-dimensional vector, and a 12-layer prediction block lets query rows attend to labeled rows without altering the training context. Eight ensemble members share weights and cycle through normalizations.
On TabArena, Nums AI used the official pipeline with 51 datasets and 816 Full splits. A TabArena maintainer re-ran the evaluation and confirmed an overall Elo of 1794. Causilo achieved an overall Elo of 1792.9, compared to TabFM's 1764.4 and EXAONE Tabular's 1758.8. In regression, Causilo's Elo of 2032.6 significantly outpaced TabFM's 1992.8. Improvability scores show TabFM still leads overall and in classification, while Causilo leads in regression. Confidence intervals at the top overlap, indicating a narrow lead. With systems included, Causilo placed 3rd of 88 overall.
For researchers and practitioners, Causilo offers a powerful, ready-to-use model for tabular tasks. It is deployable today for research and evaluation on CUDA or CPU, though commercial, production, and hosted API use require a separate license from Nums AI. This licensing approach may limit immediate industry adoption but ensures controlled usage. The model's scikit-learn interface lowers the barrier to entry, allowing seamless integration into existing workflows.
The release has generated excitement in the ML community, as evidenced by its inclusion in MarkTechPost's Editors Pick. The competitive landscape is heating up, with TabFM, EXAONE Tabular, and TabPFN-3 all vying for dominance. Nums AI's focus on synthetic pretraining and efficient in-context learning sets a new direction. Future updates may include expanded class support, more quantile outputs, and broader benchmarking. As tabular foundation models mature, we can expect improved performance, wider adoption, and potentially new licensing models to balance open research with commercial viability.
