- Published on
Xiaomi's Table-Structured Data Large Model TabLDM Open-Sourced: Top Tier in Four Benchmarks, Regression Capability To...
- Authors
- Name
- DeepHorus
On September 5th, Xiaomi officially released and open-sourced its general-purpose tabular data foundation large model, Xiaomi-TabLDM. This model aims to address the heavy deployment costs associated with traditional tabular machine learning, which requires repeated model training, parameter tuning, and integration when faced with new datasets. It achieves the core goal of "one pre-training, cross-task use." With a single pre-training configuration, the model can directly adapt to different tabular datasets to complete classification and regression prediction tasks.
For a long time, a large amount of core data in fields like finance, healthcare, manufacturing, and logistics has existed in tabular form. Traditional tabular machine learning methods typically require retraining and adjusting the model for each new dataset, leading to complex version management and low efficiency. The launch of Xiaomi-TabLDM is precisely aimed at introducing the foundation model paradigm into the domain of structured data, enhancing the capability to handle general tabular tasks.
In terms of the technical path, Xiaomi-TabLDM has innovated from three directions: pre-training data, model architecture, and inference mechanisms. The model is pre-trained entirely on large-scale synthetic data generated based on structural causal models, covering diverse data characteristics. In terms of architecture, it incorporates technologies such as dual-stream feature grouping, lightweight attention residuals, and sparse mixture of experts to model feature relationships at different granularities. During the inference stage, it explores Test-Time Scaling, continuously improving prediction performance by increasing computational effort.
In authoritative public benchmark evaluations, Xiaomi-TabLDM has shown excellent performance. It ranks in the first tier across all four major benchmarks, particularly excelling in regression tasks. Specifically, the model ranks first on the OpenML-CTR23 regression leaderboard and second in the regression tasks of TALENT, TabArena, and BCCO. In terms of balancing performance and efficiency, taking the TabArena regression task as an example, it achieves the second-highest Elo score, while its training time is reduced by 82% and prediction time by 68% compared to the top-ranked TabFM.
In validation within real-world industrial scenarios, Xiaomi-TabLDM demonstrates higher prediction accuracy and adaptability compared to traditional machine learning methods. In material property prediction scenarios, the model can improve prediction accuracy by 130% without fine-tuning and is expected to significantly reduce ineffective mold trials and testing. In part weight prediction, its ratio of error samples is reduced by approximately 31% compared to XGBoost. In production composition prediction, the average error is reduced by about 54% compared to XGBoost.
When production operating conditions change, Xiaomi-TabLDM also shows strong adaptability. By supplementing only about 30 new condition samples as context, the model can reduce the average prediction error after condition changes by approximately 62%, demonstrating excellent cross-condition generalization performance. The model provides an interface compatible with scikit-learn, allowing users to conveniently install and use it via pip.
Currently, the model weights and code of Xiaomi-TabLDM have been officially open-sourced. Developers and technical researchers can access related resources and review detailed technical reports through GitHub and Hugging Face platforms. This open-source initiative is expected to promote the further development of tabular data foundation models in both academic research and industrial applications.
