pre-training

2 items

ARTICLE↑ trendingReddit r/MachineLearning·4/26/2026

Can Geometric Deep Learning lead eliminate the need of "Brute Force" pre-training [D]

The author questions whether Geometric Deep Learning, by explicitly building symmetries and invariances into its architecture, could significantly reduce or eliminate the need for extensive, data-intensive pre-training. This raises the question of whether current massive-scale pre-training is largely a consequence of architectures lacking inherent invariance.

pre-training Symmetry Model Architecture Geometric Deep Learning

RESEARCHarXiv CS.CL·4/14/2026

Toward Generalized Cross-Lingual Hateful Language Detection with Web-Scale Data and Ensemble LLM Annotations

This research explores improving cross-lingual hate speech detection by leveraging large-scale unlabelled web data and LLM-based synthetic annotations. It shows that continued pre-training of BERT models on web data and fine-tuning with synthetic labels generated by an ensemble of LLMs significantly boosts performance, especially in low-resource settings.

Multilingual AI pre-training ensemble learning Hate Speech Detection