Position: Uncertainty Quantification in LLMs is Just Unsupervised Clustering

This paper argues that current Uncertainty Quantification (UQ) methods for LLMs are essentially unsupervised clustering algorithms, measuring internal consistency rather than external correctness. Consequently, these methods fail to detect "confident hallucinations" and may create a deceptive sense of safety when deploying LLMs in high-stakes domains.