Testing the Test: Score-Direction Instability in Class-Split Anomaly Detection

Within-dataset class-split evaluation for anomaly detection can be ill-posed when anomaly classes overlap with normal data, causing score instability or inversion. A new diagnostic, neighborhood class leakage, is introduced to predict this instability across various datasets and models.