The Annotation Scarcity Paradox in Low-Resource NLP Evaluation: A Decade of Acceleration and Emerging Constraints

Low-resource natural language processing has experienced explosive growth, but its evaluation faces a critical challenge: the scarcity of sociolinguistic expertise needed to assess complex generative systems. This creates an "Annotation Scarcity Paradox," where the technical capacity to scale models vastly outpaces the human infrastructure required for authentic evaluation.