← heapsort-ai

Small Language Models

6 items

RESEARCHarXiv CS.LG·13d ago

The Constraint Tax: Measuring Validity-Correctness Tradeoffs in Structured Outputs for Small Language Models

This paper introduces the "constraint tax," a measurement protocol to quantify the loss in answer accuracy caused by structured output constraints on Small Language Models (SLMs). It demonstrates that for SLMs, imposing such constraints improves schema validity but can compromise the correctness of the underlying answer.

29
RESEARCHarXiv CS.CL·5/7/2026

Self-Prompting Small Language Models for Privacy-Sensitive Clinical Information Extraction

This research presents a locally deployable framework enabling small language models to extract privacy-sensitive clinical entities from unstructured dental notes through self-generated and refined prompts. The study evaluated open-weight models, achieving high F1 scores with Qwen2.5-14B-Instruct and Llama-3.1-8B-Instruct after supervised fine-tuning and direct preference optimization.

27