When Cases Get Rare: A Retrieval Benchmark for Off-Guideline Clinical Question Answering

This paper introduces OGCaReBench, a new retrieval-focused benchmark aimed at evaluating LLMs' ability to answer clinical questions that go beyond typical medical guidelines. It addresses the gap where most medical LLMs are trained on common, guideline-focused knowledge, while real-world care often involves rare cases not covered by guidelines.