Audio LLMs

2 items

RESEARCHarXiv CS.CL·15d ago

EchoDistill:Alignment Noisy-to-Clean Self-Distillation for Robust Audio LLMs

EchoDistill is an alignment-based self-distillation framework designed to make Audio Large Language Models (ALLMs) robust to real-world noise. It leverages a frozen clean-audio teacher to guide an inference-time noisy-audio student, optimizing responses via group-relative policy optimization and token-level consistency.

robustness Audio LLMs Machine Learning Self-Distillation

RESEARCHarXiv CS.CL·15d ago

Direct Preference Optimization for English-Mandarin Code-Switching Speech Recognition in Audio LLMs

This paper investigates failures in Audio LLMs when transcribing English-Mandarin code-switching speech, identifying issues like language omission and translation. Applying Direct Preference Optimization (DPO) aligns models to preserve mixed-language content, leading to significant reductions in Mixed Error Rate (MER).

Multilingual AI Audio LLMs Code-Switching Direct Preference Optimization