Preference Optimization

$\xi$-DPO: Direct Preference Optimization via Ratio Reward Margin

This paper introduces -DPO, a direct preference optimization method using a ratio reward margin, to address the challenge of hyperparameter tuning in SimPO. The research analyzes SimPO and reformulates the preference objective to improve interpretability across datasets with varying reward gap structures.