Knowledge Editing

2 items

RESEARCHarXiv CS.CL·26d ago

Merging Methods for Multilingual Knowledge Editing for Large Language Models: An Empirical Odyssey

This paper investigates the effectiveness of vector merging methods for multilingual knowledge editing (MKE) in Large Language Models, focusing on reducing interference between language-specific edits. Evaluating six merging variants across two LLMs, two editing methods, and 12 languages on the MzsRE benchmark, it finds vector summation with shared covariance to be the most reliable overall strategy.

multilingual LLMs Natural Language Processing Vector Merging Knowledge Editing

RESEARCHarXiv CS.LG·12d ago

One Mask to Rule Them All: On Hidden Facts after Editing and How to Find Them

This paper investigates the internal mechanisms of knowledge editing methods such as ROME and MEMIT, revealing that diverse edits share a common functional structure reliant on a specific subset of weights. A binary mask over these edited weights reverses most changes by eliminating overattention in later layers, demonstrating this mechanism's necessity for successful edits.

AI models MLP Weights machine learning Transformer Models