Researchers Propose Reasoning and Retrieval Method for Multilingual Entity Linking

In a preprint published on arXiv, researchers investigated multimodal entity linking, which grounds entity mentions in text and images to knowledge-base entries. While previous work measured entity rarity via popularity metrics such as pageviews, the authors introduced knowledge-graph structural metrics. On the identified rare-entity slices, current state-of-the-art systems saw accuracy drops between 15.4% and 39.9%.

To address this failure mode, the team designed a training-free framework where a reasoning-capable vision-language model iteratively searches and reasons over Wikipedia. Controlled experiments demonstrated that combining retrieval with reasoning yields the best results, as reasoning alone showed minimal gains on rare entities and retrieval alone harmed overall accuracy.

Evaluated on the MERLIN multilingual benchmark across five languages—Hindi, Indonesian, Japanese, Tamil, and Vietnamese—the framework improved overall accuracy by 6.9% and rare-entity accuracy by up to 23.3%. The authors have released the framework alongside MERLIN-Rare, a dedicated test set for rare-entity evaluation.