Researchers Introduce CMNIE Benchmark for Chinese Military News Information Extraction

A new research preprint on arXiv introduces CMNIE, an information extraction benchmark designed for Chinese military news. The dataset is developed to facilitate intelligence analysis, decision-making, and knowledge base construction through the structured joint extraction of domain-specific events, arguments, entities, and relations.

The benchmark contains 13,000 instances gathered from public Chinese military news sources. It features manual annotations across 7 event types, 10 argument roles, 7 entity types, and 8 relation types under a unified domain schema.

The authors evaluated supervised information extraction models, zero-shot large language models, and fine-tuned LLM-based methods. The results show that while zero-shot LLMs capture relevant semantic content, they struggle with relation extraction and exact matching of event-argument span boundaries.