PQuAD: A Persian question answering dataset. (May 2023) - Darvishi, Kasra; Shahbodaghkhan, Newsha; Abbasiantaeb, Zahra; Momtazi, Saeedeh
Abstract: We present the Persian Question Answering Dataset (PQuAD), a crowdsourced reading comprehension dataset on Persian Wikipedia articles. It includes 80, 000 questions along with their answers, with 25% of the questions being adversarially unanswerable. We examine various properties of the dataset to show the diversity and the level of its difficulty as a MRC benchmark. By releasing this dataset, we aim to ease research on Persian reading comprehension and the development of Persian question answering systems. Our experiments on different state-of-the-art pre-trained contextualized language models show 74.8% Exact Match (EM) and 87.6% F1-score that can be used as the baseline results for further research on Persian QA. Highlights: Providing a large dataset for Persian machine reading comprehension Analyzing various attributes of dataset in order to indicate the degree of difficulty Presenting baseline results using state-of-the-art transformer-based models Providing human performance estimation on the dataset along with their analysis