Assessment of web article's completeness by capturing discussed topics. (1st December 2014) - Han, Jingyu; Chen, Kejia; Zheng, Zhu; Yang, Yanying
Completeness is one of the first-class data quality dimensions of web articles. From the point of view of the semantics, we propose to measure completeness in terms of topic-coverage and topic-depth by capturing an article's relation distribution. Given a target article, we use a generative probabilistic model, latent Dirichlet allocation (LDA), to generate its topic-coverage and topic-depth baselines by leveraging knowledge from Wikipedia. It consists of two phases, the baseline construction and the completeness calculation. During the baseline construction, we extract Wikipedia knowledge to construct baselines using the LDA generative process, which present what should be covered and how deeply each sub-topic should be discussed. During the completeness calculation, the completeness is quantified by comparing the article with its topic-coverage and topic-depth baselines. In the meantime, what should be discussed by the article is also predicted. Experiments demonstrate that our approach can effectively rate completeness and predict content.