Divergence-based fine pruning of phrase-based statistical translation model. (January 2017)

• Record Type: Journal Article
• Authors: Kim, Kangil; Park, Eun-Jin; Shin, Jong-Hun; Kwon, Oh-Woog; Kim, Young-Kil
• Is Part Of: Computer speech & language. Volume 41(2016)
• Page Start: 146
• Page End: 160
• Publication Date: 2017-01
• Subjects: Statistical machine translation -- Model revision -- Phrase table -- Entropy-based pruning -- Relative entropy; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2016.06.006
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: Entropy-based pruning has a limit in selecting a fine distribution of phrase pairs to be pruned in a threshold. Changing the distribution through other divergence metrics improves pruning efficiency in our preliminary empirical analysis. Derived problematic factors are fixed divergence distribution and missing impact of word-coupling strength. We propose a fine pruning method using two parameters to control the factors and analyze their effects to divergence change. It improves pruning efficiency compared with Entropy-based pruning in practical translations of English, Spanish, and French. Abstract: A widely used automatic translation approach, phrase-based statistical machine translation, learns a probabilistic translation model composed of phrases from a large parallel corpus with a large language model. The translation model is often enormous because of many combinations of source and target phrases, which leads to the restriction of applications to limited computing environments. Entropy-based pruning resolves this issue by reducing the model size while retaining the translation quality. To safely reduce the size, this method detects redundant components by evaluating a relative entropy of models before and after pruning the components. In the literature, this method is effective, but we have observed that it can be improved more by adjusting the divergence distribution determined by the relative entropy. In the results of preliminary experiments, we derive two factors responsible for limiting pruning efficiency of entropy-based pruning. The first factor is proportion of pairs composing translation models with respect to their translation probability and its estimate. The second factor is the exponential increase of the divergence for pairs with low translation probability and estimate. To control the factors, we propose a divergence-based fine pruning using a divergence metric to adapt the curvature change of the boundary conditions for pruning and Laplace smoothing. In practical translation tasks for English–Spanish and English–French language pairs, this method shows statistically significant improvement on the efficiency up to 50% and average 12% more pruning compared to entropy-based pruning to show the same translation quality.