Language Modeling with Power Low Rank Ensembles
Ankur P. Parikh
Avneesh Saluja
Chris Dyer
Eric P Xing
10.1184/R1/6475838.v1
https://kilthub.cmu.edu/articles/journal_contribution/Language_Modeling_with_Power_Low_Rank_Ensembles/6475838
<p>We present power low rank ensembles (PLRE), a flexible framework for n-gram language modeling where ensembles of low rank matrices and tensors are used to obtain smoothed probability estimates of words in context. Our method can be understood as a generalization of ngram modeling to non-integer n, and includes standard techniques such as absolute discounting and Kneser-Ney smoothing as special cases. PLRE training is efficient and our approach outperforms state-of-the-art modified Kneser Ney baselines in terms of perplexity on large corpora as well as on BLEU score in a downstream machine translation task.</p>
2014-10-01 00:00:00
Machine Learning