INTEGRATING LONG-DISTANCE LANGUAGE MODELING TO PHONEME-TO-TEXT CONVERSION

This paper presents a phoneme-to-text conversion system for Chinese language using long-distance language modeling. First of all, we employ extended bigrams (Huang 1993) of window size d to capture the long-distance dependent relations in Chinese language, in which d bigram tables are estimated independently from the training data for distance 1 to d. Each bigram table is associated with a mixture weight, which can be optimized based on the held-out data using deleted interpolation algorithm (Ney 1994). The system then performs the tree-trellis search (Soong 1991) to generate N-best sentence hypotheses, and integrates these extended bigram probabilities at sentence level. In our experiments, we generate 200 best sentence hypotheses and the integration of long-distance bigram reduces the error rate by about 11% as compared with word bigram language model only. Secondly, to reduce the number of parameters, we merge the extended bigram tables from distance 2 to d to form a single long-distance bigram table, disregarding the influence caused by different distances. Since the model complexity is significantly reduced, we derive a very efficient stack decoding algorithm for the integration of this augmented long-distance information. Experiments show that the error rate remains the same as that of d extended bigrams using N-best search algorithm, while the search efficiency is significantly improved.