В данной статье рассматриваются некоторые приемы, используемые при моделирования языка, такие как Long Short-Term Memory (LSTM), Recurrent Neural Networks (RNN), а также архитектура Sequence to Sequence (Seq2seq). <...> Nikolay Ivanov OVERVIEW OF APPROACHES TO STATISTICAL LANGUAGE MODELLING Recent advances in the applications of deep neural networks to the task of text generation demonstrated the possibility of building conversationals systems based on statistical methods. <...> In this article we describe some popular techniques that are used in the tasks of language modelling, including Long Short-Term Memory (LSTM), Recurrent Neural Networks (RNN), and Sequence to Sequence architecture (Seq2seq). <...> Recent advances in the applications of deep neural networks to the task of text generation demonstrated the possibility of building conversationals systems based on statistical methods. <...> In this article we describe some popular techniques that are used in the tasks of language modelling, including Long Short-Term Memory (LSTM), Recurrent Neural Networks (RNN), and Sequence to Sequence architecture (Seq2seq). 193 Актуальные проблемы современной науки, № 4, 2016 1. <...> N-grams The most frequently used language models are based on the n-gram statistics, which are basically word co-occurrence frequencies. <...> Before neural networks were applied to the task of text generation, the most successful language models were based on n-grams. <...> Feedforward Neural Networks One of the first attempts to build a language model using a simple neural network was made in [2]. <...> The simplified architecture scheme is presented on Figure 1: Figure 1: Neural network feedforward architecture, proposed by Yoshua Bengio et al. <...> It was reported that this model can be effectively applied for relatively large contexts and also demonstrated the perplexity reduction of 33% (from 336 to 252) in comparison to the state-of-art trigram models. 3. <...> Recurrent Neural Networks (RNN) As in the case with n-gram models, a simple NN architecture mentioned above only allows to take into account a predefined number of context words while making a prediction. <...> In order to handle this problem a recurrent neural network architecture was proposed by Tomas Mikolov[1,3]. <...> Mikolov’s thesis indicates that it is <...>