Neural Network Models of a Grammar Parser for the Kalmyk Language: Training Experience

Abstract

Introduction. Kalmyk language presents unique challenges for NLP due to its agglutinative rich morphology and ­limited available resources. The objective is to consider various neural network models of grammar analysis for the Kalmyk language. Materials and Methods. Several neural network models were selected for training: Lemma Accuracy, Levenshtein Lemma Distance, Morph Accuracy, Morph F1. Neural network model training methods, analysis, and comparison methods were used. The training dataset used consisted of an organizational part in depth of 2 495 sentences (including 35 049 tokens), a validation part in depth of 311 sentences (including 3 991 tokens), and a test part in depth of 313 sentences (including 3 627 tokens). Results. This paper proposes a high-performing morphological analyzer for Kalmyk language using neural network techniques. The analyzer is able to jointly predict a lemmata and morphological tags for each word in a sentence. Due to the scarcity of the data, morphological analyzers for low-resource languages often utilizes rule-based and statistical approa­ches. However, there are few studies based on deep learning approaches. Firstly, our model inputs word embedding based on characters and contextual embeddings ge­nerated by the pretrained cross lingual model XLM-RoBERTa. Secondly, the proposed model is based on a sequential architecture which inputs surface words and predicts minimum edit actions between surface words and lemmas instead of predicting characters in lemmas. Thirdly, our system does not require pretrained embeddings for the Kalmyk language  and additional morphological segmentation tools. We conducted several experiments to show that our model outperforms other models.