Филигранные парсер-решения тяжёлых задач обработки естественного языка. Елизавета Косарева

Sophisticated Parser Solutions for Complex Natural Language Processing Tasks Elizaveta Kosareva Neural networks are increasingly being used for many tasks, displacing other solutions. However, neural networks as a tool have their drawbacks: high computing costs, the need for a large, high-quality dataset for training, a long cycle of changes with often unstable results, and high latency. Sometimes we face tasks that require high-speed processing of large streams of semantically and grammatically enriched data. As in my case, extracting structured NER entities with the ability to adjust their construction rules. This is where good old grammar parsers come to the rescue. Thanks to the structure of entities, Rust, its traits, and generics combine perfectly with the meta-rules of two-level grammars, helping to create a concise and high-performance solution. Moreover, such a system can not only function as a standalone processor, but also serve as an additional post-processing layer for a small neural network model, reducing errors. Using a small number of parameters, the boundaries of a common entity can be identified, leaving the parser to focus solely on the delicate processing of subentities.