tumnigИзвестия высших учебных заведений. Нефть и газOil and Gas Studies0445-01083033-8174Industrial University of Tyumen10.31660/0445-0108-2023-2-35-54tumnig-1122Research ArticleГЕОЛОГИЯ, ПОИСКИ И РАЗВЕДКА МЕСТОРОЖДЕНИЙ НЕФТИ И ГАЗАGEOLOGY, PROSPECTING AND EXPLORATION OF OIL AND GAS FIELDSЦифровой керн: нейросетевое распознавание текстовой геолого-геофизической информацииDigital core: neural network recognition of textual geological and geophysical informationhttps://orcid.org/0000-0001-5983-4040КатановЮ. Е.KatanovYu. E.

Катанов Юрий Евгеньевич, кандидат геолого-минералогических наук, доцент кафедры прикладной геофизики, ведущий научный сотрудник лаборатории технологий капитального ремонта скважин и интенсификации притока

г. Тюмень

Yuri E. Katanov, Candidate of Geology and Mineralogy, Associate Professor at the Department of Applied Geophysics, Leading Researcher at Well Workover Technology and Production Stimulation Laboratory

Tyumen

katanov-juri@rambler.ruАристовА. И.AristovA. I.

Аристов Артем Игоревич, лаборант лаборатории цифровых исследований в нефтегазовой отрасли

г. Тюмень

Artyom I. Aristov, Assistant at the Laboratory of Digital Research in the Oil and Gas Industry

Tyumen

ЯгафаровА. К.YagafarovA. K.

Ягафаров Алик Каюмович, доктор геолого-минералогических наук, профессор

г. Тюмень

Alik K. Yagafarov, Doctor of Geology and Mineralogy, Professor

Tyumen

НоврузовО. Д.NovruzovO. D.

Новрузов Орхан Джанполад оглы, лаборант лаборатории цифровых исследований в нефтегазовой отрасли

г. Тюмень

Orchan D. Novruzov, Assistant at the Laboratory of Digital Research in the Oil and Gas Industry

Tyumen

Тюменский индустриальный университетРоссияIndustrial University of TyumenRussian Federation202314072023033554Copyright © Катанов Ю.Е., Аристов А.И., Ягафаров А.К., Новрузов О.Д., 20232023Катанов Ю.Е., Аристов А.И., Ягафаров А.К., Новрузов О.Д.Katanov Y.E., Aristov A.I., Yagafarov A.K., Novruzov O.D.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://tumnig.tyuiu.ru/jour/article/view/1122

Представлен алгоритм аналого-цифрового преобразования первичной геологогеофизической информации (на примере идентификации литотипов горных пород на базе текстового описания физического керна).В рамках работы реализовано комплексирование трех видов научных исследований — поисковое, междисциплинарное и прикладное при формировании исходной базы качественных данных.Описаны распространенные алгоритмы для классификации текстовой информации и механизм предобработки исходных данных с использованием токенизации.Концепция распознавания текстовых образов реализована с привлечением методов искусственного интеллекта.Для создания нейросетевой модели распознавания текстовой геолого-геофизической информации использован язык программирования Python в сочетании с технологиями сверточных нейросетей для классификации текста (TextCNN), сетей двунаправленной длительной-кратковременной памяти (BiLSTM) и сетей представлений двунаправленного кодера (BERT).Стек данных технологий и языка программирования Python, после разработки и апробации базового варианта нейросетевой модели распознавания качественной информации, обеспечили приемлемый уровень работы алгоритма цифровой трансформации текстовых данных.Наилучший результат (текущая версия нейросетевой модели 1.0; более 3 000 примеров для обучения и тестирования) достигнут при использовании алгоритма распознавания текстовых данных на базе BERT с точностью на валидационном сете (Validation Accuracy) ~0.830173 (25 эпоха), с потерями на валидационном сете (Validation Loss) ~0.244719, с потерями во время обучения (Training Loss) ~0.000984 и вероятностью распознавания исследуемых литотипов горных пород более 95 %.Определены механизмы модификации кода для дальнейшего улучшения точности текстового прогноза на базе созданной нейросети.

The algorithm of analog-to-digital conversion of primary geological and geophysical information (on the example of identification of rock lithotypes based on the text description of the physical core) is presented.As part of the work, a combination of three types of scientific research - prospecting, interdisciplinary and applied, in the formation of the initial base of qualitative data is implemented.Common algorithms for textual information classification and mechanism of initial data preprocessing using tokenization are described.The concept of text pattern recognition is implemented using artificial intelligence methods.For creation of the neural network model of textual geological and geophysical information recognition the Python programming language is used in combination with the convolutional neural network technologies for text classification (TextCNN), bi-directional long-shortterm memory networks (BiLSTM) and bi-directional coder representation networks (BERT).The stack of these technologies and the Python programming language, after developing and testing the basic version of the neural network model of qualitative information recognition, provided an acceptable level of performance of the algorithm of digital transformation of text data.The best result (the current version of neural network model is 1.0; more than 3000 examples for training and testing) was achieved when using the algorithm of text data recognition based on BERT with an accuracy on the validation network (Validation Accuracy) ~0.830173 (25th epoch), with Validation Loss ~0.244719, with Training Loss ~0.000984 and probability of recognition of the studied rock lithotypes more than 95 %.The mechanisms of code modification for further improvement of textual prediction accuracy based on the created neural network were determined.

распознавание символовкластеризация текстаконтекстная информацияинтерпретациятокенизациянейросетьвыборкаcharacter recognitiontext clusteringcontent informationinterpretationtokenizationneural networksamplingReferencesКатанов, Ю. Е. Исследование влияния капиллярных явлений при фильтрации двухфазных несмешивающихся жидкостей в пористых средах / Ю. Е. Катанов, А. К. Ягафаров, И. И. Клещенко [и др.]. – DOI 10.31660/0445-0108-2020-1-19-29. – Текст : непосредственный // Известия высших учебных заведений. Нефть и газ. – 2020. – № 1. – С. 19–29.Katanov, Yu. E., Yagafarov, A. K., Kleshchenko, I. I. Savina, M. E., Shlein, G. A., & Yagafarov, A. K. (2020). Studying the influence of capillary phenomena in two-phase filtration of immiscible fluids in porous media. Oil and Gas Studies, (1), pp. 19-29. (In Russian). DOI: 10.31660/0445-0108-2020-1-19-29Katanov, Yu. E. A probabilistic and statistical model of rock deformation / Yu. E. Katanov. – Text : electronic // E3S Web of Conferences. – 2021. – Vol. 266. – URL: https://doi.org/10.1051/e3sconf/202126603011. – Published: June, 04, 2021.Katanov, Yu. E. (2021). A probabilistic and statistical model of rock deformation. E3S Web of Conferences, 266. (In English). Available at: https://doi.org/10.1051/e3sconf/202126603011Katanov, Yu. E. Geological and mathematical description of the rocks strain during behavior of the producing solid mass in compression (Tension) / Yu. E. Katanov, Yu. V. Vaganov, M. V. Listak. – DOI 10.33271/mining15.04.091. – Direct text // Journal of Mines, Metals & Fuels. – 2020. – Vol. 68, Issue 9. – P. 285–293.Katanov, Yu. E., Vaganov, Yu. V., & Listak, M. V. (2020). Geological and mathematical description of the rocks strain during behavior of the producing solid mass in compression (Tension). Journal of Mines, Metals & Fuels, 68(9), pp. 285-293. (In English). DOI: 10.33271/mining15.04.091Ломов, П. А. Аугментация обучающего набора при обучении нейросетевой языковой модели для наполнения онтологии / П. А Ломов, М. Л. Малоземова.– DOI 10.37614/2307-5252.2021.5.12.002 – Текст : непосредственный // Труды Кольского научного центра РАН. Информационные технологии. – 2021. – Вып. 12. – Т. 12, № 5. – С. 22–34.Lomov, P. A., & Malozemova, M. L. (2021). Training set augmentation in training neural-network language model for ontology population. Тransactions of the Kola Science Centre. Information technologies. Series 12, 12(5), pp. 22-34. (In Russian). DOI: 10.37614/2307-5252.2021.5.12.002Сайгин, А. А. Векторизация нормативно-справочной информации с помощью модели нейронной сети BERT / А. А. Сайгин, Н. П. Плотникова. – Текст : электронный // Информационные технологии и математическое моделирование в управлении сложными системами : электронный журнал. – 2021. – № 2. – С. 52–59. – URL: https://doi.org/10.26731/2658-3704.2021.2(10).52-59.Saygin, A. A., & Plotnikova, N. P. (2021). Vectorization of regulatoryreference information using the BERT neural network. Information technology and mathematical modeling in the management of complex systems, (2), pp. 52-59. (In Russian). Available at: https://doi.org/10.26731/2658-3704.2021.2(10).52-59Соломин, А. А. Современные подходы к мультиклассовой классификации интентов на основе предобученных трансформеров / А. А. Соломин, Ю. А. Иванова. – DOI 10.17586/2226-1494-2020-20-4-532-538. – Текст : непосредственный // Научно-технический вестник информационных технологий, механики и оптики. – 2020. – Т. 20, № 4. – С. 532–538.Solomin, A. A., & Ivanova, Yu. A. (2020). Modern approaches to multiclass intent classification based on pre-trained transformers. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 20(4), pp. 532-538. (In Russian). DOI: 10.17586/2226-1494-2020-20-4-532-538Text classification models for the automatic detection of nonmedical prescription medication use from social media / M. A. Al-Garadi, Y. C. Yang, H. Cai [et al.]. – Text : elctronic // BMC medical informatics and Decision Making. – 2021. – Vol. 21. – URL: https://doi.org/10.1186/s12911-021-01394-0. Published: January, 26, 2021.Al-Garadi, M. A., Yang, Y. C., Cai, H., Ruan, Y., O'Connor, K., Graciela, G. H., & Sarker, A. (2021). Text classification models for the automatic detection of nonmedical prescription medication use from social media. BMC medical informatics and Decision Making, 21. (In English). Available at: https://doi.org/10.1186/s12911-021-01394-0Comparison of Pre-Trained Language Models for Multi-Class Text Classification in the Financial Domain / Y. Arslan, K. Allix, L. Veiber [et al.]. – DOI 10.1145/3442442.3451375. – Direct text // Companion Proceedings of the Web Conference. – 2021. – P. 260–268.Arslan, Y., Allix, K., Veiber, L., Lothritz, C., Bissyandé, T. F., Klein, J., & Goujon, A. (2021). Comparison of Pre-Trained Language Models for Multi-Class Text Classification in the Financial Domain. Companion Proceedings of the Web Conference. pp. 260-268. (In English). DOI: 10.1145/3442442.3451375Çelikten, A. Turkish Medical Text Classification Using BERT / A. Çelikten, H. Bulut. – Text : electronic // 2021 29th Signal Processing and Communications Applications Conference (SIU). IEEE. – 2021. – URL: https://doi.org/10.1109/SIU53274.2021.9477847.Çelikten, A., & Bulut, H. Turkish Medical Text Classification Using BERT. (2021). 2021 29th Signal Processing and Communications Applications Conference (SIU). IEEE. (In English). Available at: https://doi.org/10.1109/SIU53274.2021.9477847Das, S. Identification of Cognitive Learning Complexity of Assessment Questions Using Multi-class Text Classification / S. Das, S. K. D. Mandal, A. Basu. – DOI 10.30935/cedtech/8341. – Text : electronic // Contemporary Educational Technology. – 2020. – Vol. 12, Issue 2. – URL: https://doi.org/10.30935/cedtech/8341.Das, S., Mandal, S. K. D., & Basu, A. (2020). Identification of Cognitive Learning Complexity of Assessment Questions Using Multi-class Text Classification. Contemporary Educational Technology, 12(2). (In English). Available at: https://doi.org/10.30935/cedtech/8341Auto-labelling entities in low-resource text : a geological case study / M. Enkhsaikhan, W. Liu, E. J. Holden, P. Duurin. – DOI: 10.1007/s10115-020-01532-6. – Direct text // Knowledge and Information Systems. – 2021. – Vol. 63. – P. 695–715.Enkhsaikhan, M., Liu, W., Holden, E. J., & Duuring, P. (2021). Autolabelling entities in low-resource text: a geological case study. Knowledge and Information Systems, 63, pp. 695-715. (In English). DOI: 10.1007/s10115-020-01532-6Gao, X. Named entity recognition in material field based on Bert-BILSTMAttention-CRF / X. Gao, Q. Li. – DOI 10.1109/TOCS53301.2021.9688665. – Direct text // 2021 IEEE Conference on Telecommunications, Optics and Computer Science (TOCS). – 2021. – P. 955–958.Gao, X., & Li, Q. (2021). Named entity recognition in material field based on Bert-BILSTM-Attention-CRF. 2021 IEEE Conference on Telecommunications, Optics and Computer Science (TOCS), pp. 955-958. (In English). DOI: 10.1109/TOCS53301.2021.9688665Glazkova, A. A Comparative study of Feature Types for Age-Based Text Classification / A. Glazkova, Yu. Egorov, M. Glazkov. – DOI 10.1007/978-3-030-72610-2_9. – Direct text // International Conference on Analysis of Images, Social Networks and Texts. – 2020. – P. 120–134.Glazkova, A., Egorov, Y., & Glazkov, M. (2020). A Comparative study of Feature Types for Age-Based Text Classification. International Conference on Analysis of Images, Social Networks and Texts, pp. 120-134. (In English). DOI: 10.1007/978-3-030-72610-2_9Evaluating Transformer-Based Multilingual Text Classification / S. Groenwold, S. Honnavalli, L. Ou [et al.]. – Text : electronic // arXiv:2004.13939v2 [cs.CL]. – 2020. – URL: https://doi.org/10.48550/arXiv.2004.13939.Groenwold, S., Honnavalli, S., Ou, L., Parekh, A., Levy, S., Mirza, D., & Wang, W. Y. (2021). Evaluating Transformer-Based Multilingual Text Classification. arXiv:2004.13939v2 [cs.CL]. (In English). Available at: https://doi.org/10.48550/arXiv.2004.13939Research on a geological entity relation extraction model for gold mine based on BERT / X. Huang, Y. Zhu, L. Fu [et al.]. – DOI 10.12090/j.issn.1006-6616.2021.27.03.035. – Direct text // Journal of Geomechanics. – 2021. – Vol. 27, Issue 3. – P. 391–399.Huang, X., Zhu, Y., Fu, L., Liu, Y., Tang, K., & Li, J. (2021). Research on a geological entity relation extraction model for gold mine based on BERT. Journal of Geomechanics, 27(3), pp. 391-399. (In English). DOI: 10.12090/j.issn.1006-6616.2021.27.03.035BERT for Russian news clustering / A. S. Kabaev, S. V. Khaustov, N. E. Gorlova, A. V. Kalmykov. – Text : electronic // Computational Linguistics and Intellectual Technologies. – 2021. – URL: https://doi.org/10.28995/2075-7182-2021-20-385-390.Kabaev, A. S., Khaustov, S. V., Gorlova, N. E., & Kalmykov, A. V. (2021). BERT for Russian news clustering. (In English). Available at: https://doi.org/10.28995/2075-7182-2021-20-385-390Chinese named entity recognition in the geoscience domain based on BERT / X. Lv, Z. Xie, D. Xu [et al.]. – Text : electronic // Earth and Space Science. – 2022. – Vol. 9, Issue 3. – URL: https://doi.org/10.1029/2021EA002166. – Published: February, 14, 2022.Lv, X., Xie, Z., Xu, D., Jin, X., Ma, K., Tao, L., Qiu, Q., & Pan, Y. (2022). Chinese named entity recognition in the geoscience domain based on BERT. Earth and Space Science, 9(3). (In English). Available at: https://doi.org/10.1029/2021EA002166What is this article about? Generative summarization with the BERT model in the geosciences domain / K. Ma, M. Tian, Y. Tan [et al.]. – DOI 10.1007/s12145-021-00695-2. – Direct text // Earth Science Informatics. – 2022. – Vol. 15. – P. 21–36.Ma, K., Tian, M., Tan, Y., Xie, X., & Qiu, Q. (2022). What is this article about? Generative summarization with the BERT model in the geosciences domain. Earth Science Informatics, (15) pp. 21-36. (In English). DOI: 10.1007/s12145-021-00695-2Piao, G. Scholarly Text Classification with Sentence BERT and Entity Embeddings / G. Piao. – DOI 10.1007/978-3-030-75015-2_8. – Direct text // PAKDD 2021 : Trends and Applications in Knowledge Discovery and Data Mining. – 2021. – P. 79–87.Piao, G. (2021). Scholarly Text Classification with Sentence BERT and Entity Embeddings. PAKDD 2021: Trends and Applications in Knowledge Discovery and Data Mining, pp. 79-87. (In English). DOI: 10.1007/978-3-030-75015-2_8Prabhu, S. Multi-class Text Classification using BERT-based Active Learning / S. Prabhu, M. Mohamed, H. Misra. – Text : electronic // arXiv:2104.14289v2 [cs.IR]. – 2021. – URL: https://doi.org/10.48550/arXiv.2104.14289.Prabhu, S., Mohamed, M., & Misra, H. (2021). Multi-class Text Classification using BERT-based Active Learning. arXiv:2104.14289v2 [cs.IR]. (In English). Available at: https://doi.org/10.48550/arXiv.2104.14289A Fine-Tuned BERT-Based Transfer Learning Approach for Text Classification / R. Qasim, W. H. Bangyal, M. A. Alqarni, A. Ali Almazroi. – Text : electronic // Journal of Healthcare Engineering. – 2022. – URL: https://doi.org/10.1155/2022/3498123. – Published: January, 07, 2022.Qasim, R., Bangyal, W. H., Alqarni, M. A., & Ali Almazroi, A. (2022). A Fine-Tuned BERT-Based Transfer Learning Approach for Text Classification. Journal of Healthcare Engineering. (In English). Available at: https://doi.org/10.1155/2022/3498123Text classification on software requirements specifications using transformer models / D. Kici, A. Bozanta, M. Cevik. [et al.]. – DOI 10.5555/3507788.3507811. – Direct text // Proceedings of the 31st Annual International Conference on Computer Science and Software Engineering. – 2021. – P. 163–172.Kici, D., Bozanta, A., Cevik, M., Parikh, D., & Başar, A. (2021). Text classification on software requirements specifications using transformer models. Proceedings of the 31st Annual International Conference on Computer Science and Software Engineering, pp. 163-172. (In English). DOI: 10.5555/3507788.3507811Lun, C. H. Extracting Knowledge with NLP from Massive Geological Documents / C. H. Lun, T. Hewitt, S. Hou // 82nd EAGE Annual Conference & Exhibition. European Association of Geoscientists & Engineers. – 2021. – URL: https://doi.org/10.3997/2214-4609.202112807.Lun, C. H., Hewitt, T., & Hou, S. (2021). Extracting Knowledge with NLP from Massive Geological Documents. 82nd EAGE Annual Conference & Exhibition. European Association of Geoscientists & Engineers. (In English). Available at: https://doi.org/10.3997/2214-4609.202112807Smetanin, S. I. Toxic comments detection in Russian / S. I. Smetanin. – DOI 10.28995/2075-7182-2020-19-1149-1159. – Direct text // Computational Linguistics and Intellectual Technologies. – 2020. – P. 1149–1159.Smetanin, S. I. (2020). Toxic comments detection in Russian. Computational Linguistics and Intellectual Technologies, pp. 1149-1159. (In English). DOI: 10.28995/2075-7182-2020-19-1149-1159

The authors declare that there are no conflicts of interest present.