Mathematical expression recognition using a new deep neural model

Mirkazemy, Abolfazl; Adibi, Peyman; Ehsani, Seyed Mohhamad Saied; Darvishy, Alireza; Hutter, Hans-Peter

doi:10.1016/j.neunet.2023.08.045

Publication type:	Article in scientific journal
Type of review:	Peer review (publication)
Title:	Mathematical expression recognition using a new deep neural model
Authors:	Mirkazemy, Abolfazl Adibi, Peyman Ehsani, Seyed Mohhamad Saied Darvishy, Alireza Hutter, Hans-Peter
et. al:	No
DOI:	10.1016/j.neunet.2023.08.045
Published in:	Neural Networks
Volume(Issue):	167
Page(s):	865
Pages to:	874
Issue Date:	2023
Publisher / Ed. Institution:	Elsevier
ISSN:	0893-6080 1879-2782
Language:	English
Subjects:	Mathematical expression recognition; Deep learning; Encoder–decoder architecture; Attention; Scientific documents accessibility
Subject (DDC):	006: Special computer methods 510: Mathematics
Abstract:	In this paper, we propose a novel deep neural model for Mathematical Expression Recognition (MER). The proposed model uses encoder–decoder transformer architecture that is supported by additional pre/post-processing modules, to recognize the image of mathematical formula and convert it to a well-formed language. A novel pre-processing module based on domain prior knowledge is proposed to generate random pads around the formula’s image to create more efficient feature maps and keeps all the encoder neurons active during the training process. Also, a new post-processing module is developed which uses a sliding window to extract additional position-based information from the feature map, that is proved to be useful in the recognition process. The recurrent decoder module uses the combination of feature maps and the additional position-based information, which takes advantage of a soft attention mechanism, to extract the formula context into the LaTeX well-formed language. Finally, a novel Reinforcement Learning (RL) module processes the decoder output and tunes its results by sending proper feedbacks to the previous steps. The experimental results on im2latex-100k benchmark dataset indicate that each devised pre/post-processing as well as the RL refinement module has a positive effect on the performance of the proposed model. The results also demonstrate the higher accuracy of the proposed model compared to the state-of-the-art methods.
URI:	https://digitalcollection.zhaw.ch/handle/11475/28766
Fulltext version:	Published version
License (according to publishing contract):	Licence according to publishing contract
Departement:	School of Engineering
Organisational Unit:	Institute of Computer Science (InIT)
Appears in collections:	Publikationen School of Engineering

Files in This Item:

There are no files associated with this item.

Show full item record

Mirkazemy, A., Adibi, P., Ehsani, S. M. S., Darvishy, A., & Hutter, H.-P. (2023). Mathematical expression recognition using a new deep neural model. Neural Networks, 167, 865–874. https://doi.org/10.1016/j.neunet.2023.08.045

Mirkazemy, A. et al. (2023) ‘Mathematical expression recognition using a new deep neural model’, Neural Networks, 167, pp. 865–874. Available at: https://doi.org/10.1016/j.neunet.2023.08.045.

A. Mirkazemy, P. Adibi, S. M. S. Ehsani, A. Darvishy, and H.-P. Hutter, “Mathematical expression recognition using a new deep neural model,” Neural Networks, vol. 167, pp. 865–874, 2023, doi: 10.1016/j.neunet.2023.08.045.

MIRKAZEMY, Abolfazl, Peyman ADIBI, Seyed Mohhamad Saied EHSANI, Alireza DARVISHY und Hans-Peter HUTTER, 2023. Mathematical expression recognition using a new deep neural model. Neural Networks. 2023. Bd. 167, S. 865–874. DOI 10.1016/j.neunet.2023.08.045

Mirkazemy, Abolfazl, Peyman Adibi, Seyed Mohhamad Saied Ehsani, Alireza Darvishy, and Hans-Peter Hutter. 2023. “Mathematical Expression Recognition Using a New Deep Neural Model.” Neural Networks 167: 865–74. https://doi.org/10.1016/j.neunet.2023.08.045.

Mirkazemy, Abolfazl, et al. “Mathematical Expression Recognition Using a New Deep Neural Model.” Neural Networks, vol. 167, 2023, pp. 865–74, https://doi.org/10.1016/j.neunet.2023.08.045.