MathNet : a data-centric approach for printed mathematical expression recognition

Schmitt-Koopmann, Felix; Huang, Elaine M.; Hutter, Hans-Peter; Stadelmann, Thilo; Darvishy, Alireza

doi:10.1109/ACCESS.2024.3404834

Please use this identifier to cite or link to this item: https://doi.org/10.21256/zhaw-30744

Publication type:	Article in scientific journal
Type of review:	Peer review (publication)
Title:	MathNet : a data-centric approach for printed mathematical expression recognition
Authors:	Schmitt-Koopmann, Felix Huang, Elaine M. Hutter, Hans-Peter Stadelmann, Thilo Darvishy, Alireza
et. al:	No
DOI:	10.1109/ACCESS.2024.3404834 10.21256/zhaw-30744
Published in:	IEEE Access
Volume(Issue):	12
Page(s):	76963
Pages to:	76974
Issue Date:	23-May-2024
Publisher / Ed. Institution:	IEEE
ISSN:	2169-3536
Language:	English
Subjects:	Data-centric AI; Deep learning; Labeling; Document analysis; Mathematical expression recognition; Pattern recognition
Subject (DDC):	006: Special computer methods
Abstract:	Printed mathematical expression recognition (MER) models are usually trained and tested using LaTeX-generated mathematical expressions (MEs) as input and the LaTeX source code as ground truth. As the same ME can be generated by various different LaTeX source codes, this leads to unwanted variations in the ground truth data that bias test performance results and hinder efficient learning. In addition, the use of only one font to generate the MEs heavily limits the generalization of the reported results to realistic scenarios. We propose a data-centric approach to overcome this problem, and present convincing experimental results: Our main contribution is an enhanced LaTeX normalization to map any LaTeX ME to a canonical form. Based on this process, we developed an improved version of the benchmark dataset im2latex-100k, featuring 30 fonts instead of one. Second, we introduce the real-world dataset realFormula, with MEs extracted from papers. Third, we developed a MER model, MathNet, based on a convolutional vision transformer, with superior results on all four test sets (im2latex-100k, im2latexv2, realFormula, and InftyMDB-1), outperforming the previous state of the art by up to 88.3%.
URI:	https://digitalcollection.zhaw.ch/handle/11475/30744
Related research data:	https://github.com/felix-schmitt/MathNet
Fulltext version:	Published version
License (according to publishing contract):	CC BY 4.0: Attribution 4.0 International
Departement:	School of Engineering
Organisational Unit:	Centre for Artificial Intelligence (CAI) Institute of Computer Science (InIT)
Published as part of the ZHAW project:	Accessible Scientific PDFs for All
Appears in collections:	Publikationen School of Engineering

Files in This Item:

File	Description	Size	Format
2024_SchmittKoopmann-etal_MathNet-Printed-Mathematical-Expression-Recognition.pdf		1.63 MB	Adobe PDF	View/Open

Show full item record

Schmitt-Koopmann, F., Huang, E. M., Hutter, H.-P., Stadelmann, T., & Darvishy, A. (2024). MathNet : a data-centric approach for printed mathematical expression recognition. IEEE Access, 12, 76963–76974. https://doi.org/10.1109/ACCESS.2024.3404834

Schmitt-Koopmann, F. et al. (2024) ‘MathNet : a data-centric approach for printed mathematical expression recognition’, IEEE Access, 12, pp. 76963–76974. Available at: https://doi.org/10.1109/ACCESS.2024.3404834.

F. Schmitt-Koopmann, E. M. Huang, H.-P. Hutter, T. Stadelmann, and A. Darvishy, “MathNet : a data-centric approach for printed mathematical expression recognition,” IEEE Access, vol. 12, pp. 76963–76974, May 2024, doi: 10.1109/ACCESS.2024.3404834.

SCHMITT-KOOPMANN, Felix, Elaine M. HUANG, Hans-Peter HUTTER, Thilo STADELMANN und Alireza DARVISHY, 2024. MathNet : a data-centric approach for printed mathematical expression recognition. IEEE Access. 23 Mai 2024. Bd. 12, S. 76963–76974. DOI 10.1109/ACCESS.2024.3404834

Schmitt-Koopmann, Felix, Elaine M. Huang, Hans-Peter Hutter, Thilo Stadelmann, and Alireza Darvishy. 2024. “MathNet : A Data-Centric Approach for Printed Mathematical Expression Recognition.” IEEE Access 12 (May): 76963–74. https://doi.org/10.1109/ACCESS.2024.3404834.

Schmitt-Koopmann, Felix, et al. “MathNet : A Data-Centric Approach for Printed Mathematical Expression Recognition.” IEEE Access, vol. 12, May 2024, pp. 76963–74, https://doi.org/10.1109/ACCESS.2024.3404834.