Parallel encoder-decoder framework for image captioning

Saeidimesineh, Reyhane; Adibi, Peyman; Karshenas, Hossein; Darvishy, Alireza

doi:10.1016/j.knosys.2023.111056

Publication type:	Article in scientific journal
Type of review:	Peer review (publication)
Title:	Parallel encoder-decoder framework for image captioning
Authors:	Saeidimesineh, Reyhane Adibi, Peyman Karshenas, Hossein Darvishy, Alireza
et. al:	No
DOI:	10.1016/j.knosys.2023.111056
Published in:	Knowledge-Based Systems
Volume(Issue):	282
Issue:	111056
Issue Date:	2023
Publisher / Ed. Institution:	Elsevier
ISSN:	0950-7051 1872-7409
Language:	English
Subjects:	Parallelization; Encoder–decoder framework; Image captioning; Natural language processing
Subject (DDC):	006: Special computer methods
Abstract:	Recent progress in deep learning has led to successful utilization of encoder–decoder frameworks inspired by machine translation in image captioning models. The stacking of layers in encoders and decoders has made it possible to use several modules in encoders and decoders. However, just one type of module in encoder or decoder has been used in stacked models. In this research, we propose a parallel encoder–decoder framework that aims to take advantage of multiple of types modules in encoders and decoders, simultaneously. This framework contains augmented parallel blocks, which include stacking modules or non-stacked ones. Then, the results of the blocks are integrated to extract higher-level semantic concepts. This general idea is not limited to image captioning and can be customized for many applications that utilize encoder–decoder frameworks. We evaluated our proposed method on the MS-COCO dataset and achieved state-of-the-art results. We got 149.92 for CIDEr-D metric outperforming state-of-the-art image captioning models.
URI:	https://digitalcollection.zhaw.ch/handle/11475/28970
Fulltext version:	Published version
License (according to publishing contract):	Licence according to publishing contract
Departement:	School of Engineering
Organisational Unit:	Institute of Computer Science (InIT)
Appears in collections:	Publikationen School of Engineering

Files in This Item:

There are no files associated with this item.

Show full item record

Saeidimesineh, R., Adibi, P., Karshenas, H., & Darvishy, A. (2023). Parallel encoder-decoder framework for image captioning. Knowledge-Based Systems, 282(111056). https://doi.org/10.1016/j.knosys.2023.111056

Saeidimesineh, R. et al. (2023) ‘Parallel encoder-decoder framework for image captioning’, Knowledge-Based Systems, 282(111056). Available at: https://doi.org/10.1016/j.knosys.2023.111056.

R. Saeidimesineh, P. Adibi, H. Karshenas, and A. Darvishy, “Parallel encoder-decoder framework for image captioning,” Knowledge-Based Systems, vol. 282, no. 111056, 2023, doi: 10.1016/j.knosys.2023.111056.

SAEIDIMESINEH, Reyhane, Peyman ADIBI, Hossein KARSHENAS und Alireza DARVISHY, 2023. Parallel encoder-decoder framework for image captioning. Knowledge-Based Systems. 2023. Bd. 282, Nr. 111056. DOI 10.1016/j.knosys.2023.111056

Saeidimesineh, Reyhane, Peyman Adibi, Hossein Karshenas, and Alireza Darvishy. 2023. “Parallel Encoder-Decoder Framework for Image Captioning.” Knowledge-Based Systems 282 (111056). https://doi.org/10.1016/j.knosys.2023.111056.

Saeidimesineh, Reyhane, et al. “Parallel Encoder-Decoder Framework for Image Captioning.” Knowledge-Based Systems, vol. 282, no. 111056, 2023, https://doi.org/10.1016/j.knosys.2023.111056.