Divide et impera : multi-transformer architectures for complex NLP-tasks

Helland, Solveig; Gavagnin, Elena; de Spindler, Alexandre

doi:10.21256/zhaw-30386

Please use this identifier to cite or link to this item: https://doi.org/10.21256/zhaw-30386

Publication type:	Conference paper
Type of review:	Peer review (publication)
Title:	Divide et impera : multi-transformer architectures for complex NLP-tasks
Authors:	Helland, Solveig Gavagnin, Elena de Spindler, Alexandre
et. al:	No
DOI:	10.21256/zhaw-30386
Proceedings:	Proceedings of the 8th edition of the Swiss Text Analytics Conference
Page(s):	70
Pages to:	75
Conference details:	8th Swiss Text Analytics Conference – SwissText 2023, Neuchâtel, Switzerland, 12-14 June 2023
Issue Date:	2023
Publisher / Ed. Institution:	Association for Computational Linguistics
Language:	English
Subject (DDC):	410.285: Computational linguistics
Abstract:	The growing capabilities of transformer models pave the way for solving increasingly complex NLP tasks. A key to supporting applicationspecific requirements is the ability to fine-tune. However, compiling a fine-tuning dataset tailored to complex tasks is tedious and results in large datasets, limiting the ability to control transformer output. We present an approach in which complex tasks are divided into simpler subtasks. Multiple transformer models are fine-tuned to one subtask each, and lined up to accomplish the complex task. This simplifies the compilation of fine-tuning datasets and increases overall controllability. Using the example of reducing gender bias as a complex task, we demonstrate our approach and show that it performs better than using a single model.
URI:	https://aclanthology.org/2023.swisstext-1.7 https://digitalcollection.zhaw.ch/handle/11475/30386
Fulltext version:	Published version
License (according to publishing contract):	CC BY 4.0: Attribution 4.0 International
Departement:	School of Management and Law
Organisational Unit:	Institute of Business Information Technology (IWI)
Appears in collections:	Publikationen School of Management and Law

Files in This Item:

File	Description	Size	Format
2023_Helland-etal_Multi-transformer-architectures-for-complex-NLP-tasks.pdf		157.83 kB	Adobe PDF	View/Open

Show full item record

Helland, S., Gavagnin, E., & de Spindler, A. (2023). Divide et impera : multi-transformer architectures for complex NLP-tasks [Conference paper]. Proceedings of the 8th Edition of the Swiss Text Analytics Conference, 70–75. https://doi.org/10.21256/zhaw-30386

Helland, S., Gavagnin, E. and de Spindler, A. (2023) ‘Divide et impera : multi-transformer architectures for complex NLP-tasks’, in Proceedings of the 8th edition of the Swiss Text Analytics Conference. Association for Computational Linguistics, pp. 70–75. Available at: https://doi.org/10.21256/zhaw-30386.

S. Helland, E. Gavagnin, and A. de Spindler, “Divide et impera : multi-transformer architectures for complex NLP-tasks,” in Proceedings of the 8th edition of the Swiss Text Analytics Conference, 2023, pp. 70–75. doi: 10.21256/zhaw-30386.

HELLAND, Solveig, Elena GAVAGNIN und Alexandre DE SPINDLER, 2023. Divide et impera : multi-transformer architectures for complex NLP-tasks. In: Proceedings of the 8th edition of the Swiss Text Analytics Conference [online]. Conference paper. Association for Computational Linguistics. 2023. S. 70–75. Verfügbar unter: https://aclanthology.org/2023.swisstext-1.7

Helland, Solveig, Elena Gavagnin, and Alexandre de Spindler. 2023. “Divide et Impera : Multi-Transformer Architectures for Complex NLP-Tasks.” Conference paper. In Proceedings of the 8th Edition of the Swiss Text Analytics Conference, 70–75. Association for Computational Linguistics. https://doi.org/10.21256/zhaw-30386.

Helland, Solveig, et al. “Divide et Impera : Multi-Transformer Architectures for Complex NLP-Tasks.” Proceedings of the 8th Edition of the Swiss Text Analytics Conference, Association for Computational Linguistics, 2023, pp. 70–75, https://doi.org/10.21256/zhaw-30386.