Impact of mislabelling on deep learning methods and strategies for improvement

Dettling, Marcel; Frey, Martin; Walser, Manuel; Haas, Patrick

Publication type:	Conference paper
Type of review:	Peer review (abstract)
Title:	Impact of mislabelling on deep learning methods and strategies for improvement
Authors:	Dettling, Marcel Frey, Martin Walser, Manuel Haas, Patrick
et. al:	No
Proceedings:	Conference Proceedings of the 63rd ISI World Statistics Congress
Conference details:	63rd ISI World Statistics Congress, virtual, 11-16 July 2021
Issue Date:	2021
Language:	English
Subjects:	Deep learning; Mislabelling; Sequential data analysis; Time series classification; Sports analytics
Subject (DDC):	006: Special computer methods
Abstract:	This contribution revolves around classifying football player actions with 1-dimensional convolutional neural networks (CNNs) based on 6-channel inertial motion unit (IMU) data arising from tracking devices worn by the players. Our training and test data consist of eight games, where humans labelled ball actions by inspecting video records. Unfortunately, these labels are far from perfect due to various reasons (e.g., sloppiness, not all players respectively ball actions visible in the record, ambiguity what a ball action is, etc.). Such mislabelled data provide challenges on several levels. First, performance evaluation with poorly annotated data can be strongly misleading, indicating inferior performance than what is truly achieved. Second, the question is what amount of mislabelled data deep artificial neural networks can tolerate before they break down. We try to shed some light on the magnitude of these effects by simulation studies on the football data, as well as some standard machine learning datasets such as MNIST (numbers) and Fashion-MNIST (clothes). Third, we present some efficient strategies to overcome the issue with imperfect labels and aim to provide some guidelines how to efficiently invest effort in labelling data.
URI:	https://digitalcollection.zhaw.ch/handle/11475/22977
Fulltext version:	Published version
License (according to publishing contract):	Licence according to publishing contract
Departement:	School of Engineering
Organisational Unit:	Institute of Data Analysis and Process Design (IDP)
Published as part of the ZHAW project:	Entwicklung von Algorithmen zur Analyse von Fussballspielern und Spielsituationen anhand von Bewegungsdaten
Appears in collections:	Publikationen School of Engineering

Files in This Item:

There are no files associated with this item.

Show full item record

Dettling, M., Frey, M., Walser, M., & Haas, P. (2021). Impact of mislabelling on deep learning methods and strategies for improvement. Conference Proceedings of the 63rd ISI World Statistics Congress.

Dettling, M. et al. (2021) ‘Impact of mislabelling on deep learning methods and strategies for improvement’, in Conference Proceedings of the 63rd ISI World Statistics Congress.

M. Dettling, M. Frey, M. Walser, and P. Haas, “Impact of mislabelling on deep learning methods and strategies for improvement,” in Conference Proceedings of the 63rd ISI World Statistics Congress, 2021.

DETTLING, Marcel, Martin FREY, Manuel WALSER und Patrick HAAS, 2021. Impact of mislabelling on deep learning methods and strategies for improvement. In: Conference Proceedings of the 63rd ISI World Statistics Congress. Conference paper. 2021

Dettling, Marcel, Martin Frey, Manuel Walser, and Patrick Haas. 2021. “Impact of Mislabelling on Deep Learning Methods and Strategies for Improvement.” Conference paper. In Conference Proceedings of the 63rd ISI World Statistics Congress.

Dettling, Marcel, et al. “Impact of Mislabelling on Deep Learning Methods and Strategies for Improvement.” Conference Proceedings of the 63rd ISI World Statistics Congress, 2021.