Context-aware learning for generative models

Perdikis, Serafeim; Leeb, Robert; Chavarriaga, Ricardo; Millán, José del R.

doi:10.1109/TNNLS.2020.3011671

Publication type:	Article in scientific journal
Type of review:	Not specified
Title:	Context-aware learning for generative models
Authors:	Perdikis, Serafeim Leeb, Robert Chavarriaga, Ricardo Millán, José del R.
et. al:	No
DOI:	10.1109/TNNLS.2020.3011671
Published in:	IEEE Transactions on Neural Networks and Learning Systems
Issue Date:	10-Aug-2020
Publisher / Ed. Institution:	IEEE
ISSN:	2162-237X 2162-2388
Language:	English
Subjects:	Statistics; Machine Learning
Subject (DDC):	006: Special computer methods
Abstract:	This work studies the class of algorithms for learning with side-information that emerge by extending generative models with embedded context-related variables. Using finite mixture models (FMM) as the prototypical Bayesian network, we show that maximum-likelihood estimation (MLE) of parameters through expectation-maximization (EM) improves over the regular unsupervised case and can approach the performances of supervised learning, despite the absence of any explicit ground truth data labeling. By direct application of the missing information principle (MIP), the algorithms' performances are proven to range between the conventional supervised and unsupervised MLE extremities proportionally to the information content of the contextual assistance provided. The acquired benefits regard higher estimation precision, smaller standard errors, faster convergence rates and improved classification accuracy or regression fitness shown in various scenarios, while also highlighting important properties and differences among the outlined situations. Applicability is showcased with three real-world unsupervised classification scenarios employing Gaussian Mixture Models. Importantly, we exemplify the natural extension of this methodology to any type of generative model by deriving an equivalent context-aware algorithm for variational autoencoders (VAs), thus broadening the spectrum of applicability to unsupervised deep learning with artificial neural networks. The latter is contrasted with a neural-symbolic algorithm exploiting side-information.
URI:	https://repository.essex.ac.uk/28199/ https://digitalcollection.zhaw.ch/handle/11475/20503
Fulltext version:	Published version
License (according to publishing contract):	Licence according to publishing contract
Departement:	School of Engineering
Organisational Unit:	Institute of Applied Information Technology (InIT)
Appears in collections:	Publikationen School of Engineering

Files in This Item:

There are no files associated with this item.

Show full item record

Perdikis, S., Leeb, R., Chavarriaga, R., & Millán, J. d. R. (2020). Context-aware learning for generative models. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2020.3011671

Perdikis, S. et al. (2020) ‘Context-aware learning for generative models’, IEEE Transactions on Neural Networks and Learning Systems [Preprint]. Available at: https://doi.org/10.1109/TNNLS.2020.3011671.

S. Perdikis, R. Leeb, R. Chavarriaga, and J. d. R. Millán, “Context-aware learning for generative models,” IEEE Transactions on Neural Networks and Learning Systems, Aug. 2020, doi: 10.1109/TNNLS.2020.3011671.

PERDIKIS, Serafeim, Robert LEEB, Ricardo CHAVARRIAGA und José del R. MILLÁN, 2020. Context-aware learning for generative models. IEEE Transactions on Neural Networks and Learning Systems [online]. 10 August 2020. DOI 10.1109/TNNLS.2020.3011671. Verfügbar unter: https://repository.essex.ac.uk/28199/

Perdikis, Serafeim, Robert Leeb, Ricardo Chavarriaga, and José del R. Millán. 2020. “Context-Aware Learning for Generative Models.” IEEE Transactions on Neural Networks and Learning Systems, August. https://doi.org/10.1109/TNNLS.2020.3011671.

Perdikis, Serafeim, et al. “Context-Aware Learning for Generative Models.” IEEE Transactions on Neural Networks and Learning Systems, Aug. 2020, https://doi.org/10.1109/TNNLS.2020.3011671.