Bitte benutzen Sie diese Kennung, um auf die Ressource zu verweisen:
https://doi.org/10.21256/zhaw-18449| Publikationstyp: | Beitrag in wissenschaftlicher Zeitschrift |
| Art der Begutachtung: | Peer review (Publikation) |
| Titel: | Evaluation of robust outlier detection methods for zero-inflated complex data |
| Autor/-in: | Templ, Matthias Gussenbauer, J. Filzmoser, P. |
| et. al: | No |
| DOI: | 10.1080/02664763.2019.1671961 10.21256/zhaw-18449 |
| Erschienen in: | Journal of Applied Statistics |
| Band(Heft): | 47 |
| Heft: | 7 |
| Seite(n): | 1144 |
| Seiten bis: | 1167 |
| Erscheinungsdatum: | 2019 |
| Verlag / Hrsg. Institution: | Taylor & Francis |
| ISSN: | 0266-4763 1360-0532 |
| Sprache: | Englisch |
| Schlagwörter: | Outlier detection; Zeros; Robust method; Household expenditure |
| Fachgebiet (DDC): | 005: Computerprogrammierung, Programme und Daten 510: Mathematik |
| Zusammenfassung: | Outlier detection can be seen as a pre-processing step for locating data points in a data sample, which do not conform to the majority of observations. Various techniques and methods for outlier detection can be found in the literature dealing with different types of data. However, many data sets are inflated by true zeros and, in addition, some components/variables might be of compositional nature. Important examples of such data sets are the Structural Earnings Survey, the Structural Business Statistics, the European Statistics on Income and Living Conditions, tax data or – as in this contribution – household expenditure data which are used, for example, to estimate the Purchase Power Parity of a country. In this work, robust univariate and multivariate outlier detection methods are compared by a complex simulation study that considers various challenges included in data sets, namely structural (true) zeros, missing values, and compositional variables. These circumstances make it difficult or impossible to flag true outliers and influential observations by well-known outlier detection methods. Our aim is to assess the performance of outlier detection methods in terms of their effectiveness to identify outliers when applied to challenging data sets such as the household expenditures data surveyed all over the world. Moreover, different methods are evaluated through a close-to-reality simulation study. Differences in performance of univariate and multivariate robust techniques for outlier detection and their shortcomings are reported. We found that robust multivariate methods outperform robust univariate methods. The best performing methods in finding the outliers and in providing a low false discovery rate were found to be the generalized S estimators (GSE), the BACON-EEM algorithm and a compositional method (CoDa-Cov). In addition, these methods performed also best when the outliers are imputed based on the corresponding outlier detection method and indicators are estimated from the data sets. |
| URI: | https://digitalcollection.zhaw.ch/handle/11475/18449 |
| Volltext Version: | Publizierte Version |
| Lizenz (gemäss Verlagsvertrag): | CC BY-NC-ND 4.0: Namensnennung - Nicht kommerziell - Keine Bearbeitungen 4.0 International |
| Departement: | School of Engineering |
| Organisationseinheit: | Institut für Datenanalyse und Prozessdesign (IDP) |
| Enthalten in den Sammlungen: | Publikationen School of Engineering |
Dateien zu dieser Ressource:
| Datei | Beschreibung | Größe | Format | |
|---|---|---|---|---|
| 2020_Templ-etal_Robust-outlier-detection-complex-data.pdf | 2.15 MB | Adobe PDF |  Öffnen/Anzeigen |
Zur Langanzeige
Templ, M., Gussenbauer, J., & Filzmoser, P. (2019). Evaluation of robust outlier detection methods for zero-inflated complex data. Journal of Applied Statistics, 47(7), 1144–1167. https://doi.org/10.1080/02664763.2019.1671961
Templ, M., Gussenbauer, J. and Filzmoser, P. (2019) ‘Evaluation of robust outlier detection methods for zero-inflated complex data’, Journal of Applied Statistics, 47(7), pp. 1144–1167. Available at: https://doi.org/10.1080/02664763.2019.1671961.
M. Templ, J. Gussenbauer, and P. Filzmoser, “Evaluation of robust outlier detection methods for zero-inflated complex data,” Journal of Applied Statistics, vol. 47, no. 7, pp. 1144–1167, 2019, doi: 10.1080/02664763.2019.1671961.
TEMPL, Matthias, J. GUSSENBAUER und P. FILZMOSER, 2019. Evaluation of robust outlier detection methods for zero-inflated complex data. Journal of Applied Statistics. 2019. Bd. 47, Nr. 7, S. 1144–1167. DOI 10.1080/02664763.2019.1671961
Templ, Matthias, J. Gussenbauer, and P. Filzmoser. 2019. “Evaluation of Robust Outlier Detection Methods for Zero-Inflated Complex Data.” Journal of Applied Statistics 47 (7): 1144–67. https://doi.org/10.1080/02664763.2019.1671961.
Templ, Matthias, et al. “Evaluation of Robust Outlier Detection Methods for Zero-Inflated Complex Data.” Journal of Applied Statistics, vol. 47, no. 7, 2019, pp. 1144–67, https://doi.org/10.1080/02664763.2019.1671961.
Alle Ressourcen in diesem Repository sind urheberrechtlich geschützt, soweit nicht anderweitig angezeigt.