Bitte benutzen Sie diese Kennung, um auf die Ressource zu verweisen:
https://doi.org/10.21256/zhaw-3214| Publikationstyp: | Beitrag in wissenschaftlicher Zeitschrift |
| Art der Begutachtung: | Peer review (Publikation) |
| Titel: | Scalable architecture for big data financial analytics : user-defined functions vs. SQL |
| Autor/-in: | Stockinger, Kurt Bundi, Nils Andri Heitz, Jonas Breymann, Wolfgang |
| DOI: | 10.21256/zhaw-3214 10.1186/s40537-019-0209-0 |
| Erschienen in: | Journal of Big Data |
| Band(Heft): | 6 |
| Heft: | 46 |
| Erscheinungsdatum: | 2019 |
| Verlag / Hrsg. Institution: | Springer |
| ISSN: | 2196-1115 |
| Sprache: | Englisch |
| Schlagwörter: | Financial analytics; Query processing; Performance evaluation; User-defined function |
| Fachgebiet (DDC): | 005: Computerprogrammierung, Programme und Daten 332: Finanzwirtschaft |
| Zusammenfassung: | Large financial organizations have hundreds of millions of financial contracts on their balance sheets. Moreover, highly volatile financial markets and heterogeneous data sets within and across banks world-wide make near real-time financial analytics very challenging and their handling thus requires cutting edge financial algorithms. However, due to a lack of data modeling standards, current financial risk algorithms are typically inconsistent and non-scalable. In this paper, we present a novel implementation of a real-world use case for performing large-scale financial analytics leveraging Big Data technology. We first provide detailed background information on the financial underpinnings of our framework along with the major financial calculations. Afterwards we analyze the performance of different parallel implementations in Apache Spark based on existing computation kernels that apply the ACTUS data and algorithmic standard for financial contract modeling. The major contribution is a detailed discussion of the design trade-offs between applying user-defined functions on existing computation kernels vs. partially re-writing the kernel in SQL and thus taking advantage of the underlying SQL query optimizer. Our performance evaluation demonstrates almost linear scalability for the best design choice. |
| URI: | https://digitalcollection.zhaw.ch/handle/11475/17286 |
| Volltext Version: | Publizierte Version |
| Lizenz (gemäss Verlagsvertrag): | CC BY 4.0: Namensnennung 4.0 International |
| Departement: | School of Engineering |
| Organisationseinheit: | Institut für Informatik (InIT) |
| Publiziert im Rahmen des ZHAW-Projekts: | Large Scale Data-Driven Financial Risk Modelling |
| Enthalten in den Sammlungen: | Publikationen School of Engineering |
Dateien zu dieser Ressource:
| Datei | Beschreibung | Größe | Format | |
|---|---|---|---|---|
| BigDataFinancialAnalyticsUDFvsSQL_JournalBigData.pdf | 2.16 MB | Adobe PDF |  Öffnen/Anzeigen |
Zur Langanzeige
Stockinger, K., Bundi, N. A., Heitz, J., & Breymann, W. (2019). Scalable architecture for big data financial analytics : user-defined functions vs. SQL. Journal of Big Data, 6(46). https://doi.org/10.21256/zhaw-3214
Stockinger, K. et al. (2019) ‘Scalable architecture for big data financial analytics : user-defined functions vs. SQL’, Journal of Big Data, 6(46). Available at: https://doi.org/10.21256/zhaw-3214.
K. Stockinger, N. A. Bundi, J. Heitz, and W. Breymann, “Scalable architecture for big data financial analytics : user-defined functions vs. SQL,” Journal of Big Data, vol. 6, no. 46, 2019, doi: 10.21256/zhaw-3214.
STOCKINGER, Kurt, Nils Andri BUNDI, Jonas HEITZ und Wolfgang BREYMANN, 2019. Scalable architecture for big data financial analytics : user-defined functions vs. SQL. Journal of Big Data. 2019. Bd. 6, Nr. 46. DOI 10.21256/zhaw-3214
Stockinger, Kurt, Nils Andri Bundi, Jonas Heitz, and Wolfgang Breymann. 2019. “Scalable Architecture for Big Data Financial Analytics : User-Defined Functions vs. SQL.” Journal of Big Data 6 (46). https://doi.org/10.21256/zhaw-3214.
Stockinger, Kurt, et al. “Scalable Architecture for Big Data Financial Analytics : User-Defined Functions vs. SQL.” Journal of Big Data, vol. 6, no. 46, 2019, https://doi.org/10.21256/zhaw-3214.
Alle Ressourcen in diesem Repository sind urheberrechtlich geschützt, soweit nicht anderweitig angezeigt.