Please use this identifier to cite or link to this item:
Publication type: Conference paper
Type of review: Peer review (publication)
Title: SBST tool competition 2021
Authors: Panichella, Sebastiano
Gambi, Alessio
Zampetti, Fiorella
Riccio, Vincenzo
et. al: No
DOI: 10.1109/SBST52555.2021.00011
Proceedings: 2021 IEEE/ACM 14th International Workshop on Search-Based Software Testing (SBST)
Page(s): 20
Pages to: 27
Conference details: 14th International Workshop on Search-Based Software Testing (SBST), Madrid, Spain, 31 May 2021
Issue Date: 2021
Publisher / Ed. Institution: IEEE
ISBN: 978-1-6654-4571-9
Language: English
Subject (DDC): 005: Computer programming, programs and data
Abstract: We report on the organization, challenges, and results of the ninth edition of the Java Unit Testing Competition as well as the first edition of the Cyber-Physical Systems Testing Tool Competition. Java Unit Testing Competition. This year, five tools, Randoop, UtBot, Kex, Evosuite, and EvosuiteDSE, were executed on a benchmark with (i) new classes under test, selected from three open-source software projects, and (ii) the set of classes from three projects considered in the eighth edition. We relied on an improved Docker infrastructure to execute the tools and the subsequent coverage and mutation analysis. Given the high number of participants, we considered only two time budgets for test case generation: thirty seconds and two minutes. Cyber-Physical Systems Testing Tool Competition. Five tools, Deeper, Frenetic, GABExplore, GABExploit, and Swat, competed on testing self-driving car software by generating simulationbased tests using our new testing infrastructure. We considered two experimental settings to study test generators’ transitory and asymptotic behaviors and evaluated the tools’ test generation effectiveness and the exposed failures’ diversity. This paper describes our methodology, the statistical analysis of the results together with the contestant tools, and the challenges faced while running the competition experiments.
Further description: ​© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Fulltext version: Accepted version
License (according to publishing contract): Licence according to publishing contract
Departement: School of Engineering
Organisational Unit: Institute of Applied Information Technology (InIT)
Published as part of the ZHAW project: COSMOS – DevOps for Complex Cyber-physical Systems of Systems
Appears in collections:Publikationen School of Engineering

Files in This Item:
File Description SizeFormat 
2021_Panichella-etal_JUnit-tool-competition_SBST.pdfAccepted Version384.89 kBAdobe PDFThumbnail

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.