Please use this identifier to cite or link to this item: https://doi.org/10.21256/zhaw-25672
Full metadata record
DC FieldValueLanguage
dc.contributor.authorPanichella, Annibale-
dc.contributor.authorPanichella, Sebastiano-
dc.contributor.authorFraser, Gordon-
dc.contributor.authorSawant, Anand-
dc.contributor.authorHellendoorn, Vincent-
dc.date.accessioned2022-09-17T09:21:50Z-
dc.date.available2022-09-17T09:21:50Z-
dc.date.issued2022-
dc.identifier.issn1382-3256de_CH
dc.identifier.issn1573-7616de_CH
dc.identifier.urihttps://digitalcollection.zhaw.ch/handle/11475/25672-
dc.descriptionErworben im Rahmen der Schweizer Nationallizenzen (http://www.nationallizenzen.ch)de_CH
dc.description.abstractTest smells aim to capture design issues in test code that reduces its maintainability. These have been extensively studied and generally found quite prevalent in both human-written and automatically generated test-cases. However, most evidence of prevalence is based on specific static detection rules. Although those are based on the original, conceptual definitions of the various test smells, recent empirical studies indicate that developers perceive warnings raised by detection tools as overly strict and non-representative of the maintainability and quality of test suites. This leads us to re-assess test smell detection tools’ detection accuracy and investigate the prevalence and detectability of test smells more broadly. Specifically, we construct a hand-annotated dataset spanning hundreds of test suites both written by developers and generated by two test generation tools (EvoSuite and JTExpert) and performed a multistage, cross-validated manual analysis to identify the presence of six types of test smells in these. We then use this manual labeling to benchmark the performance and external validity of two test smell detection tools – one widely used in prior work and one recently introduced with the express goal to match developer perceptions of test smells. Our results primarily show that the current vocabulary of test smells is highly mismatched to real concerns: multiple smells were ubiquitous on developer-written tests but virtually never correlated with semantic or maintainability flaws; machine-generated tests actually often scored better, but in reality, suffered from a host of problems not wellcaptured by current test smells. Current test smell detection strategies poorly characterized the issues in these automatically generated test suites; in particular, the older tool’s detection strategies misclassified over 70% of test smells, both missing real instances (false negatives) and marking many smell-free tests as smelly (false positives). We identify common patterns in these tests that can be used to improve the tools, refine and update the definition of certain test smells, and highlight as of yet uncharacterized issues. Our findings suggest the need for (i) more appropriate metrics to match development practice, (ii) more accurate detection strategies to be evaluated primarily in industrial contexts.de_CH
dc.language.isoende_CH
dc.publisherSpringerde_CH
dc.relation.ispartofEmpirical Software Engineeringde_CH
dc.rightshttp://creativecommons.org/licenses/by/4.0/de_CH
dc.subjectTest generationde_CH
dc.subjectTest smellde_CH
dc.subjectSoftware qualityde_CH
dc.subject.ddc005: Computerprogrammierung, Programme und Datende_CH
dc.titleTest smells 20 years later : detectability, validity, and reliabilityde_CH
dc.typeBeitrag in wissenschaftlicher Zeitschriftde_CH
dcterms.typeTextde_CH
zhaw.departementSchool of Engineeringde_CH
zhaw.organisationalunitInstitut für Informatik (InIT)de_CH
dc.identifier.doi10.1007/s10664-022-10207-5de_CH
dc.identifier.doi10.21256/zhaw-25672-
zhaw.funding.euinfo:eu-repo/grantAgreement/EC/H2020/957254//DevOps for Complex Cyber-physical Systems/COSMOSde_CH
zhaw.issue7de_CH
zhaw.originated.zhawYesde_CH
zhaw.pages.start170de_CH
zhaw.publication.statuspublishedVersionde_CH
zhaw.volume27de_CH
zhaw.publication.reviewPeer review (Publikation)de_CH
zhaw.webfeedSoftware Engineeringde_CH
zhaw.funding.zhawCOSMOS – DevOps for Complex Cyber-physical Systems of Systemsde_CH
zhaw.author.additionalNode_CH
zhaw.display.portraitYesde_CH
zhaw.relation.referenceshttps://zenodo.org/record/3337892#.XswWby-w3yUde_CH
Appears in collections:Publikationen School of Engineering

Files in This Item:
File Description SizeFormat 
2022_Panichella-etal_Test-smells-20-years-later.pdfPublished Version3.27 MBAdobe PDFThumbnail
View/Open
2022_Panichella-etal_Test-smells-20-years-later_EMSE.pdfSubmitted Version420.55 kBAdobe PDFThumbnail
View/Open
Show simple item record
Panichella, A., Panichella, S., Fraser, G., Sawant, A., & Hellendoorn, V. (2022). Test smells 20 years later : detectability, validity, and reliability. Empirical Software Engineering, 27(7), 170. https://doi.org/10.1007/s10664-022-10207-5
Panichella, A. et al. (2022) ‘Test smells 20 years later : detectability, validity, and reliability’, Empirical Software Engineering, 27(7), p. 170. Available at: https://doi.org/10.1007/s10664-022-10207-5.
A. Panichella, S. Panichella, G. Fraser, A. Sawant, and V. Hellendoorn, “Test smells 20 years later : detectability, validity, and reliability,” Empirical Software Engineering, vol. 27, no. 7, p. 170, 2022, doi: 10.1007/s10664-022-10207-5.
PANICHELLA, Annibale, Sebastiano PANICHELLA, Gordon FRASER, Anand SAWANT und Vincent HELLENDOORN, 2022. Test smells 20 years later : detectability, validity, and reliability. Empirical Software Engineering. 2022. Bd. 27, Nr. 7, S. 170. DOI 10.1007/s10664-022-10207-5
Panichella, Annibale, Sebastiano Panichella, Gordon Fraser, Anand Sawant, and Vincent Hellendoorn. 2022. “Test Smells 20 Years Later : Detectability, Validity, and Reliability.” Empirical Software Engineering 27 (7): 170. https://doi.org/10.1007/s10664-022-10207-5.
Panichella, Annibale, et al. “Test Smells 20 Years Later : Detectability, Validity, and Reliability.” Empirical Software Engineering, vol. 27, no. 7, 2022, p. 170, https://doi.org/10.1007/s10664-022-10207-5.


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.