Login
- Title:
Why Did the Test Execution Fail? Failure Classification for SystemLevel Testing Using Only Test Step Results
- Document type:
- Konferenzbeitrag
- Contribution type:
- Textbeitrag / Aufsatz
- Author(s):
- Jordan, Claudius; Foth, Philipp; Fruth, Matthias; Pretschner, Alexander
- Abstract:
- Testing automotive electronic control units and their software, the system-under-test (SUT) in our context, requires complex test infrastructure setups. As those setups are developed in parallel to the SUT, often a problem in the test infrastructure instead of the SUT causes a failed test case execution (TCE). We call such unexpectedly failing TCEs invalid. As there are several reasons that lead to such invalid test failure, failed TCEs are manually analyzed and categorized. This failure classification is a time-consuming task. Thus, automatic classification could significantly reduce overall development time and cost. A previous study suggests using association rule learning (ARL) to classify failed TCEs as valid or invalid based solely on test step information. In this work, we extend this ARL-based approach to our multi-class setting and evaluate its application on data from five running verification & validation projects in the automotive industry. In total, we predict the defect classes of more than 85k TCEs and achieve an overall precision up to 86.7% with an overall recall up to 57.4%. With this work, we offer evidence that the application of said approach, originally presented in the context of information systems, can be fruitful in automotive integration- and system-level testing contexts as well.
- Keywords:
- association rule mining,automotive,failure classification,hardware-in-the-loop,integra-,system testing,test infrastructure,tion testing
- Dewey Decimal Classification:
- 000 Informatik, Wissen, Systeme
- Book / Congress title:
- International Symposium on Software Reliability Engineering (ISSRE)
- Publisher:
- IEEE
- Publisher address:
- Charlotte, North Carolina, USA
- Year:
- 2022
- Reviewed:
- ja
- Language:
- en
- TUM Institution:
- Department of Informatics, Technical University of Munich
- Copyright statement:
- Copyright and Reprint Permission: Abstracting is permitted with credit to the source. Libraries are permitted to photocopy beyond the limit of U.S. copyright law for private use of patrons those articles in this volume that carry a code at the bottom of the first page, provided the per-copy fee indicated in the code is paid through Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923. For reprint or republication permission, email to IEEE Copyrights Manager at pubs-permissions@ieee.... »
BibTeX