User: Guest  Login
Document type:
Contribution type:
Textbeitrag / Aufsatz
Jordan, Claudius; Foth, Philipp; Fruth, Matthias; Pretschner, Alexander
Why Did the Test Execution Fail? Failure Classification for SystemLevel Testing Using Only Test Step Results
Testing automotive electronic control units and their software, the system-under-test (SUT) in our context, requires complex test infrastructure setups. As those setups are developed in parallel to the SUT, often a problem in the test infrastructure instead of the SUT causes a failed test case execution (TCE). We call such unexpectedly failing TCEs invalid. As there are several reasons that lead to such invalid test failure, failed TCEs are manually analyzed and categorized. This failure classification is a time-consuming task. Thus, automatic classification could significantly reduce overall development time and cost. A previous study suggests using association rule learning (ARL) to classify failed TCEs as valid or invalid based solely on test step information. In this work, we extend this ARL-based approach to our multi-class setting and evaluate its application on data from five running verification & validation projects in the automotive industry. In total, we predict the defect classes of more than 85k TCEs and achieve an overall precision up to 86.7% with an overall recall up to 57.4%. With this work, we offer evidence that the application of said approach, originally presented in the context of information systems, can be fruitful in automotive integration- and system-level testing contexts as well.
association rule mining,automotive,failure classification,hardware-in-the-loop,integra-,system testing,test infrastructure,tion testing
Dewey Decimal Classification:
000 Informatik, Wissen, Systeme
Book / Congress title:
International Symposium on Software Reliability Engineering (ISSRE)
Publisher address:
Charlotte, North Carolina, USA
TUM Institution:
Department of Informatics, Technical University of Munich
Copyright statement: