While Unit and Integration testing provide quick insights into individual software components, ensuring seamless collaboration among all parts necessitates system-level testing. In the context of preCICE, a coupling library enabling plug-and-play partitioned multi-physics simulations, unique challenges arise. The coupling library stands at the center of a rich coupling ecosystem, with components organized in different repositories, and with complete simulations only possible by combining multiple components from different layers. This additional complexity, together with the heterogeneity of generated results by different simulation software that is coupled together, poses unusual challenges.
The preCICE system tests allow developers from different perspectives (library developer, adapter developer, example cases contributor, release manager) to verify that all components still work together, and that complete simulations still give the same results over time.
A collection of simple Python scripts reads test definitions defined in YAML-based configuration files. Tests describe which combinations of which versions of components to use to start a simulation, extract comparable result files, and compare them. Behind the scenes, the framework starts Docker containers as Docker Compose services, and compares results with fieldcompare.
The tests can be executed by any developer on their local system, or through the GitHub Actions infrastructure of preCICE, on a dedicated organization-wide runner. The workflow can be executed manually, or from different repositories, combining versions that correspond to the perspective of each developer. Examples include combining a feature branch with other released branches, or all development branches of each component. Test reports are generated as build artifacts, archived and easily discoverable next to the GitHub Actions logs.
This poster will present the challenges that testing such a coupling ecosystem brings, and will present a now complete and working solution for system and regression testing in preCICE.
«
While Unit and Integration testing provide quick insights into individual software components, ensuring seamless collaboration among all parts necessitates system-level testing. In the context of preCICE, a coupling library enabling plug-and-play partitioned multi-physics simulations, unique challenges arise. The coupling library stands at the center of a rich coupling ecosystem, with components organized in different repositories, and with complete simulations only possible by combining multipl...
»
Login
BibTeX