End-to-End Concolic Testing for Hardware/Software Co-Validation

Many recent approaches have been proposed to improve the quality of Systems-on-Chips (SoC), mainly focusing on a specific part of the SoC, e.g., device driver, hardware, firmware, etc. System-level validation of the entire SoC stack remains a major challenge, and so far research on end-to-end validation of SoC that covers both hardware and software (HW/SW) components is comparatively sparse. In this paper, we propose an approach to end-to-end concolic testing for HW/SW co-validation of SoC. Based on the simulation of SoC with multiple virtual platforms, we capture a set of run-time traces from different components of the entire SoC, and assemble them into holistic system-level traces. We also provide instrumentation interfaces over the SoC trace for custom validation and analysis, allowing insertions of user-defined assertions and symbolic values at various HW/SW interfaces. The instrumented trace is replayed in a concolic/symbolic engine to generate new system-level test cases that either explore new paths of the SoC stack or trigger assertions. We emulated a complete SoC stack based on several open-source projects, from which we demonstrated that our approach can generate effective system-level test cases which crosscut the entire HW/SW stack of SoC and pinpoint an IP firmware buggy path from the user inputs to the host SW, and can catch various bugs with user-defined assertions including two bugs of QEMU's E1000 Virtual Device.