With increasing complexity, chip design is associated with longer development and verification phases. Since time to market is a major factor to win the business, high-quality tests for post-silicon validation should be ready before a silicon device becomes available to conserve time after the device is ready. However, to ensure the verification quality we also need to be sure of the post-silicon tests ‘quality and readiness.
Test coverage plays a critical role in determining the quality and readiness of tests. Coverage results provide valuable input to judge whether tests are good enough to achieve the expected quality on the device or not. However quantifying coverage of post-silicon validation tests is very challenging due to the black box nature of silicon device which limits controllability and observability. It also consumes a lion’s share of overall product development time and cost due to the black box nature of silicon device and dependency on Silicon to develop, evaluate and debug the tests.
One alternative solution for early bring up of post-silicon tests is using RTL simulation or emulation. However, this is practically difficult to realize due to the slow simulation speed, expensive emulation tools cost, integration complexity of RTL design and lack of complete availability of RTL design. Even if the tests are prepared, debugging and fixing the tests takes considerable time due to the low-level abstraction and limited controllability.
An attractive alternative for this is the usage of Virtual prototype (also popularly termed as VP). VP is extensively used in development of software device drivers, application software, and fault injection validation during pre-silicon stage. Virtual prototypes provide the same transaction level functionalities as silicon devices. Transaction level implementation makes it many times faster as compared to RTL emulation or RTL simulation. It also provides higher degree of observability, traceability and controllability. As simulation speed and white box nature of the design are critical factors in development of post silicon validation tests, VP can play a very important role. Therefore VP is the best alternative to estimate the silicon device functional coverage as compared to RTL simulation or emulation and hence validating the readiness of the test suite.
In this article, we discuss various coverage metrics essential for checking the readiness of post silicon tests. We also cover virtual prototype implementation and the limitations of this approach.
PREREQUISITE
It is important to select an appropriate coverage metrics for Virtual Prototype to quantify the completeness of test suite. Coverage results observed by running test suite on virtual prototype needs to reflect the coverage on silicon device as virtual prototype models the same characteristics of the silicon device. Therefore adopting a right combination of conventional coverage metrics along with model specific coverage metric is essential. Typical coverage metrics include structural (code) coverage metrics like functional coverage, decision coverage, statement coverage, block coverage. Model specific coverage can be register coverage, internal states coverage, etc.
i. Function Coverage: It provides precise information about evaluating device features described in specification. Quantification Accuracy of Functional Coverage depends on virtual model implementation. Clear judgement can be made out of the results, if each feature in the specification is mapped to a function in virtual model.
ii. Statement Coverage: This is a simple and fundamental metric for test coverage. This makes sure that every statement in the model code is executed at least once.
iii. Decision Coverage: Decision Coverage, aka branch coverage, is another key metric because it covers multiple execution paths through the code. It obviously requires more test scenarios to achieve various outcomes but provides a more in-depth view of the source code than simple statement coverage.
iv. Register Coverage: To ensure completeness of the tests, test scenarios need to access all the registers in the module. Register Coverage is very critical in ensuring all the registers and its bit fields are implemented as per specification. Virtual model provides complete visibility and traceability support for registers.
Test coverage plays a critical role in determining the quality and readiness of tests. Coverage results provide valuable input to judge whether tests are good enough to achieve the expected quality on the device or not. However quantifying coverage of post-silicon validation tests is very challenging due to the black box nature of silicon device which limits controllability and observability. It also consumes a lion’s share of overall product development time and cost due to the black box nature of silicon device and dependency on Silicon to develop, evaluate and debug the tests.
One alternative solution for early bring up of post-silicon tests is using RTL simulation or emulation. However, this is practically difficult to realize due to the slow simulation speed, expensive emulation tools cost, integration complexity of RTL design and lack of complete availability of RTL design. Even if the tests are prepared, debugging and fixing the tests takes considerable time due to the low-level abstraction and limited controllability.
An attractive alternative for this is the usage of Virtual prototype (also popularly termed as VP). VP is extensively used in development of software device drivers, application software, and fault injection validation during pre-silicon stage. Virtual prototypes provide the same transaction level functionalities as silicon devices. Transaction level implementation makes it many times faster as compared to RTL emulation or RTL simulation. It also provides higher degree of observability, traceability and controllability. As simulation speed and white box nature of the design are critical factors in development of post silicon validation tests, VP can play a very important role. Therefore VP is the best alternative to estimate the silicon device functional coverage as compared to RTL simulation or emulation and hence validating the readiness of the test suite.
In this article, we discuss various coverage metrics essential for checking the readiness of post silicon tests. We also cover virtual prototype implementation and the limitations of this approach.
PREREQUISITE
It is important to select an appropriate coverage metrics for Virtual Prototype to quantify the completeness of test suite. Coverage results observed by running test suite on virtual prototype needs to reflect the coverage on silicon device as virtual prototype models the same characteristics of the silicon device. Therefore adopting a right combination of conventional coverage metrics along with model specific coverage metric is essential. Typical coverage metrics include structural (code) coverage metrics like functional coverage, decision coverage, statement coverage, block coverage. Model specific coverage can be register coverage, internal states coverage, etc.
i. Function Coverage: It provides precise information about evaluating device features described in specification. Quantification Accuracy of Functional Coverage depends on virtual model implementation. Clear judgement can be made out of the results, if each feature in the specification is mapped to a function in virtual model.
ii. Statement Coverage: This is a simple and fundamental metric for test coverage. This makes sure that every statement in the model code is executed at least once.
iii. Decision Coverage: Decision Coverage, aka branch coverage, is another key metric because it covers multiple execution paths through the code. It obviously requires more test scenarios to achieve various outcomes but provides a more in-depth view of the source code than simple statement coverage.
iv. Register Coverage: To ensure completeness of the tests, test scenarios need to access all the registers in the module. Register Coverage is very critical in ensuring all the registers and its bit fields are implemented as per specification. Virtual model provides complete visibility and traceability support for registers.
Figure 1 Typical Implementation of VP
IMPLEMENTATION
Figure above shows a typical VP implementation, it consists of core model, bus model and peripheral models developed using high level languages like C, C++, SystemC etc. These models are integrated in Virtual Platforms Tools like Synopsys Virtualizer, CoMET or ASTCs VLab to create complete Virtual Prototype of Silicon device. So, the same executable (elf/hex/target) code which runs on real silicon can be run on VP as well. Based on the VP environment, a suitable Coverage tool should be selected (there are many free and licensed tool available like OpenCppCoverage, CTC++ and Squish-Coco etc.).
To determine quality and reediness of post silicon tests one should run the tests on VP with coverage enabled (this process might be different for different coverage tools). It will give an estimate of the functions/register (as all registers in VP are modelled in terms of get set function) or branch coverage captured by a given test thus will help to improve the quality of test.
LIMITATIONS
Though Virtual Prototypes provide the same transaction level functionalities as Silicon device, Virtual models may not be as timing accurate as Silicon, so the test designed to validate timing behaviour of IP models might not give the desired output on VP. Therefore Coverage estimation on virtual devices may not always accurately reflect the functional Silicon device Coverage.
We hope this article helps you to get a glimpse of VP’s importance in Post-Silicon validation. Feel free to provide your feedback / opinion in the comment section below.
RSS Feed