In today’s blog we continue to look at how the ASTC Embedded Systems Group (ESG) uses simulation and virtual platforms in our everyday workflow. As is the case for many VLAB customers, the ESG group are working with increasingly powerful and complex hardware. The obvious flow on effect …. simulators need to match this increase in scope and complexity, which creates the following problems:
Over time we have found that different development and testing scenarios can use various simulation abstractions. With this approach we have answered “What to model” and found it easier to achieve our goals with the minimum required effort. Using firmware testing as an example, we use hardware models that include:
Typically, models are written in either Python or C++ depending on the needs of the use case. Simple system models can be completed in a few days with Python. By contrast, a peripheral model developed in C++ will generally be more detailed but take longer.
In some of our testing scenarios it is beneficial to stub the target software, which means that hardware models are unnecessary. Driver functions are intercepted and completed by the simulator and interrupt handlers are directly called by test environment. Similar to host emulation, VLAB’s function stubbing allows us to use the target binary as-is.
This class of models provide limited representations of the hardware, where modelling is restricted to registers and interrupt signals with no functional behaviour. To create these models we use VLAB to synthesise high level descriptions.
The advantage of using these models, other than being quick to create, is that values the firmware reads or writes to a register are available to a test script. In fact the test scripts are required to provide any functional behaviour required by the firmware. Typically we write tests that manipulate register values for the firmware to read and assert interrupts to alert the firmware. Using this test methodology we can define and validate the system interaction for the firmware.
As mentioned, to create such models is generally very quick. Take for example ARM Cortex or RISC-V, which we can generate a complete register stub model from a system descriptions such as CMSIS-SVD or IPXACT. In contrast, coding of test cases and scripts is dependent on the level of behaviour required by the test scenario and tends to be longer.
For those wondering what type of functionality can be included in a test script, we have modelled:
These models are an extension of the Register Stub Models. The same synthesised base class defines all registers and ports. Ports are defined to allow system inputs and outputs. In this case we add some event call-backs to register read/writes and ports. However detailed state, behaviour and timing are only modelled where required.
The choice to create such a ‘Passthrough’ model is usually determined by what the test case(s) require. In our work we have selected such models for the following scenarios:
This methodology typically results in a broader system model. To manage the creation and connection of such system models we use high level python components in VLAB to represent the system.
These are the final evolution of our models which again build on top of simple ‘passthrough’ models. In these models, register call-backs are extended to add state and timing behaviour. Importantly we still avoid modelling details that are not visible to the firmware.
As a demonstration of how we often build upon and reuse models to suit our needs, we have extended the previously mentioned ‘Passthrough’ models from above:
Our experience has been that you CAN test firmware early using a minimal initial investment in modelling. As the project progresses, models are built up to match growing firmware functionality. The most important lesson learnt has been that you only need to model enough of a system to enable testing.