Compliance and Interoperability with Virtual Platforms

Welcome everyone to the next installment in our blog series looking at Virtualization Based Development and what that looks like in the real world. This blog focuses on how the ASTC Embedded Systems Group (ESG) uses Virtual Platforms to support achieving Compliance and Interoperability for USBPD and USB Type-C software stacks. These software stacks are now deployed in applications such as power chargers, port controllers and active cable firmware.

Commonly the case for embedded software, our USBPD and USB Type-C software stack must conform to an industry standard. Proving conformance of the system requires successful execution of a number of compliance and interoperability tests. Before we can cross that bridge we need a fully operational system to start compliance and interoperability testing. So how do we bring that process into the software development process as early as possible?

Virtual Testing Environment

We start by leveraging VLAB Virtual Platform equivalents of both the hardware, and just as importantly the test environment. VLAB offers a Python interface, which provides an ideal scripting language to describe and build up sets of test cases. Given the structured nature of the USB specifications, Python is perfect for testing low-level blocks. When combined, these tests are ideal for constructing high-level abstract scenarios. Using this approach, we can cover the compliance testing requirements, whether it be connection state machine transitions to complex power negotiation scenarios.

For our own internal development we find this setup provides numerous advantages. The obvious question that inevitably follows… What use are the tests or the virtual environment, once hardware is available to start compliance and interoperability testing? Internal testing doesn’t carry the same weight as commercial industry compliance testing. Should we find that we can’t interoperate with a third party device, or fail a compliance test, what next?

Using virtual when hardware is available?

In our experience, our virtual environment and tests cases still offer an advantage. We start by looking at our own test for the same scenario and try to work out where we differ?

In cases where the comparison shows that our implementation is invalid, updating our testing scenario to reproduce the issue is straight forward, which helps to be able to fix issues ASAP. On the flip side, there have been numerous times where compliance tests report a failure which we disagreed with. In these cases, the virtual environment is an ideal demonstration vehicle to represent our understanding and expectation of the specification. More often than not we have been proven correct.

How does this look in the real world. Let’s just look at a slice of a test. Those familiar with USBPD conformance tests may recognise the test description below. This particular test monitors interaction between the two devices, with each side acting as a “Sink” or “Source” Policy Engine. Our USBPD software in this test is running on the RISC-V Virtual Platform, connected to models of our USBPD and USB Type-C IP. Completing the system is a USB Interface testbench with drivers and monitors to represent a Sink device.

A final word … Testing software on the final target hardware for interoperability and compliance testing makes sense. Virtual Platforms should be seen as complimentary. In our process they are essential to prepare and navigate the process required to get our software ready for customers.