Author: Michael Lingg
Automated testing of hardware-software integrated systems typically requires detailed technical knowledge of the hardware interfaces, and communication protocols used. A system that includes a Multi-Function Display (MFD), which is controlled via a Controller Area Network (CAN) interface, requires understanding of the CAN messages sent to the MFD, as well as understanding of the CAN protocol to send commands to the MFD and parse responses from the MFD. Similarly, a controller connected to a system that communicates via Universal Asynchronous Receiver-Transmitter (UART) protocol, requires understanding of the UART messages, as well as understanding of the UART protocol.
Many off the shelf tools exist that handle the understanding of the protocol. National instruments provides a large number of modules that can handle various communication protocols, including CAN [2] and UART [3]. Total Phase also provides similar devices[4]. In our previous white paper, Multi-Level Hardware-In-the-Loop Test API for Hardware-Software Integration Testing [1] (MHIL test system), we discussed a testing API designed to abstract away both the hardware interfaces under test, and the messages communicated over these hardware interfaces, on a system that integrates all hardware interfaces with a single central controller.
In this paper we provided an example of a basic temperature alarm hardware-software integrated system, that connects to an MFD via a CAN interface, a speaker, and a thermocouple. The paper further went on to describe how these external devices can be simulated through the MHIL test system, and the devices can be simulated with simple a python script that mimics the device under test:
connect_mfd()
wait(test_delay)
current_state = read_mfd_state()
if current_state is not MAIN_MENU:
report_error()
push_button(alarm_menu_button)
wait(test_delay)
current_state = \
read_mfd_state().get_menu_state()
if current_state is not ALARM_MENU:
report_error()
set_temperature(51)
wait(test_delay)
speaker_signal = read_speaker()
if speaker_signal.frequency \
is not ON_FREQUENCY:
report_error()
if speaker_signal.volume \
is not ON_VOLUME:
report_error()
set_temperature(49)
wait(test_delay)
speaker_signal = read_speaker()
if speaker_signal.frequency \
is not OFF_FREQUENCY:
report_error()
if speaker_signal.volume \
is not OFF_VOLUME:
report_error()
disconnect_mfd()
wait(test_delay)
current_state = \
read_mfd_state().get_menu_state()
if current_state is not FAULT_STATE:
report_error()The MHIL test system abstracts away the hardware protocols, the messages communicated on the hardware protocols, and allows multiple simulated devices to be synchronized together with strict real time accuracy. While there are many benefits to this system, including reducing testing of one system from 6.6 weeks to less than three days, further improvements can be added to the system. While system test developers are expected to have a high level of technical knowledge about how the system under tests functions, automated tests executed by a scripting language will also require the system test developers to be an expert in the scripting language. With this in mind we looked into how to make the MHIL test system more accessible to system test developers that may not be scripting language experts.
The Blockly [5] web application allows for the development of computer programs using a visual drag and drop interface. Blockly provides all of the basic programming functionality, including setting variables:
Performing mathematical operations:
Creation of branches:
Creation of loops:
And creation of (odd) functions:
As powerful as this functionality is, we want to be able to abstract away the code needed to test a system under test. Blockly nicely supports this need as it allows the creation of custom blocks that can be used to create our own functionality.
If we want to press a button on the screen, we create a function to press a button and specify which button is to be pressed:
Producing the following python code:
test_interface.MFD.press_button('alarm_menu_button')
Likewise if we want to check if the speaker is sent the appropriate waveform for the alarm, we use custom blocks to check the frequency and volume being sent to the speaker:
Producing the code to check the analog signal sent to the speaker and read by the ADC:
speaker_signal = test_interface.speaker_adc.frequency()
if speaker_signal ['frequency'] ! = ON_FREQUENCY:
test_interface.report_test_err()
if speaker_signal ['volumen'] ! = ON_VOLUME:
test_interface.report_test_err()With all of the custom Blockly functionality in place, we can create a visual version of the example python test from above. Note many variables used will be global variables set elsewhere, or read from configuration files.
This custom Blockly functionality can be extended to represent any system being tested and can generate test scripts that can be executed in an automated fashion in any language. Now test developers only need to understand how the system works and how it should be tested. They no longer need to know how to produce test scripts to perform automated testing of the system.
Acknowledgements
A big thanks to Daniel Burke for developing the Blockly implementation!
References
[1] http://gvsets.ndia-mich.org/publication.php?documentID=926
[2] https://www.ni.com/en-us/shop/hardware/products/c-series-can-interface-module.html
[3] https://www.ni.com/en-us/shop/hardware/products/c-series-serial-interface-module.html
[4] https://www.totalphase.com/products/komodo-canduo/
[5] https://developers.google.com/blockly
Michael Lingg is a principal research engineer at Array of Engineers, where he leads various research and development initiatives for the medical, aerospace, and space industries. Michael has a P.h.D. in High Performance Computing from Michigan State University and has over 15 years of experience in software design, development, and testing.
Daniel Burke is a software engineer at Array of Engineers. Daniel graduated from the University of Michigan in 2018 with a B.S.E in Computer Science. In addition to developing our custom Blockly application, Daniel has contributed to various software engineering projects in the defense, aerospace, and medical device industries.