How do you know that your software is working the way that it is supposed to? You test it! The real question, though, is "How do you test it?". In the old days, developers used to just manually test their software. Unfortunately, manual testing is not a great way to test software. The sheer number of test cases, the time required to do the testing, and labor intensity almost guarantee that the software will not be adequately tested.
The solution to improving embedded software testing is to use automated testing. Automated testing can come in many forms, but for now, we will focus on unit testing. Unit testing is "a software testing method by which individual units of source code—sets of one or more program modules together with associated control data, usage procedures, and operating procedures—are tested to determine whether they are fit for use".1 Today's post will explore three tips for developing and running unit tests for embedded software.
Tip #1 – Organize your software into components for testing
Typically embedded software programs will be composed of several dozen modules thrown into a single program folder. Today, it's a little more common to see some folder organization where modules are organized by application, middleware, and drivers. Program organizational structures like this are okay, but it can be much easier to manage the program by components when considering unit tests.
A component is a module that encapsulates a set of related functions2, data, and test cases. For example, a developer that is writing an application component for an FIR filter might organize it as follows:
FIR Filter
- include
-- fir.h
- source
-- fir.c
- tests
-- fir_test.c
Building out a folder structure like this might at first seem a bit painful. However, it keeps all the software modules required by the component to perform its purpose together and the test cases! Furthermore, organizing a component like this makes the component easy to port, or perhaps more importantly, easier to reuse in other software projects.
Tip #2 – Develop Software using Test Driven Development (TDD)
The Agile movement has provided software developers with many processes and tools designed to help them develop quality software faster. One methodology that Agile has produced is Test Driven Development, which is often referred to as TDD. TDD "is a software development process relying on software requirements being converted to test cases before the software is fully developed and tracking all software development by repeatedly testing the software against all test cases3".
I was first introduced to TDD around 2014 when I attended a lecture on the topic by James Grenning at the Embedded Systems Conference. My first impression was that TDD seemed to offer a lot of potential but seemed like it was much more work, and I was not convinced it could deliver on its promises. Unfortunately, it took me several years before I could seriously dig in and integrate it into my development processes. However, once I did, I started to drink the Kool-Aid and see the technique's value.
The full details are beyond the scope of this post, but my favorite references for TDD are Kent Beck's book Test-Driven Development and James Grenning’s Test-Driven Development for Embedded C. In general, TDD changes how developers write their software by focusing on test cases. Developers create a test case, make it fail, and write the code required to pass the test case. By doing this, they are building test cases that they know will catch problems if a bug is introduced into the software.
Tip #3 – Leverage Docker and a unit test harness
The tools available for embedded developers to develop unit tests have evolved considerably over the last few years. When I first started to play with automated testing, I found that getting the tools set up was a huge challenge. This is no longer the case today.
There are several ways that teams can set up their unit testing. First, they can set up their testing as part of a continuous integration and continuous deployment (CI/CD) system. CI/CD allows teams to run their test cases automatically as part of their build and deployment processes. Next, developers can just pick a test harness and install it on their system. The test harness, in this case, is running in a stand-alone environment. Finally, developers can build out their test harness and development processes and set them up within a Docker environment. Docker allows developers to run their development environment in a portable image that minimizes setup time and improves consistency between developers.
A test harness can be set up within Docker and then easily deployed to multiple developers so that they can get their environment set up using just a few commands. It's a compelling process that we will explore in several upcoming blogs.
Conclusions
Creating and using automated testing for embedded software seems daunting at the beginning. However, given how complex today's systems are becoming, it is nearly impossible to perform testing by hand. The only real solution is to develop automated tests that can be used to execute all the system's features. Unit tests are the most common tool available to developers and can dramatically improve system quality while decreasing the overall time spent developing software. In this post, we explored several tips for unit testing embedded software. In future posts, we will explore how to set up and write our test cases.
Jacob Beningo is an embedded software consultant who works with clients in more than a dozen countries to dramatically transform their businesses by improving product quality, cost, and time to market. He has published more than 200 articles on embedded software development techniques, is a sought-after speaker and technical trainer, and holds three degrees, including a Master of Engineering from the University of Michigan. Feel free to contact him at [email protected], at his website www.beningo.com, and sign-up for his monthly Embedded Bytes Newsletter.
1. https://en.wikipedia.org/wiki/Unit_testing
2. https://en.wikipedia.org/wiki/Component-based_software_engineering#Software_component
3. https://en.wikipedia.org/wiki/Test-driven_development