Smart Lock Testing
My team’s task was to test a smart lock—an iOS application that controls the lock and the lock itself. We had to check whether the application and the lock worked in tandem. We set up the environment. The environment, in this case, consisted of a door, a lock, and for these two to communicate with each other, an IoT hub. Any iOS device could have served as a hub, but we went for an Apple TV because it is the preferred device for most people. The next step—the creation of test cases—was a wee bit more challenging. We had not tested a similar application before and could not fall back on experience. Moreover, we had to consider several factors to reach the acceptance criteria. We started preparing the test cases and soon a typology emerged. More about it below.Test Case Categories
The test cases we created could be grouped into the following categories:- Integration: This is the most important type because home automation is achieved by integrating multiple components. We test the communication between the different units in the system to verify whether they communicate with each other without a flaw.
- Network/Coverage: These test cases are intended to verify the range of communication to ensure that we get the expected output regardless of whether the devices are in or out of the coverage area. In both cases, the system should respond to users in a desirable manner. Also, we test whether the system performs well at different connection speeds.
- Real-Life Scenarios: We have to think a bit out of the box and conjure up scenarios that can arise in real life. For example, will the key work when your wife tries to open the door without you? Can your housekeeper enter your house outside the access time you have allowed her?
- Calibration: Calibration enables a lock’s inside-outside sensor and sets the activation range. In our context, we verify whether the lock is able to distinguish the position of the phone (key) and lock/unlock accordingly.
Challenges for IoT Testers
There is no one-size-fits-all solution for testing IoT products, and there are few dedicated methods or processes. IoT testing can be nightmarish for several reasons:- Multiple Vendors: Devices/services from multiple vendors will be communicating with each other. Error in one service or device may hurt the entire system.
- Complex Test Data: People use systems in different ways. So, preparing test data/test cases could be tough and the final test data could be highly complex.
- Capacity of Devices: Different devices in a connected system will be of different capacities. When they are integrated, they may behave differently with different response times. The gadget for testing could be of inadequate capacity or it may not be available at the right time.
- Compatibility Issues: There could be compatibility issues as devices are made by different manufacturers with different firmware and hardware.
- Security/Privacy Issues: Security is handled differently in different gadgets. Thus the risk of a security breach on a connected system is high.
IoT Testing Best Practices
Given the complexity, there are a few things we as IoT testers need to do to stay on track:- Make sure the requirements are clear to all, including yourself.
- Define a good testing approach and plan ahead.
- Do proper unit as well as integration testing before you test the entire system.
- Constantly update yourself. Learn more about the architecture, operating system, hardware, and application protocols to understand how the components communicate with each other.
