The “Internet of Things (IoT)” is a phrase to describe any network of devices that utilize connectivity to an internet or other devices to exchange data. These devices appear in an incredibly wide range of environments in our world today, such as in your home, in a hospital, within factories, or on a farm. Due to the fact that IoT devices exist in so many different aspects of our lives, each device has to fulfill different requirements to achieve its goal successfully. Utilizing testing to ensure that your piece of the IoT network is able to perform as expected is essential to ensure that the entire network of devices can communicate with each other effectively, and provide a satisfactory experience to the user.
Join our webinar on December 12th at 11 AM (ET) to learn more about our newly developed Wi-Fi IoT test plans and continue reading for a breakdown of each test used to help improve device performance.
What is the UNH-IOL Wi-Fi Internet of Things (IoT) Test plan?
The UNH InterOperability Lab (UNH-IOL) has developed a Wi-Fi IoT test plan to bridge the gap between conformance and performance testing. This testing integrates feedback from UNH-IOL members who have specific and custom testing requests, along with research on common real-world scenarios. While conformance testing determines if a device can meet standards, performance testing determines the best that the device can perform in an optimal environment. Neither conformance nor performance testing evaluate the device under realistic conditions. The Wi-Fi IoT test plan determines how the device operates in an environment similar to the real-world, and therefore, the test cases in this test plan provide insights into how devices will perform for users.
While general Wi-Fi certification testing is essential for any Wi-Fi device, it may not adequately assess a device’s capabilities in particular environments or under specific conditions. The additional testing included in this plan accounts for the device’s environment ensuring it is evaluated under conditions similar and relevant to the environment in its intended use.
The UNH-IOL Wi-Fi IoT test plan includes a range of tests that thoroughly evaluate the device's performance. These tests effectively address critical aspects, including physical conditions, network congestion, and the device's resilience in managing failures and connectivity changes. Let's examine each element involved in testing IoT wireless devices and how this enhances confidence in user satisfaction.
Basic range and throughput testing evaluates the device’s performance under various physical conditions. The Range test will see how the device performs at increasing physical distances, while the Impaired and Variable Signal Throughput tests monitor throughput under less-than-ideal conditions. Additionally, Spatial Consistency testing determines whether the device maintains consistent performance regardless of the device’s orientation relative to the access point or network hub. These tests are essential for establishing baseline measurements for future comparisons and for understanding the device’s capabilities in different physical conditions such as distance and signal direction.
Congested network testing involves scenarios where the Device Under Test (DUT) operates in an oversubscribed network environment.
During the Saturated Network test, we test the DUT’s ability to maintain a stable connection in a network crowded with numerous other devices. We simulate a highly congested network by introducing one new station at a time, and monitor the device’s performance in terms of RSSI, throughput, and other relevant metrics. This test is vital for devices that will be used in environments with significant traffic and multiple wireless hosts, such as hospitals.
With the Initial AP Selection tests, we analyze whether the device selects an access point (AP) with less congestion over an AP with more congestion when both options are available. If the device supports the IEEE 802.11v protocol, we compare its behavior with and without the enhancements provided by 802.11v for initial AP selection. If a device frequently chooses a congested AP instead of a viable and less congested alternative, its overall performance is likely to be negatively affected.
Roaming tests. For mobile devices, such as those carried by a person, ensuring that the device can seamlessly roam between APs while minimizing connection downtime is crucial. This is also important for stationary devices, as we want to ensure that the device can switch to another AP if one goes down. The Basic Roaming and Long Term Roaming tests provide insights on how the device performs in different roaming scenarios. We can assess whether the device can roam between APs using different IEEE 802.11 protocols. Additionally, the Roaming Event Duration test allows us to measure the exact time it takes for the device to complete a roaming event. Visit our blog for more information on the importance of roaming.
Failure and change tolerance testing evaluates how a device behaves in unexpected situations. For instance, these tests observe the device’s response when the AP connected to the device loses power or internet connectivity, or the device loses power briefly. A device’s tolerance to change is also evaluated in situations when a configuration setting such as bandwidth is changed on the AP or the device.
These tests are crucial because in environments with a lot of people, accidents or mistakes can occur. We want to understand how the device will respond in less-than-ideal scenarios. One example, consider a power cable. In a busy environment, it’s possible for the cable to become accidently unplugged if it gets caught on something. Testing can determine how the device reacts in this scenario and identify any necessary changes if the observed behavior is not desirable.
Stability tests ensure that a device can not only provide good throughput and maintain a connection initially, but also sustain that connection and stable throughput over an extended period of time.
- A Long Term Stability test keeps all factors consistent to establish a baseline run.
- Following this, a Variable Signal Stability test can be done to verify that the device can handle fluctuating connection strength while remaining stable.
- Lastly, the Impaired Signal Stability test measures the device’s ability to remain stable in a less-than-ideal network environment.
Power usage and battery life tests. Understanding the power consumption of your device is crucial, especially in large-scale systems that rely on multiple devices. The enhancements in the IEEE 802.11v protocol can lead to extended battery life when in Power Save modes. This is especially useful for battery-powered devices that require regular charging. For example, in a hospital setting, it’s essential for a patient’s mobile monitoring device to remain operational at all times. Therefore, knowing how frequently the device needs to be charged is vital.
The UNH-IOL Wi-Fi Internet of Things test plan covers all the scenarios mentioned above, as well as additional scenarios. It’s essential to ensure your device is ready to perform well for users. To achieve this, you must understand how your device operates in a wide range of situations. Testing these edge cases early in the development process can save time and money in the long run. These tests are essential for identifying behaviors that other testing fails to assess. This approach is necessary to ensure that the device offers a high-quality experience for the end user.
- ‹ Previous
- Next ›