Mobile testing
Доклады:
Презентации:
Ресурсы:
Emulate different network conditions that affect application performance
Define network conditions by geographical location of user, server and network type
Emulate different bandwidth, latency and packet loss
Emulate on-the-move scenarios specific to mobile such as entering an elevator or driving through a tunnel
Identify and fix performance issues before deployment
Hello everyone. The theme of my presentation is “Wireless Network Emulation for Testing Mobile Applications”. And first of all I would like to explain why I am presenting it in English.
We communicate with our customers in English and it is really important to be self confident, to have big enough vocabulary to express our thoughts properly and comprehensively. That is why we decided (under “we” I mean QA department and our English Couch - Ira) to start a tradition to give, maybe, not all but many of our internal presentations in English. Because the best way I think to learn English is to speak English as much as possible without any fear that you`ll make a mistake. You`ll see I`ll make a lot of them :-).
So let`s begin.
Challenges of Mobile Application Testing
I think that it is quite obvious for all: technical people and not that applications seem to perform differently under different network environments. You could notice this difference while using any application under your home network conditions and under public network conditions, like restaurants or public transport etc, e.g. accessing a file that is local to you or that`s been stored in a cloud.
And when we use any application especially corporate ones and it works well in different network conditions it is not an accident, it is a testament to their developers and testers who recognized that some of the networks employed in the cloud, and corporate WANs, mobile etc. can be very challenging, especially during busy times.
So in order to make the applications work in the worst of these conditions, such as where available bandwidth is low, data loss and latency is high, they would traditionally use very long test duration to allow sufficient time for the poor conditions to occur. This is because in real world networks the conditions aren’t controllable, if during one test a ‘bad’ network condition is experienced it could take a very long time for that same condition to come back! That’s not acceptable from a time is money point of view.
Different Types of Wireless Networks
Wireless networks are at the epicenter of this trend. At its broadest, a wireless network refers to any network not connected by cables, which is what enables the desired convenience and mobility for the user. Not surprisingly, given the myriad different use cases and applications, we should also expect to see dozens of different wireless technologies to meet the needs, each with its own performance characteristics and each optimized for a specific task and context. Today, we already have over a dozen widespread wireless technologies in use: WiFi, Bluetooth, ZigBee, NFC, WiMAX, LTE, HSPA, EV-DO, earlier 3G standards, satellite services, and more.
As such, given the diversity, it is not wise to make sweeping generalizations about performance of wireless networks. However, the good news is that most wireless technologies operate on common principles, have common trade-offs, and are subject to common performance criteria and constraints. Once we uncover and understand these fundamental principles of wireless performance, most of the other pieces will begin to automatically fall into place.
Everything is Communicating through Network
Your app running on a mobile device on the left, a server on the right, and a cloud in the middle. There are very few stand-alone mobile apps in the world. Everything is communicating either to other users, or a server (or most likely both) over some network.
Wireless Networks Types
Network variability refers to the changing behavior of a wireless network. The variability is caused by the following factors:
Signal loss: Dead spots on a wireless network lead to a partial or total loss of network signal. This leads to abrupt snapping of the client server communication. For example: When a user is transferring funds via a mobile application and there is signal loss / disconnection in the middle of the transaction, even though the fund has been transferred the application might show a “transaction failed” message to the user. This primarily occurs due to the inability of the application to detect the disconnection and handle it properly.
Data loss over network: A data loss occurs when data packets sent by client/server over a wireless network fail to reach the intended destination because of network congestion. Even though resending the lost data is taken care by TCP/IP protocol, the data loss has a severe impact on the user’s experience, when the application doesn’t handle the resulting error conditions properly. For example: Data loss over network can result in an application screen freeze, an abrupt application closure or a temporary disablement of some features.
Bandwidth: The bandwidth can fluctuate significantly when multiple users are sharing the same network. For example: A slow bandwidth would have an impact on a streaming video with problems like buffering.
Network Delay and Jitter: The time taken by each packet from a source to the intended destination is defined as network delay. The network delay experienced by each packet can vary and is known as jitter. For example: A huge delay and jitter on the wireless network can result in a slow data transfer rate. The mobile application can indicate an error quickly or the user interface of the mobile application can become unresponsive.
Impact of Network Variability on Mobile Applications
Unpredictable application behavior: The mobile application might freeze, exit abruptly, behave inconsistently or throw up irrelevant error messages due to the application’s inability to handle network variation.
User interface related errors: Issues like images not displayed or the disorientation of user interface display can be caused by bandwidth variation and signal loss.
Database corruption: Besides the network failure scenarios, we also need to ensure that transactions originating from mobile clients and updating server databases are atomic processes. Database corruption can occur at the server-end due to a network disconnect caused by a signal or packet loss during a transaction, when the client and server databases are not in sync.
The network impact testing solution needs to be able to address all these issues by recreating the corresponding network scenarios. Some of the desired features in a network impact testing solution are:
Network disruption testing: The ability to validate the mobile application functionality under network conditions like connectivity loss.
Network performance testing: Emulate real-life network conditions like bandwidth fluctuation, network type (GPRS / EDGE / 3G) and also validate the end user experience under these conditions.
Network congestion testing: The mobile application validation needs to encompass scenarios of partial data loss and targeted data loss during critical transactions.
Multi-network testing: The capability to validate the mobile application’s behavior and network performance during a switch in the type of network For e.g. 3G to WLAN or 3G to EDGE and
Trace testing Lastly, the network testing solution should be able to test the mobile application with network traces collected from a live network environment.
Trace testing: (A network trace, also called a packet dump, sometimes must be generated to analyze packet traffic between systems to troubleshoot difficult application or networking problems.) Lastly, the network testing solution should be able to test the mobile application with network traces collected from a live network environment.
Last updated
