Tips for testing a mobile application - Mobile testing: features

Lecture

Это окончание невероятной информации про тестирование мобильных приложений.

...

tools for checking mobile application compatibility: BrowserStack, CrossBrowserTesting by Smart Bear, Litmus, Browsera, Rational Clearcase by IBM, Ghostlab.

Performance testing

Performance testing – is a set of testing types aimed at determining the performance, stability, resource consumption and other quality attributes of an application under various usage scenarios and loads.

The main goals of performance testing:

Checking the application's response time to different types of requests, to make sure the application works according to the requirements for a normal user load. (Load testing).
Testing the application's performance under loads exceeding the number of users several times over. (Stress testing).
Study the application's performance for long-term operation under normal load. (Stability).
Check operation under conditions of an «expanded» database during normal times. (Volume testing).
Determine the number of users who can work with the application simultaneously. (Concurrency testing).

Tips for testing the performance of your mobile application:

Determine whether the application works under different network conditions.
Find the areas of the application that reduce application performance.
Assess the application's ability to cope with certain load volumes.
Make sure the application's response time meets the requirements.
Check the application's stability under “heavy” user load conditions.
Ensure the application's performance if it operates under conditions of an unstable internet connection.

Useful tools for testing the performance of mobile applications: NeoLoad by Neotys, Apteligent (formerly Crittercism), New Relic.

Security testing

Security testing is intended to verify the security of the system, as well as to analyze risks related to ensuring a holistic approach to protecting applications from hackers, viruses, and unauthorized access to confidential data:

Make sure the application's user data (logins, passwords, bank card numbers) is protected from network attacks by automated systems.
Make sure the application's security system requires a strong password and does not allow an attacker to obtain other users' passwords.
Make sure the application does not grant access to confidential content or functions without proper authentication.
Protect the application and the network from DoS Attacks.
Protect the application from malicious attacks on clients.
Prevent possible malicious consequences of file caching.
Study user files and prevent their possible negative consequences.
Analysis of system file interactions, identifying and fixing vulnerabilities.

Useful tools for checking the security of mobile applications: the CS community, OWASP Zed Attack Proxy, Veracode, Google Nogotofail and SQL Map.

Recovery testing

Recovery testing checks the application under test in terms of its ability to withstand and successfully recover from possible failures caused by software errors, hardware failures, or communication problems:

Check the application's effective recovery after unforeseen emergency situations.
Ensure the data recovery process after a connection interruption.
Check recovery after a system crash and a transaction failure.
Check the application's ability to handle transactions in case of a power failure (low battery, improper application closure, etc.).

Localization testing

Localization testing allows you to test the adaptation of mobile applications to a specific target audience according to its cultural characteristics:

Determine the languages supported by the application.
Ensure the correctness of the translation.
Check the correctness of the translation in accordance with the application's subject matter.
Check date formats.
Check number separators.

Ubertesters, eggPlant can be useful.

Change testing

So, you've gone through all the mentioned stages and found bugs. Therefore some changes were made to your application's code.

Make sure your team has successfully fixed all the identified bugs (re-testing or confirmation testing). Simply put, the test cases that originally found the bugs are run again. And this time they must pass without errors.

Make sure the new changes have not introduced new bugs. (Regression testing).

Some useful tools for testing related to changing your application: Appium, Robotium, Ranorex.

11. Beta testing

Finally, you have a preliminary, fully functional version of your mobile application. It would be better to assess the capabilities and stability of the program from the point of view of its future users.

Beta testing – is the stage of debugging and verifying a beta version of the program. Its main goal – is to identify the maximum number of bugs in its operation for their subsequent elimination before the final version of the application is released to the market.

People who have experience working with applications of a similar type, or even better, with a previous version of the application, are chosen for the role of beta testers.

Before beta testing your mobile application you should pay attention to the following factors:

The number of testing participants.
The duration of testing.
Demographic coverage.
The cost of testing.

Even spending a little money on beta testing can be a good investment in the quality of your mobile application.

Some popular platforms for beta testing mobile applications: HockeyApp, Ubertesters, TestFlight.

Certification testing

There are certain rules for organizing the installation file (.apk) and application design rules for each app store. Certification testing confirms that the application meets the requirements of the most popular stores, such as Google Play, App Store, and Windows Phone.

Let's look at the main criteria for applications' compliance with standards, license agreements and terms of use.

Android:

The application's installation file (.apk) complies with the program's policies.
The application meets the UIG requirements.
The application contains no viruses. The Android market semi-automatically checks the application for viruses and can block your account if it finds any.
You must follow the version control procedure when publishing an updated version of your application.

IOS:

The application meets the requirements of the User Interface Guidelines.
The application must have a unique name.
You need to provide a link for feedback to the developer.
The application must be placed in a specific, defined category.
App Store checks the application for compatibility.
The application does not contain prohibited materials, unforeseen delays in operation, or repetition of existing functions.

Windows Phone:

The application meets the application certification requirements.
A clear description of hardware and network requirements.
The features mentioned in the description or shown in the screenshots are fully implemented

Tips for testing a mobile application

Let's systematize our knowledge and try to identify the main tips for testing mobile applications.

Study the application you are going to test.
Remember the differences between desktop and mobile applications.
Take into account the peculiarities of the operating system and hardware.
Use real devices whenever possible.
Don't try to find a «Swiss army knife» for testing. Use tools you are familiar with.
Take advantage of cloud mobile testing.
Confirm your results with screenshots, logs and videos.
Use the developer menu options for iOS and Android.
Don't neglect (but don't overuse either) emulators and simulators for testing.
Check the performance of your application.
Don't automate everything.
Ask real users to test your application.
Take the human factor into account.

Testing a mobile application with EasyQA

Practice is the best way to learn something. You have the opportunity to use Easy QA's features to test your mobile application.

EasyQA SDK – is a set of features developed by our team to make the software testing process more effective and simple.

Here are its key features:

Crash reports.
In-app bug reporting.
Screenshot creation.
Video recording.
It works with both Android and iOS applications.

I hope you've learned something new and useful about testing mobile applications. Perhaps you can add some other checks for testing, or you may have an idea of how to test a mobile application with greater efficiency. Share your ideas and thoughts with us.

Mobile application testing tools

Choosing devices for testing

Undoubtedly, a real device – is the best solution if you want to test a mobile application. Testing on a real device always gives you the maximum accuracy of results.

In fact, it's really not easy to choose the most suitable device. In any case, here are some actions you should take when choosing a device for mobile testing:

Analyze and identify the most popular and widely used gadgets on the market.
Choose devices with different OSs.
Choose devices with different screen resolutions.
Pay attention to the following factors: compatibility, memory capacity, connectivity, etc.

Advantages of testing mobile applications on real devices:

High accuracy of the test result.
Easy bug replication.
Things like battery capacity, geolocation, push notifications, and built-in device sensors are easy to test.
Ability to check incoming interruptions (calls, SMS).
Ability to test the mobile application under real-world conditions.
No false positives.
There are also disadvantages:

There is a huge number of commonly used devices.
Additional costs for device maintenance.
Limited access to devices commonly used in foreign countries.
As you can see, testing on real devices is a good solution, but it also has some limitations.

Emulators or simulators?

It's easy to guess that there are special tools that emulate / simulate the functionality and behavior of mobile devices.

The meanings of the words «Emulator» and «Simulator» are often confused. Although they are pronounced almost the same, they are not the same.

In fact, an emulator – is an original replacement for the device. However, you don't have the ability to modify programs and applications, although you can run them. A simulator doesn't copy the device's hardware, but you have the ability to set up an environment similar to the original device's OS.

Therefore, it's better to use mobile simulators to test a mobile application. Emulators are more suitable for testing mobile websites.

Advantages of using simulators to test a mobile application:

Simple setup.
High speed.
Helps check and test the behavior of your mobile application.
Cost-effective.
Disadvantages of using simulators to test a mobile application:

Hardware is not taken into account.
False positives are possible.
Getting incomplete data on simulation results, which creates certain difficulties for a full analysis of test results.
Cloud testing of a mobile application

Testing mobile applications using cloud tools appears to be the optimal choice. This can help you overcome the disadvantages of real devices and simulators.

The main advantages of this approach:

Easy availability.
The ability to run mobile devices on multiple systems.
The ability not only to test, but also to update and manage applications in the cloud.
Cost-effective.
High scalability.
The same script can be run on multiple devices in parallel.
Disadvantages of cloud mobile testing:

Less control.
Not as high a level of security.
Dependence on the internet connection.
Some useful cloud tools that can help you test a mobile application: Xamarin Test Cloud, Perfecto Mobile Continuous Quality Lab, Keynote Mobile Testing.
Manual and automated mobile testing

Currently, many specialists support the opinion that manual testing will eventually stop being used. Of course, this is not true. We cannot do without test automation, but there are situations where manual testing is preferable.

Advantages of manual testing of mobile applications:

It's more cost-effective in the short term.
Manual testing is more flexible.
Better simulation of user actions.
Disadvantages of manual testing of mobile applications:

Manual test cases are difficult to reuse.
Performing a certain repetitive task is less efficient.
The testing process is slow.
Some types of test cases cannot be performed manually (load testing).
Advantages of automated application testing:

The testing process takes little time.
Cost-effectiveness in the long-term perspective of use.
Automated test cases are easy to reuse.
The only solution for some types of testing (performance testing).
Test results are easily accessible.
Disadvantages of automated application testing:

Some mobile testing tools have limitations.
The testing process takes a lot of time.
Automated testing is least effective at determining usability.

Of course, there are no unambiguous answers as to which strategy is best to choose. However, a combination of different options is most optimal. For example, you can use simulators at the earliest stages of your testing. But it is better to use real devices (physical or cloud-based) at the final stages. Automated testing is preferable for load and regression testing.

iOS testing tools

First of all, you need to familiarize yourself with the official developer library

1) https://developer.apple.com/develop/

2) Beta testing tool

https://developer.apple.com/testflight/