Mobile App Testing Checklist

[mobile-app-tesing-blog]

A few approaches for performing functional testing of mobile applications.

The unpredictability of the fragmented mobile market calls for an optimal test coverage for mobile applications. Functional Testing of Mobile Applications tests its basic user interactions such as launching, logging in, checking account updates and other direct user flows. Implementing automated and unattended functional testing helps a great deal to manage the aggressive timelines for releasing the mobile applications.

In order to be at the top of the list of trending mobile apps, a mobile application needs to deliver high performance under all different sets of usage conditions. Some of the best practices followed while testing can help in making a mobile application truly versatile.

- Devising a mobile application testing strategy: A comprehensive mobile application functional testing strategy needs to consider the following key points:

-Target Device Selection: An optimal mix of testing on real devices and simulators can help in maximizing the test coverage.

-Test Automation: Selecting an effective test automation tool can help in maximizing the test automation.

-Network Environments: All possible network conditions need to be considered by using the network simulation tools to simulate various network conditions.

- Shift Left: Shifting left does not necessarily mean implementation of sophisticated testing tools. It literally means testing as early as possible. Mostly testing starts once the development has reached a logical stage. Testing early and often can help to identify and resolve issues early, thus minimizing the risk of extensive debugging later on.

- Multiple testing of application usability: Mobile applications need to meet all the guidelines of usability. Users generally have very high expectations about the usability of the apps. Hence, it is important to ensure that all the UI elements of an app are easy to use and understand. Also the app should be able to provide very clear interruptions and error messages.

- Multi-lingual support: If the mobile application is a global one, then it needs to support different languages. A mobile application needs to be tested against the various languages, currencies as well as the prominent social network of that region.

- Power requirements of the application: Mobile application which consume or need too much power while performing are likely to get deleted in the long run unless it is as useful and handy as the GPS. Hence, the battery power requirement, suspended connectivity and network strength, all play a significant role in determining the functional efficiency of  mobile application.

- Handling interruptions and stand-by modes: Mobile apps need to be tested for any UI glitches, data handling and its ability to maintain state while resuming or leaving the stand-by mode. With multiple apps installed in a mobile, notifications are very common to occur. These notifications can have an impact on the application currently running on the mobile phone. The functional logic of the application should be well tested when faced with interruptions.

- Testing against the major hardware features: Mobile applications need to work around all the powerful hardware and sensors. A mobile application needs to all combinations of sensor specific scenarios. It is also important to test the mobile applications against all the different hardware features of mobile devices.

Conclusion

It is not a matter of choosing one approach over the other. It is the right combination of the approaches that need to be taken. Maximizing functional test automation can prove to be an effective way of long term debugging actions. Combining testing solutions with the best practices listed above helps to address the challenges associated with functional testing of mobile applications.

[mobile-app-tesing-blog]

Mobile Test Automation – Trends and Tools

Due to the excellent usability of mobile applications by millions of mobile end users, the mobile application landscape is growing across all business verticals. To gain more and more mobile users and convert them into customer base, most of the organizations have started creating their own app store market as per their customer preferences. This, however, poses challenges on the ability to support various types of mobile devices and operating systems like iOS, Android and Windows. Organizations therefore have to do such testing more often due to multiple OS upgrades, launch of new device models and releases of new mobile features. Such situations demand the need of mobile test automation, the relevant tools and understanding the current trends.

-      Mobile Automation is not going to remain isolated. It will eventually integrate with multiple business channels and technical processes. Companies need to look for web, mobile and desktop application can be combined into a single strategy with the best reuse of knowledge, tools and innovations.

-      Mobile Automation is not limited to the activities of QA alone. It can also be used by development teams for unit testing to find and prevent defects at source. Companies need to assess how automation tools can integrate with the development technology landscape in order to improve the overall development quality of the application.

-      Companies need to ensure how automation can be enabled from anywhere, so that the entire development tools and QA teams have access to the tools and devices.

-      Companies need to watch out for any interdependency between development and QA. Such code-intrusive tools need to be avoided along with such interdependencies.

-      It is an added advantage if the mobile automation tool can also address the performance testing under the same investment.

Mobile Automation Testing Current Trends:

Every technical advancement brings new challenges in the world of mobile application testing. Recent trends in this field can be listed as follows:

-      Tests that can run on a various language versions of the same application.

-      An end-to-end mobile testing from the application build process to the test execution results – a process to achieve continuous integration.

-      Ability to test the most recent device that has been launched in the market.

-      Full test coverage by complementing test automation with exploratory manual testing.

Mobile Automation Testing Tools and how to evaluate them: More often testers are concerned on which mobile test automation tool is the best for testing the application. There are several must haves for a mobile test automation tool. All the tools will not satisfy all criteria. Hence, it is a project specific requirement as to concentrate on which criteria more and which can be taken lightly. The following table will help to evaluate an automation tool based on the criteria it meets.

- Identification Criteria : OCR (Text Recognition) : Tools :Appium, Selenium, MonkeyTalk

- Identification Criteria :Ability to test devices which has been bought out from retail stores and test it as-is:Tools :MonkeyTalk, Appium, UI Automation, Robotium, Selenium.

- Identification Criteria : iOS(iPhone, iPad)Android:Tools :UI Automation, Appium, Robotium, Monkey Talk, Selenium

- Identification Criteria : Emulators,iOS (iPhone, iPad) , Android (inc. tablets) :Tools :UI Automation, Appium, Robotium, Monkey Talk, Selenium, See Test

- Identification Criteria : Plugs-in to existing test environment:Tools :Appium, Robotium, Selenium, See Test

- Identification Criteria : Recorder:Tools :Selenium, Robotium(Using TestDroid app), Appium, UIAutomation, MonkeyTalk, See Test

- Identification Criteria : Same OS - different devices:Tools :Appium, UIAutomation, MonkeyTalk, Robotium, Selenium, See Test

- Identification Criteria : Different OS – different devices:Tools :See Test, MonkeyTalk

 - Identification Criteria : Serial Test Execution:Tools :Appium, Robotium, UIAutomation, Selenium, MonkeyTalk, See Test

- Identification Criteria : High Speed Test Execution:Tools :Selenium, UIAutomation, See Test

- Identification Criteria : Multi-touch:Tools :Appium, UIAutomation, Robotium, MonkeyTalk, See Test, Selenium

- Identification Criteria : Swipe:Tools :Appium, MonkeyTalk, Robotium, MonkeyTalk, See Test, Selenium

- Identification Criteria : Drag and drop:Tools :Appium, UIAutomation, Robotium, MonkeyTalk, See Test, Selenium

- Identification Criteria : Zoom:Tools :Appium, UIAutomation, Robotium, MonkeyTalk, See Test, Selenium

Identification Criteria : Scrolling:Tools :Appium, MonkeyTalk, Robotium, MonkeyTalk, See Test, Selenium

- Identification Criteria : Click on Security alerts:Tools :Appium, MonkeyTalk, Robotium, MonkeyTalk, See Test, Selenium

- Identification Criteria : Reboot device:Tools :Appium, Robotium, See Test, MonkeyTalk, Selenium, UIAutomation

- Identification Criteria : Awake device from sleep mode:Tools :Appium, UIAutomation, See Test, Robotium, MonkeyTalk

- Identification Criteria : Operate device Back, Call/Hang Call, Home, Menu buttons:Tools :Appium, UIAutomation, Robotium, MonkeyTalk, Selenium

- Identification Criteria : Device Connectivity - USB:Tools :Appium, UIAutomation, Robotium, MonkeyTalk, See Test, Selenium

- Identification Criteria : WiFi:Tools :MonkeyTalk, See Test

- Identification Criteria : Cloud (On-Site Private Cloud): Tools :See Test

 Conclusion

Effective mobile automation testing can be achieved by following the trends and the best tools fit for the application under test. Several factors like availability and compatibility of tools, environment and network setup, UI, optimal system configurations, etc., are all considered effective for an efficient test automation of mobile applications.

[mobile-app-tesing-blog]

The Challenges of Mobile Load Testing.

It is an obvious fact that mobile is here to stay. Smart devices are now the major media of interaction for businesses and consumers globally. The growth of mobile devices have made organizations integrate them with the computing environment.

Thus, mobile devices, integrated with mobile applications, now deliver complex functionality on platforms with limited resources for computing. This requires a performance assessment of the applications to ensure that an application will perform optimally under load with an extreme number of users.

Let us review the main challenges in Mobile Load Testing:

-   User Simulation

During a typical functional test of mobile applications, user actions such as tapping, swiping, zooming and text entering are emulated on the device emulator. The objective of load testing is not to evaluate the interaction experience of one user rather it is performed to see the handling requests from a large number of users and evaluate the response times.

An effective load testing needs to simulate a huge number of simultaneous users via the application under test. Here, it is impractical to to use real devices and emulators as it requires acquiring, configuring and synchronizing thousands of machines running emulators.

-   Network Simulation

Mobile Device access the servers over networks that are usually slower than those used by desktop computers. Network conditions put a significant effect  on the user experience and this effect may be bigger or smaller depending upon the application. An application while being tested in the QA department of a company will access the first class server operating user an ideal phone network. It is tested with the assumption that the network conditions will be the same for all users. The real challenge occurs when a user in a rural area has a hard time executing the functions than in a modern suburbs. This makes network virtualization a top priority.

-   Simulating browsers and browser capabilities:

When a web browser requests for a resource from a web server, it identifies itself through the user-agent header sent with each request. This header contains the information about the browser and the platform on which it is running. For mobile applications, based on the information in the user-agent header, content is delivered accordingly with less text and smaller images for smaller screens. This eventually leads to bandwidth consumption and variable load times depending on the browser and the platform being used. While load testing it is important for the testing tool to be able to manipulate the user-agent header.

-   Identifying the most appropriate settings for realistic tests

Finding the most appropriate values for test settings can be a challenge. These key test settings can be the user-agent, bandwidth and the number of simultaneous connections. Advanced load testing tools can help to determine these values. The process is eased more when the tools can calculate the most appropriate settings for upload and download bandwidth based on the technology used.

-   Testing Via Cloud

For consumer facing apps, it is difficult to predict the number of users that the application may have to handle. Traffic spikes pertaining to special days such as offers, promotions, an unexpected social network buzz etc., can be substantial. Generating such loads in-house needs access to hardware resources. Cloud here provides an opportunity to increase the scale of testing, provide resources, most economically. However, it is to be noted that only the move to cloud is not the solution, rather it depends on the testing tool selected and how efficiently it uses cloud technology for its purpose.

-   Analyzing Results:

The default results of mobile load testing are generally quantified in averages. Averages do not provide an accurate picture of the application performance. Hence, to gain meaningful insights and validate the SLA’s and performance requirements for each condition, it is important to go beyond the default averages and analyze the results for every kind of user.

Conclusion

In many ways, Mobile Load Testing can be similar to web application testing. However, there are  some very specific parameters that need to be addressed for mobile load testing. Addressing the challenges listed above is essential in order to ensure that mobile web applications are tested sufficiently prior to release and that they are capable of performing well under load in the production environment.

[mobile-app-tesing-blog]

5 Open Source Test Automation tools for Mobile Applications.

As mobile applications are now becoming more and more critical to businesses, testing professionals are compelled to adopt new tools and frameworks for effective mobile application testing. Evaluating and ensuring the quality of a mobile application is a complex process and depends on the time and resources available.

There are multiple mobile application test automation tools available online which help in building cases for mobile application testing. One needs to make a choice between open source tools and premium ones depending on requirement and technical skills available.

Let us review some of the top open source test automation tools available for mobile automation testing.

-      Appium: It is a test automation tool for automating native, hybrid and mobile web applications on Android and iOS platforms. It allows writing tests across multiple platforms using the same API. It helps to automate any mobile app in any language with complete access to back-end APIs and DBs from test code.

Requirements

Android:

Mac OSX 10.7+ or Windows 7+ or Linux Android SDK ≥ 16 (SDK < 16 in Selendroid mode)

iOS:

Mac OSX 10.7+ XCode 4.5+ w/ Command Line Tools

-      Selendroid for Android: It is a test automation framework for native, hybrid and mobile web. As it reuses the existing Selenium infrastructure, tests can be written using Selenium 2 client APIs. Selendroid can be used both on emulators and real devices. The four main components of Selendroid are – Web Driver Client, Selendroid Server, Android Driver App, Selendroid Standalone.

Selendroid is able to work on Windows, Linux and Mac OS.

-      MonkeyTalk for both Android and iOS: It is a simple tool which helps to automate real, functionally interactive tests for Native, Hybrid, Web/HTML5 and Flex apps. It mainly consists of two basic components:

- MonkeyTalk IDE: It is a tool based on Eclipse that records, plays, edits and maintains the functional test suits for Android and iOS applications running on devices, emulators and simulators.

- MonkeyTalk Agent: These are the libraries for Android and iOS that need to be linked into applications to be tested. These are responsible for enabling applications to record and play the MonkeyTalk Commands.

-      Calabash: It is a cross platform test automation framework that supports native apps in iOS and Android. It consists of libraries that enables test scripts to interact with native and hybrid apps programmatically. Calabash also supports cucumber, which helps to express the behavior of an app using non-technical language that can be understood by the QA staff and business experts. Calabash can be used to write automated acceptance and functional tests and helps in test automation from the UI level down.

-      Robotium: It is an android test automation framework which can be used to test native and hybrid android applications. It helps to write system, functional and user acceptance test cases which can be executed in an emulator as well as real devices. It can also be easily run through continuous integration tools.

Conclusion

Every testing tool has its own pros and cons each of which need to be evaluated relative to the requirements and type of product. Proper planning, research and practice are required to perform

mobile application test automation

. Hence, it will be a safe bet to hire a professional Android and iOS application developer who has the skill to use open source tools for mobile application testing.

[mobile-app-tesing-blog]

Mobile Application Testing Strategies.

Mobility is now a way of life. The proliferation of mobile device across platforms and operating systems have rendered it more challenging for developers to develop and release new products. The goal of testing is not finding errors, instead, it is to ensure the quality of the software being provided. Mobile Applications have become increasingly complicated. This has increased the requirement for end-to-end testing.

There exist unique challenges to mobile application testing. End-Users are completely unforgiving when it comes to faults in mobile applications. A fault or failure in the app performance instantaneously loses customers. A bad experience, delayed response time, etc., causes a section of the users to completely delete the app, with a majority of them skeptical to even try other apps from that vendor.

Hence, it is very important for getting the app function right for increasing and retaining customers. Some of the following tactics and strategies for mobile app testing can be considered for ensuring a successful mobile app testing.

-      Implementation of Agile Development Model: The fundamental of an Agile software development model lie in involving all personnel involved in the project, right from the customer, developer, tester to the analyst. It enables continuous integration in which the process of development and testing are coordinated through shared code repositories. This has proved to improve the speed, quality and time to market of the app, but it still remains unused by most development organizations.

-      Consider Mobile Automation Testing : Automating complex, repetitive tests has a significantly positive effect  on testing time scales. It covers more tests in lesser time with a higher level of code coverage than manual testing. However, manual testing should not be completely abandoned as it is not feasible to automate each and every test. A true combination of the both the testing approaches is said to provide optimal results.

-      Virtual Testing Resources: Use of virtual testing resources on both the client and the server side can prove to scale performance, stress and load testing most effectively. With less costly hardware resources, this approach is relatively inexpensive in terms of infrastructure, set-up and cost.

-      Cover your testing through both emulated and real devices: Testing only through real devices or through emulators only cannot achieve the sufficient level of quality confidence. Cloud testing services provide real time access to a wide collection mobiles devices and network operators in addition to built-in mobile testing frameworks. Thus, testing can be covered in the most cost-effective way, for most of the available devices in the market.

-      Analyze End-User Experience of the Mobile App through analytical tools: Consider the current analytic SDKs to gain a real insight into the user interactivity with your app. Review the detailed report on crashes and identify problems that need to be corrected immediately.

Conclusion

With the continuous release of new tools and techniques, mobile application testing is upgrading everyday. Regarding the combination of the different techniques and methods used in mobile application testing, several tradeoffs and choices need to be considered. There is no single testing method that will completely satisfy the required criteria. Each has its own pros and cons. Hence, consider a strategy that combines many different options of testing and provides the best results with a balance between quality, time-to-market and cost.

[mobile-app-tesing-blog]

The best of both Real Devices or Emulators for Mobile Application Testing.

Mobile Application Testing is a crucial step before its launch. The default for most of the organizations is to go with real devices. While it generates more accurate results, it is not ideal for test automation. In terms of expense, emulators can be a more convenient option. However, each of these options has their share of positive and negative effects in mobile application testing.

With Mobile Applications many organizations prefer to carry out the majority their testing on real devices. Applications are still built on developer’s Mac, and are tested manually on the few devices that are available in the device lab. Many devices are thus ignored, which result in inefficient launches.

Real Devices in the cloud run tests on actual phone hardware and software. These cloud based devices are remotely housed in vendor premises and are accessed by sending scripts to the devices over the network. These scripts are then executed on the devices and test results are received in the form of detailed logs and error reports. Such a device lab can be built in-house, but such an effort takes months and requires resources that can otherwise be assigned to better tasks.

Testing under real conditions makes a significant difference in the quality of a mobile app. Using real devices in a cloud brings a marked improvement in flexibility, scalability and cost-efficiency. Considering real devices in a cloud can be considered as a key component in a QA strategy.

EMULATORS FOR AUTOMATED MOBILE APPLICATION TESTING.

Testing on Emulators are used in an attempt to move away from testing on real devices. An emulator emulates the hardware and the software of the device on a desktop PC. It can be considered to be a re-implementation of the mobile software written in a machine level assembly language.   A simulator, on the other hand replicates only the device user interface and not the hardware.

As emulators are software driven, they are much faster than real devices. Additionally, they enable parallel testing and test automation via mobile test automation frameworks. This improves the speed of tests as compared to the manual testing on real devices. Tests on emulators are software defined and hence can be simultaneously run on multiple emulators without any manual effort for setting up the devices.

A COMBINATION OF REAL DEVICES AND EMULATORS

Today, organizations are divided into two extremes based on their priority. Some organizations rely only on real devices considering they are not compromising on quality, whereas some organizations test exclusively on emulators because they are fast, cost effective and easy to maintain. However, siding with any extreme is actually a compromise. Real devices are expensive and not scalable while emulators are unable to provide a real- world test-environment.

The best QA strategy is to combine the use of emulators and real devices. Thus drawbacks due to one gets covered by the other. The next decision to make is about which tests to run on real devices and which ones are to be automated on emulators. This decision varies through each organization and through each project. Certain guiding principles can be followed t help in this context.

CONCLUSION

Real Devices and emulators when used together in mobile test automation, delivers the most out of the efforts on mobile application testing. Any organization that plans to compete in the mobile market cannot ignore the value of real devices or emulators in their testing efforts.

[mobile-app-tesing-blog]

Performance Testing of Mobile Applications

The users of mobile application are very demanding. They look for feature rich, fast and responsive mobile apps, no matter the network, location or the device. They want immediate fixes in case of glitches. In case they are not satisfied with the performance or the service, they are very fast to uninstall the app and look for the next application in the market.

Performance Testing is an important aspect of the overall mobile application testing. A proper performance testing of mobile applications ensures that -

-      The response time of the mobile application is well within the acceptable performance boundaries. This can be referred to as UI based performance testing.

-      The load generated  on the back-end infrastructure is manageable. This check falls under load generation or stress testing.

-      The performance of the application remains consistent after its release. This can be referred to as performance monitoring.

The mobile platform offers a wide range of new challenges that requires to be taken into account while testing the performance of the mobile application. Processor and memory constraints are more on mobile platforms. Additionally, the network connectivity between the mobile device and the data centers or the cloud are also more constrained. The impact  of the network layer is considerable while testing the response time of the application. Tests executed close to the data centers on high bandwidth, low latency networks may not generate the same performance as that experienced by the users. Integration with Network Virtualization for Mobile allows to siulate the various mobile network conditions across the globe, allowing to assess the performance of the application in a specific geography without even having access to devices in that geographical location.

As the popularity of the application grows, more and more users start connecting to the back end servers. Load generation allows  simulating the load generated by these users.

The same techniques can be used in UI based performance testing to measure the response times experienced by the users.  Load can be generated at two levels: at the protocol level and the UI or device level. The device level load generation is a more precise match of the load generated in production. But the protocol level load is easier to set up and configure. Both physical and virtual devices can be used for generating the load.

It is obvious that the performance testing of a mobile application is a continuous process. With the increase in app usage, the back-end infrastructure becomes overloaded, which may lead to degradation in the app performance. Additional factors such as network reconfiguration and changing user patterns also impact the performance of the application. For monitoring the infrastructure, the existing performance tests an be re-used. This saves from re-implementing the same test scripts all over again.

Conclusion

There are various automated solutions available in the market for performance testing and monitoring. However, the choice of the tool does not depend on the analysis and achievement of your objective. Once you get the results, you need to analyze them carefully for uncovering inconsistencies and further run more tests to verify the cause of these inconsistencies. Cross verification of tools is also necessary in order to rule out any inconsistent performance characteristics.

[mobile-app-tesing-blog]

Use of mobile analytics to optimize Mobile App

Industry statistics indicate that smart phones and tablets now account for the majority of the global internet traffic. Organizations are able to reach consumers more than ever through mobile websites and applications. For an application failure, majority of the customers hold the mobile app responsible, not the device or the back end. Hence, it is very important to ensure optimal performance of a mobile application.

All users expect mobile apps to work seamlessly regardless of their choice of technology. The whole point of optimizing mobile applications places users at the core of our strategies. User experience is how users think, receive and feel. There are various attributes of user experience and they are intimacy, immediacy and privacy as key factors. A good quality software may not be enough to provide a brilliant user experience. However, poor quality adversely affects the application.

The most practical considerations of optimizing App Testing can be summarized as follows:

Improve the mobile testing beyond     the ‘shift left’ recommendations

Leverage the     mobile analytics to increase user experience and the work quality.

Move from the     traditional testing into the design, analysis and application of data     generated by mobile apps.

Keep testing     focused on user experience and business success.

Reduce     irrelevant practices and prioritize development and testing efforts.

Testing mobile apps helps the team to find out flaws and problems and get them addressed rather than letting the users to find them. In testing mobile apps, much of the testing done is wasted effort. Automation simply helps us to run tests faster,  but this does not necessarily imply that testing is of any better quality.

Certain features when discontinued by organizations, leads to loss of existing customers. By focusing on mobile analytics, and understanding user interaction with the different features, better quality app can be generated for the customers.

Each review leaves some sort of clues about a problem in the app. By paying attention to the ratings and reviews of the applications, bugs can be found and fixed before it affects the user base. Sometimes users leave a detailed technical description of the bugs they came across while using the app. They also suggest some additional features that they have been looking for. Giving importance to them and working to solve them and finally updating a reply to their feedback and reviews can further enhance the application and make it more usable.

Conclusion

Mobile Analytics is one of the several different places which can be used to improve the quality of the app. Most often companies who are trying to test applications do it on a limited number of devices. Leaving out certain devices may generate cases where the application may be exhibiting very poor behavior. Mobile analytics help to understand the effects of the software running on a variety of technical infrastructure. Discovering how the app behaves  virtually across all population and understanding its real usage an bring realism to the testing efforts.

[mobile-app-tesing-blog]

5 features of Appium tool for both iOS and Android platforms that stands out

The main concept of Appium is that native apps should not need an SDK and should be able to use preferred test frameworks and tools. Appium thus allows to automate any mobile app, native or hybrid, developed from any language and any framework.

 It uses the JSON wire protocol to interact with iOS and Android along with Selenium WebDriver. It is this interaction between Selenium client libraries and node.js server which works together with the mobile application. One of the core features of Appium is that the test codes can be written in any framework or language without having to modify them. Appium is able to run on a variety of devices and emulators making it one of the most apt choices for mobile automation.

 Appium on iOS

 Apple provides an application called instruments which is used to perform a lot of activities like profiling, controlling and building iOS apps. It also has a component for automation where javascript commands can be written using UIAutomation APIs to interact with the AppUI. To automate iOS Apps, Appium makes use of these libraries.

 In context of a command life cycle, the Selenium WebDriver sends a command in the form of JSON via http request to the Appium Server. The Appium Server understands the automation context and sends the command to the Instruments command server. This further waits to be picked up by the Instruments command client and execute it in Bootstrap.js in the iOS environment. Once the command is executed, this message is sent back to the Appium server by the command client. This cycle repeats itself until all the commands get executed.

 Appium on Android

 The cycle in Android is similar in to iOSl. Here Appium proxies the command to a UIAutomator test case running on the device. The UIAutomator is Android’s native UI automation framework which supports running JUnit test cases from the command line directly into the device.  Although it uses Java as the programming language, but Appium makes it run from any of the WebDriver supported languages. Unlike iOS, here the Bootstrap.jar is in place of Bootstrap.js. It represents the test case when compiled in java. On getting launched, it spawns a TCP server. Here, the TCP server resides in the device and the client resides in the Appium process. This is exactly opposite to the way it is is iOS.

 The pros and cons of Appium

 -      All the complexities here are under the Appium Server. This makes the whole experience same for an automation developer irrespective of the platform being automated.

-      Appium opens up the door to cross platform mobile testing. This makes it possible to execute the same test on multiple platforms.

-      Appium can automate Native, Hybrid and Web mobile applications.

-      Appium uses UIAutomator for Android which only supports Android SDK platform. Fro older APIs, Selendroid is available. This may not be taken as a limitation, but can be considered as an overhead in the configuration.

 Conclusion

 The Appium architecture has grown continuously with new feature additions and is constantly under regression. Hence a fair amount of rearchitecture is under consideration to solve these issues and migrate the codebase to JS which is easier to understand and work with.

[mobile-app-tesing-blog]

Concepts and basics to get started with Appium

APPIUM is an open source, cross platform automated testing framework for testing Mobile Application. It allows the testing of both native and hybrid applications. It supports test automation on both physical devices as well as emulators and simulators. Appium supports all languages that have Selenium client libraries.

Concepts of Appium

-      Architecture: Appium works on the concept of REST. It receives the client connection, listens for commands, executes the commands on a mobile device and gives out the result of the execution with an HTTP response. Having a client/server architecture allows us to write test codes in any language with http client API. However, it is easier to use Appium client libraries.

-      Desired Capabilities: It is a set of keys and values sent to the Appium server to define the automation session to be started. Various capabilities can can modify the behavior of the server during automation.

-      Session: Automation is performed in context of sessions. Clients initiate a session with the server, which is specific to each library with a JSON object called the ‘desired capabilities’ object. The server starts up with the automation session and responds with a session ID used for sending further commands.

-      Appium Server and Client: Appium Server is written in node.js and can be built and installed from source or directly from NPM.        

-      Appium.app and .exe: These are GUI wrappers which come bundled with everything required to run the Appium server.

Appium Prerequisites.

Android     SDK

Java     Development Kit

TestNG

Eclipse

Selenium     Server JAR

WebDriver     Language Binding Library

APPIUM     for Windows

APK     App Information on Google Play

Node.js

Appium is an HTTP(HyperText Transfer Protocol) server which is written in node.js. It creates and handles multiple WebDriver sessions for different platforms like Andriod and iOS. It is very similar to Selenium in perceiving http requests from client libraries and handling them as per the platform.

Appium starts a test case on the device which spawns a server and waits for proxied commands from the main server. Android and iOS have a different mechanism to run a test case on the device. Appium runs the test case after listening commands from the appium server.

Getting Started with Appium on Windows

Install     Android SDK in the system

From     the left panel link follow Control Panel> System and Security>     Advanced System Settings> System Properties> Advance Tab>     Environment Variables. A pop up will appear, double click on PATH and set     the ANDROID_HOME variable pointing to your SDK directory.

Use     and Adroid device or an emulator to the system.

Navigate     to Android SDK directory through the command prompt.

Run     ‘adb devices’ command which lists the connected emulator instances. Run     ‘adb start-server command’. This starts     the ADB server that is used by Appium to send commands to the     Android Device.

Navigate     to the Appium directory and click Appium.exe to start Appium.

Click     the launch button without altering the IP address or port number.

Click     on start button to start running Appium server on your system.

Getting Started with Appium on Mac

Download     and start the Appium server with 0.0.0.0 as the default IP Address and     4725 as the port number.

Start     the simulator on Mac machine

Click     a launch button. This enables a blue color icon. Clicking on this blue     icon will open up the Appium inspector and Simulator with pre-selected     application.

The     Appium inspector shows the hierarchy of the element in a column wise     structure.  Also actions like tap     and swipe can be applied.

Clicking     on Stop, stops the recording.

Conclusion

Appium provides a very high level API. It is now the only known test automation framework which can support both Android and iOS. Test automation is expected to enable parallelism and concurrency. Appium enables this concurrency and makes the development and testing agile with faster turnaround, better quality and robust apps.

[mobile-app-tesing-blog]

Security Issues of Mobile Banking Apps.

The smart phone banking makes use of the mobile channel servers to replicate online banking onto the mobile devices. The devastating effects of cyber attacks on mobile banking apps are very intimidating. Hence, such apps are in need of constant refinement. The refinement of mobile banking apps is as important as retaining customers. Banks and financial institutions are investing more on the security of mobile banking apps.

Many users consider banking apps to be completely secure and hence pay bills, initiate transfers and check accounts on the fly. But doubts over the security has also kept a significant percentage of the consumers away from trying the app. Similar concerns over the security has also kept the banks to delay additional range of financial services through the channel. It has been predicted that mobile malware and security breaches are going to increase significantly in the times to come. Thus, it is important to recognize a few issues of concern before using a mobile banking app.

-      To ensure security, banks need to authenticate users accessing the mobile channel. Depending on the risks of transactions, a two factor authentication should be implemented. Two factor authentication typically relies on the one-time password delivered to the user via hardware token or a text message. But they are not a guarantee of protection from phishing or malware. Consumers find it cumbersome to carry a hardware token with them all the time and with frequent attacks against GSM and 3G cellular networks, the confidentiality of text messages cannot be assured.

-      Considering the technical variations in mobile devices, every platform specific app needs to accommodate the device specific set of challenges. The mobile app developers need to adequately understand the risks that proliferate in this extremely complex ecosystem ensuring secured mobile data, connections and transactions.

-      Fake apps or hacked apps are a great security threat. Consumers are at a greater security risk by downloading apps from unofficial app stores. The two most widely used mobile operating systems are the Android and Apple’s iOS. Android is the most widely used and allows users to download apps from play stores with less stringent policies. On the contrary, iOS have a very strict process of reviewing apps to eliminate the policy of malware.  

-      Many mobile banking apps lack the adequate implementation of SSL or certificate validation. This makes them even more vulnerable. SSL(Secure Socket Layers) and TLS(Transport Layer Security) are the two most important digital security protocols.  They are mainly used to establish encrypted links and ensures server authenticity.

-      Mobile devices are very hard to tell apart. This is due to weak device fingerprint. It is about gathering information about a remote computing device to identify it uniquely. Some enterprises attempt to identify devices in this way, but privacy concerns have limited the amount of data that can be shared. This has made difficult to identify individual mobile identification. This also limits the two factor authentication process which needs to be implemented without fail.

Conclusion

Mobile banking is evolving very rapidly. To boost the adoption of mobile apps and unlock its true potential, financial institutions need to address their customers’ data security through investing in high technology and using social media campaigns to highlight their efforts. For the success of mobile apps, it is important to design a secured mobile banking app that provides high user experience.

[mobile-app-tesing-blog]

Cross platform threats for Mobile Applications

The security of Mobile applications is very different from that of desktop applications. The goal of mobile app security is to make the supporting platforms secure rather than making the users rely on third party security services. The cross platform threats for mobile applications include malicious attacks involving multiple platforms. With the advancement of mobile devices, cyber criminals have now taken into the portable platform for their attacks. Some of these threat approaches are:

-      Same for all the platforms.

-      Specialized for every platform.

-      Based on supporting components that help it run on different platforms.

-      Begin on one platform evolving through to the next platform.

Some the recently identified threats are:

-      ANDROIDOS_USBATTACK.A : This is a malicious app which not only performs information theft, but also downloads malware that gets triggered only when the device is connected to a PC through a USB port.

-      TROJ_DROIDPAK.A : It is a Trojan that downloads and installs malicious apps into any Android device when it is connected to the affected desktop. These apps are malicious versions of banking apps which have the capability of compromising a user’s online banking account.

Cross browser mobile threats are possible due to multiple factors. One of them is the popularity of the social engineering apps like WatsApp, Facebook etc., which makes users click on random links for more connectivity and interesting quizzes. Another factor is the obsession with anything contemporary like celebrity facts, contemporary gossips etc., which require a user go through random links and grant access privileges to their account. Mobile phones are also used as PC substitutes. Users most of the time stay connected to their home PC through their mobile phones. This is a viable security hole for hackers. The Android fragmentation problem is also a risk that has not been solved as yet.

Cyber criminals can also use infected mobile devices to carry malware to facilitate targeted attacks. The other way round, an affected business can target all the mobile devices connected to its network. Either way, critical information is at stake.

Cross platform mobile apps can also be used to hack into home automation systems and take over control. Further, this can extend upto recording of the activities going around at home and disengaging the security devices in the house. Hence, it is very important to secure the points of entry of cross browser threats. By securing these end points, this chain of devastating infection can be prevented.  

Conclusion

With great advantages, the risks are greater as well. The mobile platform is a very viable source for the hackers, especially with all the sophisticated apps that are being developed which almost represent our identity online. With all the sensitive credentials stored in the mobile apps, it is very easy for a hacker to impersonate us once he gets through into the system. Many of the security threats have not been detected thus far,but it is only a matter of time before one of us becomes the victim.

[mobile-app-tesing-blog]

Mobile Application API Testing

API or Application Program Interface is a set of routines, tools and protocol blocks for building applications. This also helps to reduce complexities since the API’S are already tried and tested. Every application nowadays has API’s and Web Services integrated into it. An API service needs to meet performance metrics like response time, stability, security and scalability.

Hence, while testing a mobile application, it is equally important to test the involved web services and APIs as well.

Mobile applications communicate with its associated APIs through standard protocols. These protocols are mainly based on HTTP and REST principles through JSON and XML data format.

In the Agile method of software development, testing is done iteratively. In waterfall model, testing is the final phase before delivery. In either case, the integrated API’s require thorough testing regarding technology, security, data format, authentication and management. API testing can be considered as an integration testing of the classes, objects, scripts and data stores that  have been used to implement them.

The initial API testing focuses on the functionality to return the expected responses, as per the input provided. A keyword driven approach can be followed         which allows the testers to test by providing the inputs. Alternatively, a data driven approach can be followed to obtain expected outputs by providing large sets of input data.

It needs to be made sure that the performance of the API is in line with the requirements and works as expected under different load and stressful conditions. The entire point of load testing is to make sure that the API is able to handle demand efficiently. Whether we already know the limit or want to check its threshold limit, we can achieve this by

-      Throwing multiple varying loads of traffic at the API simultaneously

-      Record and replay traffic in large volumes.

-      Check the effect of the load test or the server and the network resources.

Since we have chosen to integrate a third party API, its security testing is extremely important. We need to ensure that the integrated API is not vulnerable to attacks such as SQL injections, altered inputs and boundary violations. It is very important to make sure that whatever be the kind of error condition, it does not expose any detailed or sensitive internal information of the system.

Having APIs further dependant on other APIs, increases the challenges by several degrees. One example of this is the frequently updating map coordinates in Google along with the route calculations. If the responses of the integrated API depend on the values provided by the other API, we need to ensure that these third party API’s always return the same outputs for a given set of inputs. This can be ensured by setting up simulations of the external API. These simulations can be configured to return a predefined set of results which behaves as a fixed environment for the actual integrated API. Alternatively, assertions can be made that the tests are immune to the changes in the external data.

Conclusion:

The quest for quality never really stops. Monitoring the functionalities and performance is crucial at all points of time. In continuous delivery model, this becomes all the more important. All API tests need to be designed and performed early, preferably continuously throughout the development process as the system evolves.

[mobile-app-tesing-blog]

Mobile Application Test coverage using Mind Maps

Mind map represents an idea or a concept graphically. The various phases of testing are represented using colors, lines, symbols  and images, thus interpreting the entire test plan and its activities in a visual format. Main maps is used in requirement analysis, test design and assessing test coverage. They can also be used in validating a test case as well as representing the steps of a complicated test case.

Steps for creating a sample test in mind map:

Draw     the main subject in the center. This can be referred to as the node.

Draw     radiating branches from the center to depict the various themes in the     main subject. These branches lead to the child nodes.

Split     each branch into twigs to further represent the activities that take place     under each theme. The child nodes (now a parent node) further leads to its     child nodes.

Use     short sentences to reduce verbiage.

Differentiate     different tasks by color coding them.

The     mind map needs to be balanced. It cannot be too elaborate, neither too     short of information. If some activities of the mind map are too     elaborate, it might as well be moved to a separate mind map. The main mind     map can be linked to the sub mind map as well.

Mind     map needs adequate research and brainstorming to ensure it is complete and     correct.

Xmind     and Free Mind are two are two open source tools for mind mapping. They     arrange information logically by nesting sub-functions under main functions     in the form of expandable and collapsible nodes. This helps to keep the     diagram clutter –free.

Example of a mobile app testing mind map:

1     Mobile Testing

-      Structure

-Product Type

- Is it downloadable

- What happens when OS is updated

- What happens if it is removed

1.1         Function

-What does it do?

-UI

-Errors

-Redirection

-Non designed tasks

-Designed tasks

-Does it need certain services to be turned on?

1.2         Platform

-Mobile Features

-Software

-Standard or pre-installed

-GPS

-Multi-tasking

-Email

-Switching between apps

1.2.1.1         3G/Wifi

-UI

-Syncing

-Keyboard

-Touchscreen

-Peripherals  

1.2.2     Application Store

-Device Used

-Submission requirements

-Development Specifications

-User manual

1.3         Operations

-Connectivity

-In motion

-Airplane mode

-Intermittent

-3G

-Wifi

-None

1.3.1     Interruptions

-Voice mails

-App notifications

-Battery

-Phone Calls

-Text Messages

-Forced Updates

1.3.2     Feedback

-User Reviews

-App Reviews

-Ratings

-Comments on social media

1.4         Data

-Input/Output

-Languages

-Locations

-Preferences

-Contacts

-Friends

1.4.1     Files

-Size

-Media

1.4.2     Tracking and Updating

-Analytics

-Check ins

-Friends

-History

-Purchases

-Reward Points

-Syncing

-Web and mobile account consistency

1.4.3     Time

-Time Zones

-Server Time

-Device Time

 The above is a simplified textual representation of a mind map for mobile application testing. Many nodes can be included, excluded and expanded. It depends on the application and its functionalities.

Conclusion:

All that is required to implement Mind Maps is the knowledge about the product and clear understanding of the flow of application development. It requires all the attributes of each use case. Mind Map focuses not only at the granular level, but also on the overall picture. It ensures comprehensive test coverage, be it manual testing or automated testing.

[mobile-app-tesing-blog]

Ensuring quality through automated testing of Mobile Applications

It has been a  long time since the simple mobile phone has transitioned from a mere medium of mobile communication into a multi-purpose, personal gadget that is eventually replacing most of the digital and electronic devices. The proliferation of mobile devices across different platforms and operating systems, have made it more challenging for developers to design and release an all compatible application. Mobile applications need to be tested in order to ensure that they run on the main operating systems and across multiple networks.

The best way of ensuring the quality of mobile application can be achieved through test automation.

There is an increased need for mobile application test automation for improving its speed and efficiency. A well implemented automation strategy can enable the developers and testers to reduce efforts and the time to market. However, there are a few challenges in mobile application test automation:

- Hardware challenges: These are limitations in processing speed, memory size and differences in communication protocols.

- Network challenges include multiple network types, variable connectivity speed multiple network operators with custom features.

- Device challenges include diversity in platforms, browsers, display rendering, and difference in run times.

Mechanisms for Test automation of Mobile Apps

- Lab based automation: This method involves leveraging devices and simulators in a test environment by using open source testing tools.

- User based automation: This method makes use of the user agent add-ons that come with desktop browsers. Here, the functionality can be automated by using popular automated testing tools such as QTP, LeanFT, Selenium etc.

- Cloud Based automation: These solutions are offered as a unified package. They are highly effective as they support web based cross platform QA.

Principles for selecting the right mobile testing tools

- Before narrowing down the tool or the tool set, it is important to start with the PoC(Proof of Concept) on critical platforms

- Organizations need to consider how the testing tools will integrate with the test management environments.

- Create appropriate time saving framework by identifying and grouping cross platform reusable scenarios.

- Choose critical tests for native apps that can run across releases. Also ensure its multi-browser compatibility and cross browser re-usability.

- While determining the form factors, always consider all the form factors.

- Consider the different aspects of connectivity for automation execution runs.

- Selection of both devices and platform simulators can help optimize the test runs on different platforms. - Perform the feasibility of the test tools considering the variety of mobile platforms and technologies.

- Consider test automation in non-functional scenarios that include battery state changes.

- As most of the mobile projects are Agile, consider the early implementation of test automation.

- Determine the ROI for each mobile platform and the time period of execution. Consider data driven automation support. This increases iterative executions which in turn increase ROI.

- Smoke tests, regression tests, user acceptance tests, synchronization tests, compatibility tests, GUI tests and cross platform tests are good candidates for test automation as they have repeated business logic.

Conclusion:

The underlying principles and the success factors of mobile application test automation are different from those followed for conventional applications. Devices, environment, test coverage, tool sets and the variations in the associated supporting factors add to the complexity. Effective test automation  of mobile application requires an  insightful approach and a well orchestrated strategy.

[mobile-app-tesing-blog]

Ensuring a realistic test coverage of Mobile Applications.

Over the years, the software testing discipline has evolved from a mere tactical approach into a  highly strategic process. Numerous factors have contributed in this dramatic reshaping of the testing process. The growing software sophistication, along with unimaginable features have also introduced severe functional and performance issues. Hence, test cases of equal complexity as that of the code is required to keep these issues in check. Keeping the following considerations in mind can help in a better testing of mobile applications.

1. Knowing the User – A good testing starts with a comprehensive understanding of the user and his expectations of the software. Testers need to take their time to fully understand the context of the application usage. Before test initiation, it is very important to understand the final performance goals to be able to test properly for the real user experience. For a better understanding of the user go through the initial requirement specification. Investing in market research for identifying user preferences and behavior can help in planning tests.

Considering the following factors may help:

- Time of ac - cessing the App during the day. - Duration of the session - Mode of browsing – in a hurry fro immediate answers or just as a pass time. - Any second screen experience - Likelihood of simultaneously accessing multiple apps. - Cart duration of shopping items. - App performance and revenue correlation.

2.Knowing the App – Users do not care about the features, they are more particular on their experience. Testing is mostly inclined towards functional testing – in mapping the features and its performance as specified in the requirements document. But to properly cater to the user experience, it is important to understand how the user would like to interact with the application. This user intimacy will to a long way in determining the core features and the edge cases.

3.Determining your Test Matrix – The scale of a test matrix is directly related to the budget and value of the project. However, all tests will not be equal, since all functions are not  equal. Hence, analyzing the following points can help in prioritizing the test cases :

- The core functions - The core experience - The core differentiators - The edge or the less important functions.

4.Determine your Devices: The above three points have helped us to determine what to test. Now it is required to understand how it is to be tested. With the rate of new releases and upgrades in the mobile portfolio, for testing purposes, it is a difficult to decide how many will be enough and how many will be too many.  Once the number of devices has been determined, the most important device functions need to be prioritized. The most effective way of determining this is by analyzing the real user data which gives the following statistics:

- Your actual user base - Frequency of visit - Devices and OS used for access. Further comparing device OS vs correlated revenue can help is assessing the risk of test elimination for any given device.

5.How to Automate – Step 3 helped to determine the test matrix for the application and prioritizing the core functions and the edge functions. Here, we need to determine which tests need to be automated. Build the regression test suite with simple and repeatable tests for automation and assign complicated tests to the testing team

6.Knowing the performance – An application’s true quality can be considered as the intersection of its real user data, functional testing and the mobile performance. This data can be used to create real world user test cases and its performance with business metrics. The following factors can help in determining the best performance testing strategy for your app:

- Develop in-depth mobile performance strategy encompassing scenarios and data from the real world. - Analyze the back end metrics and the invisible factors of your app that affect its performance. - Analyze the front end performance metrics to get an understanding of the user experience with your app. - Identify the hot zones where the performance of the app can be effectively improved.

- Bind the performance of your app with revenue generating transactions.

7.Knowing your edge cases: Edge cases refer to the less frequently occurring scenarios. These are also the greatest source of crashes and obstacles of a seamless app performance. With all the time spent in checking core cases, edge cases often remain untested. These can however be covered with automated testing.

Conclusion

The impact of the above steps results into a fast, comprehensive, test strategies which are more aligned with the market goals and product strategies. Depending on the overall product strategy test resources can be allocated for a faster time to market or for a deeper testing to achieve the highest level of performance quality.

[mobile-app-tesing-blog]

Testing with Selenium and Appium

Mobile Application testing has been a manual process for a long time. It has always been limited by the available devices at hand or in the cloud. Emulators too, have not been enough to for a complete test coverage and there has always been a  need for better testing techniques and automation tools. 

Appium is a  mobile test automation framework for native and hybrid mobile applications  for both iOS and Android. It is more suitable for testing when real devices and real browsers are being considered. Appium is based on Selenium. It internally uses the JSONWireProtocol to interact with iOS and Android using the Selenium’s WebDriver.

Concepts of Appium

- Client/Server Architecture: Appium is a webserver that makes use of the REST API. It receives connection request from a clients, listens for a command, executes it on a mobile device and sends back the execution result through an HTTP response. Having a cloud server architecture enables writing test codes in any language that has the http client API. The server can be put on a different machine and further rely on cloud services to receive and interpret commands.

- Appium Server: It is written in Node.js and can either be installed directly from the NPM or can be built and installed from the source.

- Appium Clients: It has client libraries in Java, PHP, JavaScript, C# and Python to support its extensions to the Web Driver Protocol.

- Session: This is always required to perform an automation. Each session is initiated as per the library used, but finally it is a POST/session request to the server calling a JSON object.

- Desired Capabilities: These are a set of keys and values, which inform the Appium server the type of automation session is to be started.

Selenium Testing of  Mobile Applications with Appium The following steps show how to use the Appium testing framework to run Selenium test on a local machine or a real Android Device.

Options for running Appium

- Use Appium App – Download the app> Run the downloaded file> Run a test using the language of choice.

- Command line for iOS

$ npm install -g appium $ appium& Make sure to authorize the iOS simulator before starting. - Run Appium through Command Line for Android mkdirappium-test && cd appium-test npm install -g appium  # might have to do this with sudo sudoauthorize_ios # enable developer use of iOSsim npm install wd curl -O https://url/appium/master/sample-code/examples/node/simplest.js appium& node simplest.js

Writing Selenium tests with Appium

A subset of Selenium WebDriver JSON Wire protocol elements is supported by Appium. They can be found by  using Selenium/WebDriver's element-finding strategies.

Selenium Mobile test on real Devices

If there is  a real Android Device connected and recognized, then it will work exactly as an emulator.

Conclusion:

Selenium is already a de-facto test automation standard for web application and now Appium provides very high level API which can be easily adopted by Selenium developers.  It is currently the only known framework to support both Android and iOS. It also uses the same test script API and the same workflow It is an open source independent framework that can be downloaded and used freely on ones own machines and devices.