Automation Testing

[test-automation-blog]

The Role of Test Automation in CI-CD

In this competitive market with rapidly changing requirements and evolving products, continuous delivery, continuous integration and continuous testing are all intertwined. This extremely fast adoption of business evolution, puts a serious pressure on software development. Conceptualization, design, implementation, and deployment, all needs to be done within a span of a few months or someone else will come up with it before you do. Also, after the product is deployed, be it a web or a mobile application, constant improvements and enhancements are expected.

Continuous Integration and Continuous Deployment are most often used together and are referred to as CI-CD. Individually, they represent two different processes. CI refers to running integration tests at every code change, while CD refers to deploying every change that are validated by the tests. However, now-a-days CI-CD is used to refer to the automation of processes from development to deployment. Hence, CI-CD can be said to consist of the following steps:

-      Change of Code or Commit

-      Building the software based on the code developed.

-      Automated testing

-      Deployment of the changed code

In the above process, Automated Tests can be said to be the most important part of the CI-CD process. There can be no Continuous Integration or Continuous Delivery without automated tests that run fast, have a good coverage and no errors.  

Test automation can be divided into multiple suites, each having its own objective.

-      Unit Tests: This test is run first before any changes are made to the code repository. Normally, individual classes and functions are tested here. This test is most often performed by the developers themselves.

-      Integration Tests: This test makes sure that the different modules of an application work properly with each other. Generally, integration tests need to be performed in environments that represent production environment.

-      System Tests: This tests the entire system in an environment that closely resembles the production environment.

Through Continuous testing, developers are able to perform a live test of the functionality and behavior of their code with test automation tools. Regression Testing, User Acceptance Testing and Performance Testing, all need to be executed in order to ensure the quality of the application in various customized environments. When the test module configuration is correct, tests can be automated until the alert systems notify a build failure.

Without proper execution of Continuous Testing, CI-CD will only remain an unfulfilled requirement. Continuous testing or Automated testing,  helps saving in terms of production costs, as bugs reported after deployment in the production environment is a costly matter.

Conclusion

On the road to achieving continuous delivery, the main requirement is to minimize the cycle times and keep the costs low at the same time.  Automated testing helps to achieve this and build an effective system. The best approach on the way to CI-CD would be to invest in a robust and scalable test automation suite.

[test-automation-blog]

Bringing efficiency into software testing and test automation.

In order to maintain a competitive edge, organizations constantly need to improve their products. This also requires them to maintain the quality of their products. This increases the pressure on test automation framework with constant changes, shortened release cycles and reduced time allocated for testing but with uncompromised efficiency and effectiveness.

 Generally, a high proportion of testing is usually dedicated to regression testing, which makes sure that the current system is still working even after new changes and enhancements. Such testing is generally tedious and prone to errors. Automation ensures that regression testing is performed with consistent quality within the available time.

Test Automation Frameworks help to increase the speed and efficiency while maintaining the product quality.

 Some of the best test accelerating factors can be listed as follows:

-      Hire experience testers. Many organizations start testing as an entry level job. As a result the testing process is not executed efficiently. Experienced developers and testers can foresee code defects and perform effective testing in lesser time.

-      Facilitate interaction between the testers and developers. Better communication saves the nuances of email updates and creates a friendly environment for work. This benefit though not quantifiable, but is effective in the long run.

-      The reporting system for testing should be independent. The testers should be made to report directly to the project manager and the business analysts. This helps the project manager to plan the development accordingly. This also involves the testers in the development cycle early on.

-      Allocate dedicated testers into specific systems. This deepens the testers understanding of the system and how it needs to work for optimum results.

-      Providing a practical business training to the testers allows them to get a real understanding of the business application that they are testing. It also enables them to identify potential loopholes in the system and suggest features for future enhancements.

-      Testing mostly involves technical fiddling. Involving non-technical users into the process of testing provides a different perspective in finding out the performance issues of the software application.

-      Many organizations prefer to run their testing phases directly in the production environment. This approach is faced with several limitations regarding security protocols due to which complete end to end testing is not possible. Hence, organizations need to proceed towards replicating the production environment to get an idea the software performance in the actual environment.

-      The Agile and the DevOps methods have involved testing right from the requirement gathering stage. Practicing this approach helps to detect flaws early on and saves time in the long run.

-      Equivalence class partitioning is a mathematical approach which helps in identifying additional functional requirements which may not have been articulated. It gives a good idea of the number of test cases required to test a system’s functional requirements. It is a way which can list all the ways a system can be used by a user.

Conclusion

The discipline of software testing is undergoing seismic shifts from a wide variety of perspectives. Tactical processes have now evolved into strategic exercises. Among them, understanding the user, the application, the economic resources, access devices, test automation framework  result in a more comprehensive, aligned and faster testing cycles.

[test-automation-blog]

An overview of Scriptless Test Automation.

With time, the complexity and maturity of applications have increased and so have the approaches of testing and test automation. There are many testing tools available in the market now a days, which claim to be simple, easy to maintain and have extensive features. However, for better test coverage, testers are always required to leverage coding and scripting. More codes imply more maintenance costs for scripts and frameworks.  Thus, efforts were put on developing processes with little coding involved and still achieve quality and maintainability. This paved the way for Scriptless or Codeless Test Automation.  

Most of the functional test automation is used in regression testing. Here the effects of change can be seen only when the developers have delivered the code. It gives the automation tester very less time to adapt to the changes made. This makes it very important for the testing teams to be ready for such situations. To cater to these challenges, many automation frameworks with sophisticated code development and modularization came into being. They are Keyword Driven Framework, Modular Framework, Hybrid Framework, etc. Each automation framework resolved issues as per project priority.

But most of the time, the maintenance efforts exceeded the development efforts due to debugging complexities.

With the introduction of the Agile development methodology, test automation became more challenging. Here more and more time was being spent over test automation development, maintenance, creation of tests and updating them as per scope changes finally executing them.

With increasing competition, organizations focus more on the cost benefits and improve cost efficiency. This finally evolved into the development of a script-less test automation in order to simplify the development and maintenance of test automation.  The most aggressive, encouraging factors for script-less automation can be listed as follows:

-      Delivery Speed

-      Overall maintenance

-      Configurability

-      Robustness

-      Efficiency

The benefits of scriptless test automation are numerous.

-      Test Automation becomes easier and comprehensive.

-      Its pre-defined set of methods reduces the time required for automating test cases.

-      There is no code level debugging. Hence, maintenance becomes easier and less time consuming.

-      As no programming language is required, it is easy to train non-technical engineers and business analysts, which enable them to contribute towards test automation.

-      As lesser to no time is required in scripting, testers are able to focus more on analyzing results, detecting inconsistencies and focusing on functional aspects.

-      Automated and Manual tests, both are located at the same place which gives a transparent view of what has been automated and what has not been automated.

-      Increases the productivity of the test process.

Conclusion

Scriptless Test Automation is an idea to resolve most of the limitations and challenges of test automation. It does not confirm the obsolescence of code based test automation. However, with the increasing complexity of software technology and their features, scriptlessness is set to be the future attribute of test automation frameworks.

[test-automation-blog]

Test Automation: The focal point of CI/CD workflow.

In today’s hyper-competitive market of technology, it is an advantage to have a competitive edge. This pushes organizations to  adopt development techniques which help to establish the practice of continuous integration and continuous delivery. Continuous Integration calls for code integration into a shared repository many times in a day, while continuous delivery enables the delivery of usable build to the user.

The main challenge for software professional is to deliver a good idea to people as soon as possible. The traditional waterfall method is inadequate to achieve this goal, hence organizations are adopting development techniques which support continuous integration and continuous delivery.  A typical development pipeline is an implementation of the building, deployment, testing and release process of an application. For achieving CI/CD, it is recommended to automate this pipeline as much as possible. If concentrated enough on this pipeline, one will realize that test automation is a vital element in the process of continuous delivery. Hence, concentrating on test automation will help to establish a result oriented environment for CI/CD.

Automated Testing forms an integral part in continuous delivery. An organization has three options in order to select the right automation tool – to build an in-house automation framework, use an open source tool or purchase a premium test automation tool. Building an in-house automation tool is a time taking affair and a software project in itself. Purchasing premium tools can be an added investment without a guarantee on its ROI. So the best economic and technical approach is to leverage an open source tool. Among them, Appium and Selenium are ideal test automation solutions for mobile applications and web application respectively.

Automated Testing is all about using a special software to control test execution and compare the actual result with the desired results. For effective results it is important to select the right tool, framework and technical approach. The best approach is to understand which tests are to be automated and which are to be performed manually. As suggested by Mike Cohn, low-level unit tests need to be automated more than higher level end-to-end GUI tests. This is because unit tests verify and validate the initial source codes. Usability tests and one-off tests should be done manually. Selecting the right automated testing framework enables extension of test automation to larger projects. Whichever framework is chosen, it is important to keep the test script separate from the framework. This makes it easier to maintain and extend them later on.

Conclusion

Organizations striving for continuous delivery need to evolve their testing practices from an existing manual system to an increasingly automated one. Past the initial hurdles, the investment in automated testing is worthwhile. An ideal test automation solution can be based on Selenium and Appium which come without the burden of maintenance and manual configuration. Organizations investing their efforts on automated testing will eventually be the ones to establish a result oriented CI/CD workflow and will finally deliver quality products into the market.

[test-automation-blog]

Test Automation and challenges in implementing it.

Test Automation is capable of boosting productivity and enhancing software quality. But this occurs only when all the key challenges of test automation are overcome. It is wrong to assume that automation will substitute poor testing practices. In fact, poor testing practices with test automation are likely to make things even worse.

Some of the most common challenges of Test Automation are as follows:

-      Choosing the right test automation tool: There can be no single solution which fits all software applications. Factors such as technical feasibility of the application, economic feasibility of the organization, user-friendliness, technical support of the tool, etc., all need to be considered thoroughly before choosing a test automation tool for a software application.

-      Investment cost: Software testing automation is a substantial investment. The benefits of this investment can be seen only after months of dedicated implementation. Many firms do not step forward with this investment as they may not be ready with such a huge investment with little idea about its ROI.

-      What is to be automated: Having a test automation tool does not imply automating each and every test script. A combined effort of manual and automated testing yields the best results. Hence, it is very important to understand what is to be automated and what is to be left for manual testing. Tests that need to be executed frequently are ideal for automation. Tests that require different data sets, greater precision and are difficult to be achieved through manual testing, can be considered for automation.  

-      A suitable automation framework: The technical feasibility, the functionalities to be tested, actions to be performed, and the validation types help to determine the type of automation framework. Some additional factors to be considered while choosing an automation framework are performance, maintainability and reliability. The framework should also provide for reliable reporting and be able to handle unpredicted events that can occur during testing.

-      Setting realistic expectations: Testing can never be considered as complete. Testing is not a replacement for manual testing, rather it needs to be used in conjunction with it. Certain tests cannot be automated at all. The real value of testing comes from the information that it provides. Repeating the tests more frequently and increasing the number of tests more than required, does not yield better results.

-      Resistance to change: The prevailing work culture of manual testing may not be open to the prospect of changing to the automated testing software. Many people may interpret is as a strategy for cost cutting and employee layoff. A mental resistance in new technology is not an ideal environment for it to succeed.

Conclusion

Test automation saves time. All challenges related to automated testing may not be mitigated, but once it is implemented, the return on investment is rapid. The benefits of test automation are numerous, but it may not be implemented based on reasons unique to each individual organization. Before implementing, it is necessary to perform a thorough research on its basic requirements and what can be done to establish it based on the available economic and technical resources.

[test-automation-blog]

The Business benefits of test automation.

Software Quality is an important aspect in the welfare of an organization. By automating necessary and repetitive testing tasks, organizations can benefit in many number of ways. Still, it is given lesser priority with minimal investment in technology, training and processes which can ensure the quality of the software.

The business benefits of  maintaining software quality throughout all the phases of application development are both deep and wide. Embarking on a program of continuous quality control  always keeps costs less than that required for fixing them later on. Deploying software test automation solutions have a significant positive effect on testing time scales.  Let us review these business benefits in detail.

-      Relieves testing bottlenecks: Testing usually takes up to 40% of the total development time. Once automated tests are set up, they take lesser time to complete than manual tests with effective results. These tests can be run unattended and hence can be executed over holidays and weekends and results can be analyzed later on. Thus, more tests are completed in lesser time.

-      Reduces overall testing costs: With more tests being covered in lesser time and effort, the costs required for testing are reduced as a result.

-      Increased code coverage: As more tests are covered in lesser time, a higher level of code coverage can be achieved than manual testing. Thus, more features can be tested and the tests conducted are more in-depth. This results in higher quality software and satisfied customers.

-      Increased credibility in the market: With satisfied customers, the goodwill of the organization spreads at a steady rate. Potential customers who depend more on the reviews and experience of their peers and friends are more likely to consider the organization for their projects. Thus, increased software quality, lesser turnaround time and satisfied customers help to retain existing customers and gain new ones.

-      Reassign resources: Automating testing helps to free up a significant amount of the testers time. Thus, they can be either reassigned to other projects or engaged into more indepth testing, which further ensures robustness of the application.

-      Application scalability of the software: For some applications, it is necessary to perform load and stress testing on them. Replicating the performance of such applications under extreme stress helps to avoid machine overloads, unnecessary infrastructure investments and implementation of enhancements.  Being able to simulate such extreme conditions is highly unlikely for any organization. Automated tests help in this regard which ensures that when he code is put live, there will be no surprises caused by the full production load.

-      Data Management and Security: Effective testing depends on the quality of test data used. Automation solutions help with creating, manipulating and protecting test database, allowing the re-use of data time and again.

-      Consistent and thorough testing: With test automation, tests can be recorded and reused. This enables each and every test to be run with consistently with equal efficiency regardless of who executed them.

Conclusion

There are numerous  benefits of test automation. But it is still not used widely. The reasons are unique to each organization. By automating the key areas of testing, organizations can benefit huge reductions, downtime and inefficiencies. This also improves, quality, and the time to market of their application development program.

[test-automation-blog]

Digital Enterprises and automated testing.

Every business environment is being significantly influenced by an unprecedented wave of innovation, an all pervasive networking, the capacity of big data, the power of advanced analytics and the future potential of the mobile technology. Digital does not affect a few selective parts of a business, rather it ripples through almost everything about the business. Harnessing this digital force requires a holistic view of the enterprise in terms of both technology and culture.

With differences blurring between physical and virtual environment, an effective business strategy is required to leverage technology and create new business models and end user experience. This blistering pace of change in the business and its technology runs significant risks and which need to be effectively managed. Test automation significantly accelerates the process of business digitization and its transformation into a digital enterprise.

First and foremost it is to be known and understood that we cannot escape testing. In the good old days, testing was squeezed to the end which did not have any positive impact on the project deliverables. Hence testing is engaged early on and over and over again. Testing a modern application is difficult than test old monolithic applications as architecturally they are more complex and it is going to be more so as time progresses.

The schedule and cost are tremendous. One solution to this is to get smarter in case of testing. Automated testing is imperative and continuous. There is never enough resource or the time to perform testing, hence tests need to be prioritized. With so many different technologies, interdependencies, integrations and the environment of operation, a solution for automation can quickly become very complex.

In today’s world, there are three main problems test automation solutions try to solve.

-      Easy adoption of test automation solution

-      Address the complexities that make it hard to test modern digital systems.

-      Extract the maximum ROI out of the test automation solution.

Digital Enterprise test Platform

An enterprise test platform supports automated testing as opposed to test automation. It helps to create automated tests using a variety of chosen tools. It helps to create a log of the tests created by all the systems under test. New tests can be created as well as reused. New tests sets which can be a combination of manual or automated tests can be defined or created. They can be run right away or advanced scheduling capabilities can be used to run at a later date. Furthermore, conditions can be defined to run those tests so they can be tied to continuous integration process practiced in the organization. Additionally, it comes with an API interface which enables testing the other tool set in the system.

Conclusion

With an all time high competition, digital enterprises work towards faster time to market and enhanced user experience. This results into higher conversion rates and a positive Net Promoter Score. These benefits can be derived through QA approaches like Continuouns Testing. The ideal way forward is however trying to automate the testing not the tests.

[test-automation-blog]

Considerations for choosing the right Test Automation Framework for multi-channel sales.

Multichannel selling refers to selling across a variety of channels – online and offline. These channels can range from shopping cart solutions, mobile app solutions, online marketplaces to brick and mortar stores.  Additional multi channels include text messaging,emails, social media and GPS to sending out mobile alerts on customer proximity.  This allows a retailer to reach out to a wide base of customers.

Multi channel usage is changing the way of business and quality assurance. The major sellers run under enormous pressure to stay ahead in the competition. This puts an ever increasing pressure on the technology providers to get the applications into the market faster with quality. Here, the companies require a full scale implementation of test automation framework  which utilizes the most suitable test automation tools for the efficient management and maintenance of the quality of the application. With multi-channels involved, they need to move away from traditional testing and use an integrated testing approach in order to reduce costs and maximize productivity and profitability.

The decision on the right automation framework can be made by reviewing the following five aspects.

-      Multi-channel testing capability: The test automation framework should be able to facilitate seamless automation and test the web, mobile and desktop applications. Automation of end-to-end solution is necessary to test the full functionality and this needs to be supported by the automation framework.

-      Multi-layered testing capability: The automation framework should have the capability to verify functionalities and data flows at various layers of the architecture. This enables the project team to isolate issues and address them. Common middleware and back-end services should be identifiable and efficient, reusable tests should be supported.

-      Modular design:  The framework should have a modular architecture in order to minimize interdependencies and segregate functionalities. Module based and library architecture based test automation frameworks support high level of modularization in such a way that changes in one test script do not cause changes in another test script. This makes the test structure more robust and maintainable.

-      Integrated solution: It is very important for test solutions are integrable with other activities or components in the Software Development Life Cycle. Such components may be test data generators, log analyzers, crawlers, reusable components for browsers, mobiles, social channels, UIs, etc. This will help to leverage the maximum benefits from the implementation test automation framework. The framework design should be agnostic to the tools that are used to automate the testing and the other related activities in the SDLC.

-      Reporting Dashboard

A comprehensive reporting dashboard helps to provide a complete overview of the test execution and their status reports. The automation framework should have a reporting dashboard which can function as a point of reference for the process.

Conclusion

The process of reporting and testing helps to increase sales and business, but even when the smartest systems are in place, it should be remembered that what works for one channel may not work for another.  Each channel performs differently, hence our focus should be testing the key elements and avoid testing them multiple times all at once.

[test-automation-blog]

Choosing the right Test Automation Framework for Banking Applications

A Test Automation Framework is a set of guidelines or standards which help to produce beneficiary outcomes while test automation. The benefits can be increased code reuse, reduced maintenance cost, higher portability, etc. It is to be noted that they are just guidelines and not mandatory rules. However, following these rules gain certain advantages in the long run.

There are various types of Automation frameworks which are available. They are:

-      Module Based Testing Framework: This is based on the concept of ‘Abstraction’. It reduces the entire AUT functions into their atomic forms, each having a dedicated test script. All these test scripts taken together build a larger test script representing a larger module. These modules are separated in such  way that changes made in one section do not affect the other section.

-      Library Architecture Testing Framework: It can be considered as an extension of module based testing framework. Here, the functions of the AUT are segregated into common functions and a common library is created for them. These libraries can then be called for test scripts whenever required.

-      Data-Driven Testing Framework: In both the above frameworks, test data is lodged in the test scripts, thus changes in the test data require changes in the test script as well. But for testing an application, we need to test a functionality, multiple times with different sets of input data. Data driven testing helps to segregate the test script logic and the test data   storing it in an external database. These data sets can be used to populate the data within the test scripts.

-      Keyword-Driven or Table-Driven Testing Framework: It is an extension of the data driven testing as it segregates test data from test scripts. Additionally, it also segregates the certain set of codes, known as keywords, in the test script into an external data file. These keywords serve as guides as to what action needs to be performed on the application.

-      Behavior Driven Testing Framework: This framework allows the automation of functional validations in an easily perceivable format for the developers and testers. Such frameworks do not need the user to be acquainted with programming language.

-      Hybrid test Automation Framework: As suggested by its name, it is a combination of more than one testing frameworks. This helps to leverage the benefits of all the frameworks.

For financial applications, the process of test automation can be mainly planned as follows:

-      Model all the system interactions.

-      Break large functions into smaller ones

-      Identify the input data

-      Group together common test cases in a library and reused when required.      

The above plan calls for modular, library, data-driven and keyword driven frameworks. Hence a hybrid testing framework can be selected for  leveraging the benefits of each in automated testing of the banking application. Also, Open source automation software has emerged over the years as an adaptable and robust solution which can be extended as per project specifications. Considerations can be made in this regard as well after a thorough cost-benefit analysis.

Conclusion

While there are many advantages of

test automation frameworks

,  the most valuable of them are ease of scripting, scalability, modularity, process definition, low maintenance and reusability. Ideally more than one frameworks need to be implemented in order to maximize the benefits. This does not imply applying all frameworks on all modules. It is more about choosing the ideal framework as per requirement and applying it to that module.

[test-automation-blog]

Main Challenges in Automated testing of Digital App.

A financial or a banking application facilitates the management of business processes, automates, assists and stores financial information. It can be executed as a standalone software or as a part of a financial information system.

  The main characteristics of a typical banking application are as follows:

  -      A multi-tier functionality to support thousands of concurrent sessions

-      Large scale integration

-      Integration with other applications such as Bill pay, trading accounts, and other real time and batch processing with high rate of transactions.

  Testing banking application requires an end to end testing solution involving multiple testing techniques, mainly to ensure the efficiency following characteristics-

  -      Full coverage of banking and business workflows

-      Functional aspects of the application

-      Application Security

-      Data Security

-      Concurrency management

-      User Experience

-      Application performance

  Automated testing tools that are most effective in testing the system are most often limited while testing the complete system. The limitations are a result of the following situations:

  -      Establishing shared services that are used by many business groups and systems.

-      Use of workflow technologies where the business logic is spread over a set of systems.

-      Increased implementation of services and process management techniques which focus on reusing of business components.

-      Increased integration of packaged solutions.

    Challenges of Testing Digital Banking Apps

  Testing the above inter dependencies are very critical for core applications. The banking industry interfaces with a huge volume of  customers and transactions. Additionally, there are multiple implementations across geographic regions. Changes in such systems need to be rigorously tested in orders to avoid downstream inconsistencies that could eventually lead to serious consequences.

  Regarding regression testing, it is required to focus that the original functionality is not disturbed due to the introduction of new functionality or changes made in the system. Regression test needs to be performed in isolation assuming the other interconnected systems are not undergoing changes. As the consequences of the errors are high, hence the end to end testing of such systems are very costly. As such the tests cases are limited to basic conditions only.

  Banking applications automate a number of processes and handle multiple product lines, multiple flow paths and with a vast amount of real time information. As a result, these systems are not  suitable for  the traditional test automation. Here, the core application logic needs to be tested for which record/playback feature is not enough. The record/playback feature offers only a stop gap solution and creating all the scripts for the application will only make it difficult to maintain in the face of changes to the application.

  Using the concepts of functional decomposition and test asset reuse can significantly solve these problems. It involves reducing all the functions and process to their lowest unit of work, each defined by its set of parameters and attributes. Test cases can then be created using these atomic processes. Errors made in this situation result in inaccurate execution of the test cases. Also, the test data needs to be separated from the test case in order to ensure test reuse.

  For a rigorous automated testing, the QA team needs to have exhaustive and comprehensive test cases. This requires considering all external inputs as independent parameters to generate test cases. In practice, exploring all combinations becomes impractical. This leads the QA team to define a set of cases which do not account for all input variations. A pairwise coverage is then implemented which is considered to be a more thorough approach to cover all critical cases, while keeping the number of test cases under control. For areas which are not covered by the pairwise approach, test cases need to be created manually.

  Conclusion

  Test automation has always been considered as a means to achieve the desired amount of consistency and coverage in the testing process. Clearly defining the boundaries of the system that needs to be tested and optimizing the test  cases to a manageable number can provide significant benefits in carrying out the testing process more effectively.

[test-automation-blog]

Automated Testing tools for testing financial or banking apps

With the adoption of digital banking, there has been a significant rise in the number of services through digital applications. Now mobile applications enable users to scan checks, submit loan applications, buy investments, etc. Not only that, with embedded ATMs such applications have become even more sophisticated. With all these attractive features, the complexity of financial applications has grown, so also has the need and complexity of testing these financial apps. Software failure in this regard can have devastating effects. The risk of software failures is also liable to rise with frequent software updates.

There are a number of factors which push for the automated testing of financial applications. Automation is more used for performance testing while functional testing are kept for manual testing. But considering automated functional testing for banking applications can reap rich benefits. In an environment of mobile apps and software solutions, automated functional regression testing boosts speed and efficiency and reduces the potential for software defects

Automated testing support the trend of Agile development. Hence, most financial institutions push for automated testing to reduce test costs over time and improve their capability of continuous delivery. As banks and financial institutions plan to overhaul their systems  with new technology, they also run on a deadline to release their products into the market.

It is a proven fact that without a good test data management, application quality is often reduced with defects being revealed only in production. Being heavily dependant on legacy mainframe systems, banks face challenges with data testing, profiling, data masking and sub-setting. It is here that the Datamaker tool helps by creating synthetic data. The Datamaker tool helps to facilitate legal compliance by ensuring the data tested is good and correct. It also helps in automating and improving the data provisioning process. By providing test data warehouse and test data on demand, it improves the time to release/production of the application and ensures that once the applications are made live, they are fit for purpose and designed with the users in mind.

IBM AppScan is an automated tool for security testing, which scans the application prior to deployment. It helps to identify security vulnerabilities, generate reports and fix recommendations. Another such automated tool for security testing of banking applications is HPE WebInspect. It provides a dynamic application security testing, which enables delivery of timely security intelligence across the enterprise irrespective of the size. Its distributed architecture, remote sensor technology and capabilities of scheduling and controlling supports the simultaneous scanning of many applications. It also lets organizations manage their application assessments in a central location. It helps to monitor critical metrics, progress and trends across large scale security testing programs. The HPE Software Security Center helps to build the process and utilize the information from different scanning technologies representing the truest picture of the probable organizational risks.

[test-automation-blog]

Understanding ATDD, GUI Automation and Exploratory Testing

A project can leverage ATDD(Acceptance Test Driven Development), GUI Automation and Exploratory Testing to be able to successfully learn, automate the application development.

Acceptance Test Driven Development is a continuous process where the development team discusses the acceptance criteria collaboratively and forms a set of test scripts before the development initiates. It is a core practice of Agile. Here, a programmer creates a test first which focus on the software expectations.  Once the test fails, the developer proceeds to create a function that will pass the defined test. This approach is called refactoring.  It makes sure that the functions and procedures satisfy the functionality needs and the code developed has a minimum level of replication and duplication. ATDD is about building the code correctly. It places the focus on the desired functionalities from the user’s perspective. It uses tools that enable a more natural language to present the requirements. They look like manual test scripts that map the underlying automatic code. However, it  cannot ensure that the functionality is helpful or user friendly.

GUI Automation

Most people react with computers and mobile device through the Graphical User Interface(GUI). Majority of the web and mobile activities are performed through the Graphical User Interface. The GUI Automation is used to simulate the user activities. This can be a light weight form filling tool or a full web aplication automation tool. GUI Automation is most of the time used to create acceptance test cases. It is extremely useful while dealing with repetitive steps. It forms an important part of acceptance test verification to ensure that the workflow behaves appropriately. The GUI automation acts as a human proxy allowing the tester to examine user interactions directly. However, it tends to be the most fragile layer of the application.

Exploratory Testing

Exploratory Testing can be referred to as a real-time scientific thinking. It is a process of simultaneous test design and test execution. In simple words, the tests that are already performed, inform the tests that will be performed next. It is not like scripted tests. They are not pre-defined and are not pre planned. It allows the tester to choose a sequence of steps at a particular time and context and get answers about the state or condition of the product. Exploratory testing allows a very dynamic way of evaluating an application. It is at the core of ATDD.

Creating balance-

ATDD, GUI Automation and Exploratory testing are all interdependent. ATDD and GUI Automation are by nature exploratory endeavors as they define the acceptance criteria that drive development. Some of the ways to creating balance between ATDD, GUI automation and Exploratory Testing are as follows:

- Focus on delivering business value: Here understanding the acceptance criteria is the most important aspect. Understanding the acceptance criteria guides in the testing approaches and the development approaches used.

- Automation is not required to be permanent: Automated tests help early on but eventually, after several integration and regression tests, they do not help anymore. Hence, all automations should not be taken as permanent. Consider provisions for throwaway automations for exploring different avenues.

- Exploration is a mindset: Exploratory testing is a manual process, in which a tester navigates through the system based on the result of a previous test. It should not be taken as a testing technique, rather it should be considered as a way of thinking.

- Use personas to visualize goals: This is one of the most effective tools for an application exploration. It implies that understanding the interests and behavior of different user groups  helps in getting an idea  about the preferences of each type of user.

- Use session based techniques: a 100% regression testing cannot be achieved in one day. It is a time taking procedure. Hence developing charters, executing them over a period of time helps focus the test efforts and allows diverse ways of examining an application. Focusing on smaller subsystems, through specific persona, within a specific context helps   work on the most important aspect and deliver the best business value.

Conclusion:

The avenues of explorations are many along with possible queries and imaginations of the team members. Quality now is no longer the final task of a group of testers, rather it is an integrated series of processes practiced by the entire team. Exploration happens at all levels of ATDD, GUI Automation and Exploratory testing and it is not a methodology, but a mindset.

[test-automation-blog]

Software efficiency through Tests Automation

Due to the growing complexity and shorter time to market, testing forms is an important part of the software development life cycle. Automated testing provides many benefits to the developers. Testing and release of software applications involves many routine processes, repeated operations such as obtaining the latest version of the code, compile and run different test cases and build binary files. These steps are generally the same for all types of applications with a little variation considering the development environment and the system configuration. Automating these routines in testing reduces the software engineering costs. Additionally, not only does it reduce manpower, but also improves the hardware efficiency. Automated Testing is the best way to increase the efficiency, effectiveness and coverage of the software testing. The true benefit from automation comes from reusable tests as well as those which could not be performed manually. Depending on the project size, test automation will provide a return on investment(ROI).

How big the ROI actually depends on how the test automation is adopted and executed.

Every time the source code is  modified, software tests need to be repeated.  For each release, it needs to be tested on all variants of the hardware and operating system. Test automation is an easy process when adopted from the beginning. Automated software tests reduce the testing time from weeks to hours.  This significant time saving results in significant cost saving. Futhermore, determining the number of relevant test cases along with its releases, also indicate how well  the test automation is going to work for the project. It also reduces the likelihood of failures and errors caused by human mistakes.

Automated Testing Frameworks provide a continuous integration system, thus, making it easier for the developers to integrate changes to the project and obtain a fresh build. This increases productivity. It also  monitors the execution of externally run cron jobs and notifies when something is wrong.

Automated Testing increases the scope of the tests and helps to improve the software quality. It can easily execute thousands of complex test cases providing a code coverage which is not possible with manual testing. Use of open, transparent API’s based test cases help in reusing of test cases later on.

A professional automated software testing helps produce a high quality, reliable and robust software despite the ever growing technological complexity. Apart from detecting bugs, it increases the test efficiency, decreasing the software development time and cost. Also, it increases the test effectiveness, giving out a better quality product, thus, ensuring customer satisfaction and loyalty, and it shortens the time to market yielding maximum benefits.

Conclusion:

Software nowadays no longer stays in complete isolation. It now has to be flexible and integrable with a wide range of applications. Automated Testing makes it easier to track down defects. It helps to break down waterfall’s coding-testing paradigm and thus in a way lays down the base for Agile practices.

[test-automation-blog]

An overview of Selenium Open Source Test Automation

Testing  has eventually become an important phase in a software development life cycle. Hence the cost of testing has also become an important factor for any project. Thus, organizations have started implementing open source automated testing tools instead of premium commercial testing tools. The greatest advantage of open source tools is that it does not require any licensing costs.

Open source testing tools can be implemented for all test types such as functional, performance, compatibility, user acceptance, etc.  For other testing types such as white box, black box, unit testing, there are one or the other type of testing tools.

Selenium is one of the best options for an open source automated testing tool. It can also be used as a Unit Testing tool for Java Script.

Selenium has three modes of executing test cases: Selenium IDE – Record and Playback – It records the test cases in Firefox. Individual tests can be written and save either individually or as a group. These test cases can be exported into other programming languages and run on different testing frameworks. The recorded tests can also be played back at a later time to check for the validations.

Selenium RC – Remote Control Mode – Here, Selenium starts with multiple browsers. Tests can be run in a preferred language. This enhances the test cases and covers the required test scenarios.

Test Runner Mode – In this mode, the test cases are recorded and played back in the form of HTML tables. The most commonly used commands in Selenium IDE: open: Opens a page through a URL.

clickAndWait: A click operation is performed and  waits for a new page to load.

verifyTitle: Verifies title and continues to run if it fails.

assertTitle: Verifies an expected page title and execution stops if it fails.

verifyElementPresent: Verifies an UI element, defined by its HTML tag.

verifyTextPresent: Locates that the expected text  somewhere on the page.

verifyText: Verifies that the text and its corresponding HTML tag are present on the page.

waitForPageToLoad: Pauses execution until an expected new page loads.

waitForElementPresent: Mainly used with AJAX calls, it pauses execution until an expected UI element, as defined by its HTML tag, is present on the page. Writing a Test Suite:

It is mainly a collection of test cases. Pre existing test cases can be loaded by going to File -> Add Test Case menu option. Pre existing test cases can be added to a new suite.  A test suit is basically an HTML file of a one column table.  Each cell in the table contains a link to a test case.

Execution of Selenium IDE Tests

Tests that have been developed using Selenium IDE can be run on other browsers as well. This can be done using the command line interface which invokes the  Selenium RC server.

Conclusion

Selenium is comprised of 4 components – Selenium IDE, Selenium RC, Webdriver and Grids. Because of their differences in architecture, they support different browsers.Hence, it is important to understand the concept of each to choose the right Selenium tool for your project.

[test-automation-blog]

Facts to consider before Automating Testing

Certain factors should be taken into consideration while automating test scripts. Automation of tests is generally done to accelerate the process of testing and meet tighter deadlines. Proper implementation of automated tests results into an effective Quality Assurance(QA).

The need for automation

The need for automation comes when all the modules of a release need to be tested for initial bugs and further bugs due to fixes made. Each time a fix is made, the entire release needs to be tested. Hence, automated testing is a viable option when a significant level of repeated regression testing is required.

A web application may have thousands of users interacting simultaneously. Creation of such large number of users manually is an inefficient process of testing. This process can be carried out by automating load testing, which will work by creating virtual users for checking the load capacity of the application.

Automated testing can be implemented for reviewing repeated functional changes in the same Graphical User Interface(GUI).

Facts to Consider

1.    Availability of skilled resources

Automation will need skilled resources having sufficient knowledge of programming or those who have a technical background with adaptability to new technologies.  If such resources are not available, then the shift to automated testing is not suggested as handing over expensive tools to unskilled professionals will not generate the expected results.

2.    Requirement of initial investment

Like manual testing, automated testing also has a     significant investment cost which is high at the initial level. The cost for purchasing automation tools, training and maintenance of the test scripts is very high. Its return on investment can only be realized only after a significant period of time through optimum utilization of the tools.

3.    User interface automation

User Interface should not be automated if it is prone to frequent changes. This is because with frequent changes in the UI, the associated test scripts will also need to be changed which will constantly require additional maintenance cost.

4.    Application stability for supporting automation

Testing automation is to be implemented in environments that can support automation. The agile environment is best suited for automation testing. Other environments will call for extra investments to adapt to such new changes.

5.    Achieving 100% automation

It is may not be economically and technically feasible to achieve a complete automated implementation. Certain areas of testing such as regression testing, load testing etc., will need to be automated extensively, while areas of UI testing, usability and compatibility can be restricted to manual testing only.

6.    Exclude rare tests from automation

Test cases which are run only once or rarely can be excluded for automation.

7.    Automation lifetime

The automated tests should be used repeatedly to realize the cost incurred for its implementation. Roughly, an automated script suite should be used at least 15 – 20 times to test separate releases, to generate a standard Return on Investment.

Conclusion:

Automation of test cases is an effective way of accomplishing testing objectives. A proper research and analysis should be carried out before implementation of automated testing. A complete match to the requirements is unlikely to be available at any time. Hence, instead of depending 100% on either manual or automated testing, the best approach is to proceed with a balanced combination of both. However, automated testing is not the complete solution. Real time testers should also be used to along with automation.

[test-automation-blog]

Importance of Automated Testing in Quality Assurance Activities

To grow and compete in today’s dynamic business scenario, you have to upgrade continuously and respond rapidly to new scopes and challenges. This may need to augment the business apps you have or implement and develop new applications. For quality implementation with least IT and business risk, you have to test changes to your IT setting before you start production.

As several testing activities require manual labor, quality assurance teams are under tremendous pressure to keep costs minimal and complete the procedures within short time frames, especially when they deal with heterogeneous system scenarios. To meet the deadline and stay within budget, the teams are forced to skip or delay critical tests. By using conventional record- and –play tools to perform automate testing activities, organizations often find it hard to maintain the crucial scripts. In the interim, expert quality engineers are required to build and update the tests. The outcomes are risky testing procedures and higher expenses. Automated testing helps organizations to ensure proficient testing and optimal app performance while lowering business risks related to outages and slow performance.    

      Automation system development procedures

Industrial automation systems for manufacturing, power plant, etc. comprise of distributed software components to regulate the system behavior. The escalating complexity of software products need well-defined methods and processes for system construction, verification and validation. Several system engineering procedures and softwares support engineers and automation testers by providing a series of steps for planning and control. Nonetheless, the standard of the processes focuses upon the structure of software systems with restrictions on method support, synchronizing and tooling different heterogeneous disciplines. Semantic and technical integration of tools and data across disciplines facilitates frequent data exchange and synchronization, supports resourceful change management procedures, and enables more efficient QA. Besides, processes across tool borders and disciplines become noticeable and, as a result, facilitate effective project monitoring, management and control.    

Improving test management

To perk up testing efficiency and help ensure high quality, your approach must be standardized for quality management. Your quality assurance team  requires technology, tools and best practices that will assist them organize and automate their activities anywhere, store and track outcomes on one platform where collected testing acquaintance is within the reach of everyone. Through test automation you can make better use of your resources. Professional testers are freed from giving more effort to design and perform superior tests while idle machines can be used to run unattended automated tests.

Optimize teamwork and productivity

Accelerate cross – team exchange and transparency through real- time collaboration. You can optimize scheduling, prioritization and releasing through automated planning and tracking. Quality management solution also facilitates easy access and synchronicity through browser-based access, leverage real-time reports for superior decision making, streamline risky procedures, validation and verification.It also lowers operating costs  and develop overall application life-cycle development. Dynamic workflows can get real-time visibility. Changes in requirement, source control and notifications help teams to fine-tune faster, fix problems earlier, respond in context speedily and deliver high quality.  

  Lowering business risk

Like several other organizations, you may be unwilling to change your production systems because you are deficit of visibility into the effect of these changes in significant business processes. Besides, most likely you do not have time to test every feature of your applications before setting out the changes. Ideally, you would like to test only those scenarios and business processes that are influenced by a specific change. However, performing such analysis often involves an erroneous and lengthy process of exporting data into an unconnected evaluation tool and reviewing multiple transactions supported by software. Therefore, counting on a professional firm can help you make out the impact of upgrades or new support systems and prioritize the automated test scope depending on business risk.

This helps you to plan testing more successfully and pay attention to those places affected by the altered or new applications. By recognizing the business risk of your strategized changes, you can align your business with your IT organizations more efficiently.