Devops testing

[devops-testing-blog]

Quality Assurance in DevOps.

DevOps can be considered as the next evolution of Agile. This organizational method is being adopted by many companies in order to facilitate automated software development. DevOps provides a structure that enables continuous delivery and continuous deployment of a software application. Quality can be said to be built into DevOps. Inability to provide high quality software implies the failure in DevOps implementation.

In DevOps, infrastructure is the code. Traditionally, there was a clear difference between development and operation. A product was developed, deployed into the testing environment, if a problem was found, it was fixed else it was deployed in the production system. But as the line blurs between development and operation, as suggested by the very name ‘DevOps’, here the development environment itself becomes a part of the product.

Role of QA in DevOps

Quality Assurance in DevOps is about preventing defects, not detecting them. QA in DevOps has the visibility and directive for pushing code out when it is working and rolling it back when it is not working. QA teams here are assigned with a task of preventing defects from occurring at all.

-      QA in DevOps identifies the defect in the product as well as the development process and accordingly recommend changes wherever required.

-      QA influences both the processes of development and operations in order to maintain quality. It tries to improve predictability and repeatability.

Test automation is clearly more preferable over manual processes in DevOps. But there are certain challenges in automated testing, which need to be overcome.

-      Tests that are built over commercial tools slow down over time and face scalability challenges, more often from licensing issues. Migrating tests into more open source tools can enhance the effectiveness of a DevOps practice.

-      The  business perspective over regression issues influences the development teams to build long-running tests. Hence, subsets of total regression tests need to be carved out in order to accelerate test runs.

DevOps puts into practice the theory of Continuous Testing. Its core philosophy is to test every change made to the application as soon as possible. Non-functional tests such as performance testing and security testing also need to be included in this process. In order to effectively use all the available resources and be able to run load tests on demand without wasting time and resources, building a cloud infrastructure is a good option.

Conclusion

DevOps practices a very powerful set of principles which improves the communication between the development and the operational team. It helps them to share their knowledge, experience and skills and deliver quality applications ensuring success. In DevOps, testing and quality assurance are addressed throughout the entire lifecycle of the software application. Here, quality is built from the beginning.

[devops-testing-blog]

Getting started with continuous testing in DevOps.

With the development and operations team working together, the question is where the testing team fits in DevOps. Although it is not being involved in the definition or conversation, testing is the most important aspect in the transition to DevOps.

The testing team is more aware of the flow of information from development to production. They are responsible for the process of moving the application from development environments, testing and delivering them into production. They understand the infrastructure, its working and what requires to be done to get them deployed. Hence, DevOps advocates on engaging testing early and often.

One of the main reasons for the emergence of DevOps is to overcome the communication gap and poor collaboration resulting from a psychological divide between the development, operations and the testing teams. Practicing continuous testing is the first step in a DevOps journey. Continuous Testing(CT) sets into motion, a mechanism of continuous feedback, that drives the software delivery life cycle forward. If the feedback requires to stop, then the involved processes are halted immediately and corrective measures are taken.

Establishing an environment for CT involves the following steps:

-      Establish a single authentic source: This can be done by shifting the automation test scripts to an enterprise version control tool. The automated code base should be stored in a version control repository.

-      Enable a centralized execution and reporting process by integrating the automation suite with the deployment tool.

-      Enable a process of faster feedback by classifying the automation suite in multiple type of tests. All these tests will be run as a part of the build deployment process. In case any of them fail, the deployment process is halted and corrective measures are taken. These tests are of the following types:

-      Health check

-      Smoke Test

-      Full Regression

-      Intelligent Regression

Automated testing is more efficient than manual testing. But it faces the following challenges

-      Test built on commercial tools are limited by license constraints. These test face scalability challenges and slow down over time.

-      Different tools are required for automating different artifacts of the application.

-      Regression tests require building long running tests.

Testing teams play an important role in overcoming these challenges. They orchestrate an infrastructure for continuous testing and create a loop for early feedback with the development and operations. By implementing continuous testing, build deployments an be automated which further helps to achieve a full cycle of continuous integration, continuous delivery and continuous deployment. Introducing intelligence to the automated tests and migrating them to a more open source tool further  enhances the effectiveness of DevOps. An intelligent test automation framework can turn test scenarios on and off based on the changes made in the application.

Conclusion

Once the testing team in involved, they get embedded into the Development and Operations. This prepares an environment of experimentation for continuous quality testing. This finally leads to a combined focus on the development process  with continuous testing for continuous delivery.

[devops-testing-blog]

Ensuring continuous delivery with DevOps

DevOps focuses on development, operations and testing, all working in close proximity to accomplish rapid, iterative development and deployment.

‘Continuous’ is a word which is associated with every phase of DevOps. That can be Continuous Integration, Continuous Deployment, Continuous Testing or Continuous Delivery.

Continuous delivery aims at automating the full software delivery life cycle until the last preproduction phase, so that the software can be deployed into production any time. Continuous delivery does not mean every new feature is deployed as soon as possible, rather it means that every new feature is deployable at any time. Continuous delivery is mandatory while continuous deployment is optional.

Focusing on continuous delivery increases the capacity of releasing changes.

The concept of continuous delivery has got a lot of focus nowadays. A number of factors contribute to this.

- The increasing business pressure for faster delivery of new functionalities.

- Huge advances in the field of automated tools such as data center automation, test automation, cloud and virtualization and database management.

- The possibility of applying continuous delivery techniques to large organizations.

DevOps engineers perform multiple functions in supporting software development and quality assurance. They configure test server, build servers for supporting continuous integration and automate code merges, write custom scripts etc.

Implementing DevOps successfully brings an organization one step closer to continuous delivery. It is the idea that as soon as a feature is ready, it can be deployed into production. Some of the attributes of continuous delivery are as follows:

- Releasing work rapidly allows to gather feedback early as well. This to a great extent, helps in improving the quality of the product.

- It helps to reduce risks. Larger deployments are more susceptible to errors and risks rather than small incremental builds. These small builds can be checked thoroughly during each release.

- New features can be released early by analyzing the responsiveness of the user.

- This form of development and delivery goes very well with the present day competition in the market.

DevOps and its inclination towards continuous delivery is shaping up the business landscape for the future.

Continuous delivery is about applying a scientific method to software development life cycle. DevOps is a journey towards better engineering and delivery principles. Continuous delivery is only a process, DevOps is the bigger cultural shift. Continuous delivery can be considered to be a reference book to be used for solving certain software delivery problems.

Conclusion

Any large organization with separately operating departments, has a tough time delivering software on time due to poor communication and an underlying cultural divide. DevOps works on this factor and advocates on working together and focusing on a goal collectively rather than optimizing individual domains.

The most important factor here is for people to care about their work and try to make the process better. The teams need to come together, evaluate hypothesis and experiments for getting better at what they have been already doing.

[devops-testing-blog]

Better competitiveness with DevOps

Research shows a link between investments in IT and organizational performance. Businesses have always tried to find the link between the value of technology and higher productivity. Only an investment in the latest technology does not make a business competitive. Any company can do it. Competitive advantage cannot be gained on technology since it keeps on advancing. Hence, technology cannot be considered as a business differentiator. The methodology of leveraging that technology is something that can really set a company apart from its competitors.

DevOps is not a technology. It is a business initiative, which when executed well greatly increases the business bottom line. It can turn an organization’s web performance into a differentiator and elevate it above the competition. By providing a more efficient user experience than others in the industry, a company can gain a greater market share over their competitors.

DevOps enables a business to make frequent, low-risk releases to production. This makes the IT departments more agile and decreases the deployment time. With frequent deployments, feedbacks received about the quality are also frequent which are responded to almost immediately. This creates an environment of continuous learning. Additionally, it also prevents problems from cascading further and consuming more development and testing time.

DevOps greatly reduces the waiting time between the development and release of new features. Thus, features and can be made available in days rather than months, which results in a faster response to business needs. This ability of continuous delivery, results into scalable systems which are also highly resilient to significant work pressures. Hence, production levels are met successfully even in the face of abnormal levels of demand.

By automating manual efforts and reducing the chances of failures, potential bottlenecks are identified. Teams are more productive and operational costs are reduced.

DevOps is not information technology. It is a practice of information technology. To drive differentiation and innovation, the DevOps team needs to integrate closely with designers. DevOps breaks down the conflicts that exist between the development and operations team. This allows all the teams to work together, delivering and operating projects more effectively.

DevOps establishes a collaborative working relationship and enforces a common set of objectives which provide value to the business. This allows to achieve Continuous Delivery through Continuous Integration and Continuous Deployment.

The business challenges existing today have taken traditional delivery approaches to a whole new level. The DevOps system delivers systems faster and reduces the risk of production level changes through automated testing and shorter development iterations. A successful DevOps implementation requires a careful management of organizational and cultural change along with acquiring new skills.

Conclusion:

Implementing a Devops culture in place and leveraging the tools required to support it, enables business to compete better in today’s cloud connected, app driven worlds. DevOps is a  methodology that breaks down the barriers between the often separated functions of development and operations. It improves the quality, security and speed of delivering new softwares. In today’s software driven landscape, in order to remain competitive, continuous and innovative releases need to be made with high quality applications satisfying the customer’s growing needs.

[devops-testing-blog]

How does DevOps impact Testing.

DevOps is a software development methodology which integrates all the functions of software development from start to finish within the same cycle. An ideal DevOps cycle starts from writing code, building and deploying the files in the QA environment, executing the test cases up to deploying the application in the production environment.

Businesses today have a strong sense of urgency.

They are now driven by design thinking along with innovative software engineering practices. Software projects today prefer to have a continuous delivery practice. DevOps is based on ‘Test First’ concept and follows a  ‘Fail Fast, Fail Often’ approach. DevOps implements testing is a continuous and a more extensive process. It allows cohesiveness among the various teams and automates everything possible. Some of the benefits of DevOps are:

- Intelligent Test Automation

- Continuous Test Augmentation

- Improved Regression

The fundamental element in DevOps is that the responsibility of attaining the highest quality is shared.

All teams are completely involved at all times and quality is executed throughout all the phases of the development. Unlike traditional development methods, here testing is not located at the end of all the processes being controlled by a separate group. In fact, it is everyone’s concern and decisions on the amount and extent of testing required is made throughout the entire development lifecycle.

The DevOps methodology of software development addresses the following challenges in testing:

- It simplifies the virtualization of the test environment, automated deployment and management of configuration along with data refresh and maintenance.

- It is centered around test first, test driven development. These concepts help in achieving improved continuous integration, automation and verification.

Changes in QA process:

- DevOps makes the QA align their efforts  to ensure that all the test cases are automated and cover near 100% of the code.

- The QA also needs to make sure that the test environments are standardized and deployments are automated.

- All the pre and post testing tasks are automated and aligned with continuous integration cycle.

DevOps merges the role of the developer, operations and the QA, either by pairing or by creating a role that merges development, testing and operations all into one job role. Theoretically, this implies that if a person writes a code script, then he requires to test as well as implement it. Not only does this save time, but also creates a support line in the long run. Each team member here contributes to the success of the customer. This gives a greater advantage of communicating with the entire team right from the start to the finish

Conclusion:

Waterfall gave way to the V model. Then came Agile and DevOps appears to be the future. DevOps is based on a automating whatever and whenever possible, but it is left to the senses of the team to decide the saturation level.

[devops-testing-blog]

Software Testing - The DevOps Strategy.

The process and purpose of software testing is always the same for all development methods. What differs is their placement and execution in the cycle. The weightage given to the different test types and the time invested in them makes all the difference in the quality of the software delivered.

DevOps is all about continuous delivery and continuous integration, which necessitates continuous testing. An ideal DevOps cycle starts from coding, deploying the binaries in the quality assurance environment, executing the test cases and deploying the unit process into production. Hence, it greatly emphasizes on automating builds and using CI(Continuous Integration) tools. It requires a very high level of coordination between the different functions of the process chain.

DevOps requires its environment to resemble the production environment at all times. This is because DevOps concentrates on the following main implementation processes:

- Continuous Integration

- Continuous Delivery

- Continuous Testing

- Continuous Monitoring

The continuous integration and  delivery process get the right code to the right environment and enable the next two processes to validate that the right application has been built which functions as specified. The developers create the code, which they forward to the other developers in the team. Along the way, unit tests would be run on their code. The code by both the teams are then integrated without changing. Once the code integration is done, unit tests are performed in the integrated code. Other tests such as White box security tests,  code performance tests are also performed.  Following this process, the work of all the teams on the project, are integrated making the service, application or the system. The point here to be noted is that in this process all the code integrates are validated at all levels without error and all the tests run without error.

Continuous Integration and Continuous Delivery both do not have any meaning without continuous testing. After validating that the entire application is free of errors, it is delivered to the QA. The role of Quality Assurance is different in DevOps-

- The QA team is required to align their efforts in the development cycle.

- They need to ensure that all the test scripts are automated and have 100% code coverage.

- They need to standardize their environments and have automated deployment on their QA boxes.

- All their testing tasks are automated and aligned with the cycle of Continuous Integration.

The main differentiator of the best practices in DevOps infrastructure, is its testing maturity. Many organizations automate their development and delivery processes, but fail at orchestrating test automations. Testing architects and testing teams play a vital role in test design, test automations and development of test cases in DevOps.

Conclusion

Thus, testing in DevOps helps to proactively find and fix errors very early in the development process in turn increasing the team productivity. It identifies and and acts on core builds and test feedback and delivers a substantially better user experience of the software application then those delivered by following other software development methodology.