DevOps is not a goal, but a never-ending process of continual improvement – Jez Humble
I am going to share five handy tips for testers in today’s DevOps world from my experience in this post.
Learn Continuous Testing
The first and foremost thing to learn about is: what is continuous testing? How do DevOps processes impact the tester’s approach for transitioning from traditional waterfall to agile testing? Before starting with continuous testing, for each sprint, we were testing incrementally and then did the regression testing before deploying to production. Up to now, our sprint cycles for some projects have been weekly or bi-weekly. But this cycle becomes too long to roll out a feature or customer request in a timely way. So, we have adopted DevOps in our process. Testing over the whole lifecycle by shifting left and shift right, i.e. testing continuously from beginning to end.
Learn about DevOps Jargon
I recommend taking a look at Lisa Crispin’s course in Test Automation U to learn basic concepts and terminology around continuous delivery and DevOps. That will help you communicate and collaborate with everyone involved in delivering your product, including operations and infrastructure specialists.
A Culture Shift from an Independent Team to Whole Team Approach
In a waterfall methodology, testing is confined to an independent verification and validation team. In agile development, it becomes a collaborative effort.
People come with baggage: preconceived notions, past experiences, prejudices. The primary responsibility is educating the team about DevOps culture.
In DevOps culture, the focus is on the product, not building fiefdoms and amassing political power. The DevOps slogan is, “Everyone involved with delivering software is responsible for delivering high-quality software.” In short, the delivery becomes a shared responsibility.
Build A Successful Test Strategy
Building the right test strategy is critical in continuous testing. When we have 500 automated tests, we can’t execute all the 500 tests for each change committed in the pipeline. It will take too much execution time. So, we have analyzed and prioritized the test cases by their core functionality as well as user input from the customer support and business teams. In the end, we have come up with 100 automated regression tests which are enough to give us confidence that the latest change hasn’t broken existing features.
Don’t Just Make The Tests Pass, Focus on Efficiency
When we started with DevOps, in our continuous pipeline, our automation tests failed in between or at the last step of the pipeline. We analyzed the test results and the error logs. Most of the errors were due to timeouts or because of the new feature or enhancement changes. We have updated our scripts periodically to take care of this. After deployment, when we cross verified the product/project using manual exploratory testing, we saw a few more failures. Then we did a brainstorming session with our entire product team to make our automated tests more efficient. For this, we have created/updated the automation scripts in the preproduction phase to save time and resources.
Choose Your Tests Wisely
This is related to the points discussed in the previous paragraph. When we started doing the deployment through the DevOps practices, as testers, we needed to choose our tests wisely to add those to the pipeline. Along with the regression test suites, 24/7 monitoring should be added. This helps us to understand how the product is to be used by the customers. For choosing the tests, we use input from the user behaviors that we learn from our monitoring tools, as well as discussions with the business stakeholders.
We hope the information from this experience helps you to enhance your testing!
References:
Building a DevOps Culture – Mandi Walls
https://continuousdelivery.com/foundations/test-automation/
https://testautomationu.applitools.com/the-whole-team-approach-to-continuous-testing
https://www.mabl.com/blog/what-does-continuous-testing-actually-mean
https://testautomationu.applitools.com/test-automation-in-DevOps/