Traditional testing, which is focused on technical, business and functional requirements to test all possible scenarios, is simply not enough anymore. It misses out on a key critical factor: how the actual user or customer will utilize the system and what s/he would want from it. With so many options available at the click of a mouse, it is important to ensure the application meets users’ demands for ease of use and functionality.
Considering the sheer volume of features and functionalities in an application, a mechanism is required to prioritize functionality testing based on its criticality from the users’ perspective as well as potentially weak/vulnerable features which need additional focus. After all, there are still time-to-market and update deadlines to meet.
It is important to focus, from the testing perspective, on enhancing the customer experience by being more customer centric. Furthermore, is important to be aware of customer behavior patterns and customer responses to different scenarios. Based on this information, a testing strategy can then be formed. Most importantly, it should be done in a continuous fashion during development, vs. waiting until development is completed right before release for optimal real-time feedback.
How can testing be optimized to ensure a better customer experience? Predictive analytics can help.
Predictive analytics can fill the gaps in testing
With advent of Big Data, Artificial Intelligence (AI) and Machine Learning (ML), missed aspects during testing can be revealed through predictive analytics. Predictive analytics incorporates several techniques from data mining, statistical modeling, artificial intelligence and machine learning to analyze both historic and current data to accurately predict possible future patterns, trends and user behavior.
Customer-centric approach to testing for improved customer satisfaction
Based on customer usage data, predictive analytics can help uncover customer usage patterns, which can then lead to creation of test cases for critical process flows with high importance functionality. This gives powerful insight into customer sentiments, which can drive more customer-centric testing. Take, for example, a scenario for mobile wallets – a convenient way for a user to make in-store payments via their devices. Typically, a test suite is built based on the stories written by the product team. Speed and ease of use are critical properties of mobile wallets from the user’s perspective. An important user requirement is to be able to easily locate necessary functions and switch seamlessly from one function to the other in the mobile pay application. Certain users may prioritize the need to easily access usage history in their mobile wallets, which could be effortlessly checked if it appears on the initial page of the mobile service. The test suite should include these type of user experiences when creating test cases, flows, views, and sequences.
Reduction in overall cost from early detection of critical defects
Early identification of defects helps improve quality in less time and cost. Based on past production failures, issues or defects data, analytics can predict future failures. These potential defect areas can then be used to guide testing focus and prioritization.
Let’s look at an example of a finance department’s payment function. A finance department decided that doing remittance receipt reconciliation manually was more time-consuming and costly as compared to using a proposed software platform. The department opted to move to the new software platform. A key goal for the transition was error-free reconciliation of receipts. A typical testing approach would have been to check whether all receipt formats could be read, and data retrieved and processed as per the current manual process.
Predictive analytics made a large difference when building their testing strategy. Based on past reconciliation errors, issues faced by those handling the function, along with issues faced by other companies using the same software for reconciliation, the team uncovered certain processes with high errors/failures that even resulted in system crashes at times. The testing team then focused on those areas first, resulting in speedy and timely execution.
Even though the first instance of data extraction was successful, validation at further stages was important for timely correction (e.g. digital separator, values post digital separator or placement of commas etc.). The system didn’t show any errors in the first stage of the process but then needed correction at a later stage. Simple point-to-point testing would not have identified this issue. If this critical testing at every stage had not been completed initially, the whole process would have been prolonged, without any insight into why and where there were breakdowns in the process. Testing costs would have been higher, subsequently increasing overall project cost.
Optimized regression test suite
Every time a testing cycle is run, logs are created, and observations are noted to highlight various defects. These defects give insight into how it will impact the user experience. By identifying these issue patterns using predictive analytics / machine learning algorithms, predictions can be made for possible future failures and an optimized regression suite can be formed.
For example, a product created by a very talented team of developers had challenges in every single release cycle. Each new update would get pushed out due to defects that the business needed fixed before the release was made available. Based on an analysis of the data, it was found that the development team was brilliant in building clean flows for the functionality listed in requirements, but they almost always missed the “what ifs”. The “what ifs” were critical enough to call for fixes before the new release. The project manager changed their test strategy to first discover the “what ifs” with typical failure scenarios and then performed the remainder of the testing. These scenarios were then added to the regression suite, which reduced the regression suite size as well as regression cycle time considerably.
Increased testing efficiency for better quality assurance and faster time-to-market
Rather than focusing on business or functional requirements, focusing on actual customer responses and usage makes testing more effective and efficient, which reduces cost and test cycle time while increasing quality. Further, prioritized testing helps teams push faster releases of new critical functionalities. With predicted area of defects, defect identification and resolution are faster as well. This greatly speeds up time-to-market.
Controlling release dates and their impact
Monitoring timelines and using predictive modeling to determine delays and their potential impact is critical. By identifying any issues and their reasons, a proper course correction or optimization can be done to ensure the release schedule remains intact.
For example, for an insurance life cycle product, there was a lot at stake to launch the product quickly, to retain their competitive edge in the market. The product team incorporated predictive analytics into the product management strategy. Based on data related to unit and integration testing during early stages, the team was in a good position to predict the overall completion date that was still quite a bit away from the target date. When predictive analytics was then incorporated, the team took the product features most preferred by users and then planned the go-live accordingly. The business team was able to then launch the product on time with a limited set of features, based solely on customer preferences, which resulted in high customer satisfaction.
Benefits of predictive analytics in software testing
Final thoughts
Utilization of predictive analytics doesn’t stop at release but continues as an ongoing activity. Collecting real-time feedback from customers throughout the lifetime of the product usage will only help further enhance the customer experience in future releases of new product functionality. Testing is all about exceeding user demand for a powerful experience with speed and reliability while optimizing both cost and time. And that’s exactly what predictive analytics is helping achieve today.
Contact us to discuss how you can deploy predictive analytics for improving software testing.
