Test automation is the process of running test automatically, managing tests data and using the results to improve the quality of software. It is primarily a measure for quality assurance and it requires the commitment of the entire software development team. Many people know the fact that we cannot automate all the things.
But a lot of managers want to automate all the tests or things. They are not crazy in this matter and they have noble intentions. They want this to increase the speed of work and eliminate regression testing.
Issues With Test Automation
We cannot automate all the tests as there are certain issues with automation. These problems are described below:
- Automation requires maintenance
- Lots of automation requires lots of time
- Redundant automation is noisy
So due to these reasons, we cannot automate all the things. We should automate only right things. But the problem is to determine the right things that require automation. There is a model that describes the right things that require automation in the section below.
Model For Explaining Automation of Right Things
Automation of right things is described by Twitter Profile Scenarios. Many people nowadays use Twitter application. There are various profile features on Twitter app. This is shown in the table provided below.
|Twitter Profile Scenarios|
|1||Add a tweet|
|3||Pin a tweet|
|4||Follow a user|
|8||Block a user|
|9||View analytics of a tweet|
|10||Balloons appear on birthday|
Various features of Twitter application are described in table. Various factors corresponding to each feature are indicated in the columns. These factors include Gut Feeling (G), Risk (R), Value (V), Cost-Efficiency (C), History (H) and final Score in the last column. The factors are described below.
Gut feeling means your views about the features of the app. The feeling may vary from person to person. For example, one person may want to automate any feature of the app while the other may not want to do this. You would use “tick” mark if you want to automate any feature of the app and “cross” mark if you do not want to automate in the relevant box.
The score of Risk factor is calculated by multiplying score of Probability (frequency of use by customers) by score of Impact (if broken what’s the impact to customers?). The range of score of Probability and Impact is 1-5. Score 1 means low Impact and 5 means high Impact and same is for Frequency. So maximum score of Risk factor is 5*5=25 and minimum is 1*1=1.
The score of Value factor is calculated by multiplying score of Distinctness (does this test provide new info?) by score of Induction to Action (how quickly would this failure be fixed?). The range of score of Distinctness and Induction to Action is 1-5. So maximum score of Value factor is 5*5=25 and minimum is 1*1=1.
The score of Cost-Efficiency factor is calculated by multiplying score of Quickness (how quickly can this be scripted) by score of Ease (how easy will it be to script this). The range of score of Quickness and Ease is 1-5. So maximum score of Cost-Efficiency factor is 5*5=25 and minimum is 1*1=1.
The score of History factor is calculated by multiplying score of Similar to weak areas (volume of historical failures in related areas) by score of Frequency of breaks (volume of historical failures for this test). Similarly, maximum score of History factor is 5*5=25 and minimum is 1*1=1.
The total Score for any feature of the app is calculated adding score of all the previous factors like Risk, Value, Cost-Efficiency and History. So the maximum score can be 100 and the minimum can be 4 for any feature.
Final Criteria for Automating Any Test
This is the final criteria for automating any test:
- If the Score for test is 67-100, you will definitely automate it.
- If the score is 34-66, you will possibly automate the test. This is slightly a complex case as in this case, there are various other factors that determine the test will be automated or not. These factors are not considered in this model here.
- If the score is 0-33, do not automate it.