Ever since I have realised the importance of heuristics and oracles, I don't remember a single day I tested something without using them. Be it for critical thinking, recognising a problem, test planning or test design; there is heuristic for almost every testing problem.
I thank all those brilliant people in software testing field who have created heuristics for solving problems. Those heuristics are coming handy to many testers like myself who use them regularly.
I don't deny that using multiple heuristics and oracles to solve/recognize some problem helps; but applicability of some ideas from different heuristics is subjective to context of each testing assignment i.e. not all ideas from all heuristics/oracles will help you solve/recognize problems. What has always helped me, is 'knowing the failure patterns' of systems that I test. To this end I am saying failure patterns not specific defects only.
These failure patterns are likely to change from project to project, or from module to module within the same project, as project and product elements change with it (read HTSM by James Bach) . However, if you are well aware of failure patterns of products you test; it helps you in multiple ways such as:
1. To find obvious as well as predicted problems quickly
2. To find more problems that are likely to be created because of those obvious problems
3. To plan your testing missions
4. To strategies your testing based on foreseen/predicted risks
When it comes to learning, my experience with people and products is similar. We can't know one's nature just in few interactions, can we? Similarly with fewer tests; we can't conclude something as a consistent or prevailing failure pattern with that product.
Be it about people or products, we have to spend enough time to understand them well. If systems/products are bigger and complex (which are most of the times) finding out failure patterns with them becomes tough. What helps in this case is; one's skill to recognize. Don't you know people who are very good at 'knowing others' in very less time?
So what are those things that can help you find prevailing failure patterns in your own system? What is that oracle heuristic to create your project specific heuristic? Over the period of time, I realised that it's HEEENA that helped me create heuristic for my own project. And I hope it will help you too.
Here is how I would explain HEEENA!
H - History
Try to learn history of your product/project. If you know its history well, you stand fair chance to be able to predict its future. You can know its history by:
E - Explore
Practice focused exploratory testing, apply different testing heuristics and see if there is any failure that occurs consistently. Does that appear only in one area? Does it occur somewhere else too? Explore, explore and explore until you come to any conclusion.
So you noticed some problem? What caused it? What test did you perform? Can modifying that test uncover another problem like that? Is it happening on other machines too? How about testing with different users and user permissions? Something is wrong somewhere.... what is it and where is it? Tried focus/defocus, creep and leap or galumphing?
Make best use of your experience.
- System/Product experience
- What does your experience with your product tell you?
- What were typical failures with them?
- Do you see that happening in your product too?
E.g. Any add-ins/pre-requisites that are must for your product to work, HTML5 not supported on all web browsers, flash not supported on iOS etc.
- Do you know those common/repeated mistakes that your programmers, fellow testers or BAs do or may be their weak areas? Newly joined programmers or BAs are likely to make mistakes or they can be weak in some areas. Can you figure out which areas are likely to be affected because of those mistakes or weakness?
N- Note Taking
Make a habit of taking notes while you test. Note down everything that you observe. Not every behaviour shown by product will be catchy enough to tell you there is a problem. But your notes can help you recognize if there are any hidden patterns and failures. Remember, your notes can be your best guides if you write them well.
If you recognize a problem, try analysing it in all possible ways. Root Cause Analysis of your findings can give you more information that you can refer in future. Your analysis can become important contribution to 'history' files of your project.
So, that was HEEENA which helped me find out heuristic for my own project. Try it and let me know if it worked for you.
You can download the mindmap (in xmind format) from here. And if you see any benefits/problems with this approach; I will be happy to learn them.
This story is from Mahabharata. Pitamaha Bhishma was very angry with Arjuna because he eloped Subhadra. This was against the tradition because Subhadra’s brother Balrama had already promised for Subhadra’s marriage to Duryodhana.
Krishna tried to convince Bhishma about how Arjuna was right but Bhishma was adamant. The conversation that then happened between Krishna and Bhishma is what I want to talk about today. I will let you read their conversation first.
Bhishma: I am not happy with this marriage because this is against the tradition. A person can trust on another person only when traditions are followed and respected. Without following traditions, a trust cannot be established.
Krishna: (Giggles) Traditions…! Traditions are like mango fruit. In the beginning it tastes bitter. Some days later it becomes sour. People, who like sour taste, eat that fruit with great pleasure. After some days, mango becomes sweet and thus becomes everybody’s favorite.
But, after some more days, it gets rotten. It starts smelling bad. People fall ill who still eat that rotten mango. And as time passes on, that fruit remains of no use. Not even its pit.
I don’t have any problem with traditions as such, but when same traditions become reason for one’s exploitation, cause more harm than good and become hurdle in the change for the better, then such rotten traditions should be destroyed and new traditions should take birth.
Now you must be wondering, why am I talking about this? I read and re-read this conversation and felt that what Krishna is talking about traditions, is very similar to what standards and best practices are doing to our profession today.
Followed by excellent presentation by James Christie at CAST, some of our colleagues in community have come up with petitions to stop ISO 29119 testing standard. I feel that they are right in what they are fighting for. I would urge you to read this, this, this, this or this post if this whole controversy makes you curious about it.
People who are against this opposition may question, “Who decides if standards are good or bad for our profession? You guys?”
Coincidently, Bhishma had asked the same question to Krishna. I will let you read their remaining conversation.
Bhishma: Who decides which traditions are rotten? You, Lord Krishna???
Krishna: No! Time will decide it! And no one can disobey time’s order.
This is the period of solstice for entire Aryavarta. The way Sun changes its path in solstice; whole Aryavarta too is going to change its path. Old traditions will break, old dynasties will get destroyed. New dharma will be established and new era will begin.
And a time will come when you will have to decide on which side you want to be. Because, everyone will be caught in this trap of time. People who promote lies will become cause of this transformation and people who walk on path of truth will establish the new era.
And this is decided!
Krishna’s answer sums it all. I have signed the petition and I’m done with my part. It’s your turn to decide on which side you want to be.
Note: This blog is formed after my editorial for July/August 2014 issue of Tea-time with Testers.
A passionate & thinking tester. Trainer & student of the craft of testing. Chief Editor and Co-founder of Tea-time with Testers magazine.