Disclaimer: Purpose of this blog is not to undermine the importance of automation skills for testers but rather to highlight the lack of awareness in our industry for tester's usefulness in teams beyond typical testing and automation tasks.
Recently, I was invited to conduct a workshop around "Whole Team Testing" at Test Leadership Congress in New York. Participating in this conference has been a great experience, especially because I truly believe in the need for conferences dedicated to test leadership and career path for testers in changing era.
The discussions I had there with fellow testers, managers and directors of engineering/testing were interesting. Well, not only interesting but insightful and with lots of ideas, concerns, observations to ponder upon. Out of the things we discussed, what held my attention the most was the growing concern of evaluation of testers in Agile teams. The evaluation can be for hiring or for their overall performance in team.
What is the problem exactly?
If you analyse the trend from State of Testing report, it is evident that with increasing Agile adoption, centralised testing units are mostly getting dissolved and testers are now reporting to Dev/Team leads. With this change of structure, hiring new testers and evaluating those in teams naturally becomes the responsibility of the Dev/Team leads. And this is where things start to get interesting. How? Let's take a look:
But this situation is dragging us into bigger problem. We are giving too much importance to things that are just a part of big scheme of things that matter and contribute to software quality and teams' overall ability to ship quality software, faster and frequently.
Of course automation skills and other technical skills are helpful, rather important I would say. But that has become a norm now and what we really need at this point of time are tools to see beyond this norm. If we want excellent testers in team who can really add value to software quality, it's high time that hiring managers come out of their obsession of hiring testers based on the norms that they can most comfortably evaluate. Please, it is not about what we are most comfortable with but what Agile team really needs from a tester to ship a quality software.
Heavy emphasis on technical skills for testers is not my big concern, but lack of awareness around other things that testers must get evaluated against, concerns me the most. Purpose of this article is to highlight some of those areas (beyond typical technical skills) that I think managers/leads may want to consider while hiring new tester or evaluating those in their teams.
Why do we need testers at first place?
One may argue that in an era of AI and advanced automation where every check can be easily automated, why do we need human testers at all? That's interesting argument and before I explain why we are talking about it, I want you to have a look over the diagram below:
The diagram represents a system that we as a team operate into. I created it based on my experience as a tester so far and by interviewing some programmers I have closely worked with.
Looking at bunch of things mentioned there, you can figure out what all can happen (rather typically happens) when tester in team is not available. If you read the diagram carefully, you will notice that impact on finding and reporting bugs is just one segment that gets highlighted when tester is missing in team. Impact on writing new tests or automating them is another segment. But is that all you need testers for in a team? Certainly not!
The purpose of having testers in team is way beyond finding bugs, writing tests or automated scripts for that matter. If utilised to their full potential, testers in team can very well serve as a mode to protect your system from running into collapse or explosion mode (please read more about that in "How Software is Built" by Jerry Weinberg). How? It's simple. By providing system related feedback to the controller which is typically team lead or test manager in typical team set-up. The better you utilise tester for their abilities the better feedback you get from them which in turn can help you command better control over your system and protect it from collapse/explosion. You need tester in team to provide you with information that you as a stakeholder can use for making better decisions.
And when I say feedback, it can be anything from information about bugs, risks highlighting, asking context revealing questions, questioning user stories or decisions made, finding out historical data, highlighting third-party dependancies, sharing news about decisions made by cross functional teams, collaborating with other disciplines to create valuable assets, their observation around team dynamics, their predictions about possible failures and so on. And if you are lucky, they can also tell you some great things about quality of your product.
The point is, tester in your team can add far more value beyond finding bugs, if you allow them to contribute beyond their typical role. If not, you can always empower them to do so. Here are couple of things that in my experience testers can contribute to and against which you could evaluate them (for hiring or performance reviews etc):
1. Primary analyser of production logs and alerts
Production deployments are typically done by programmers who then closely follow the logs to see if there are any obvious issues created by latest build. Consider handing over this responsibility to testers. It is likely to serve multiple purpose:
2. Enhancers of your product's coverage for quality
It has very little value if tester in your team just sticks with usual acceptance criteria for the ticket and automates the stuff for you to top it. The real value a skilled tester can add is when they question the very product coverage you have in place and help your team see elements of product that matter from quality aspects.
A lot of elements matter when it comes to your product's quality. Functional acceptance criteria is just tiny part of it. A skilled tester would know about these elements and they would educate the team about same.
Check out SFDIPOT from HTSM by James Bach or simply have a look on mindmap below (thanks to Albert Gareev)
3. Advocates for Testability
Testability is one of the key area I would expect skilled tester in my team to help programmers and designers get it right. A skilled tester would know what makes product testable, how to evaluate it for testability and also advocate the team as in where and how they can improve it. Nothing beats the pleasure of testing better testable product.
Here is a heuristic for testability if that makes you curious. Out of various types of testability mentioned there, I would expect the tester to at-least know Intrinsic Testability and how to help their team improve it.
4. Better allies for UX peeps
Ever wondered what would happen if System Thinking meets Design Thinking? I firmly believe that challenges faced by testers and UX professionals are more or less same when it comes to ensuring better quality and better user experience.
A regular and close exchange between these two disciplines has tremendous potential to create a better user experience with enhanced product quality which I would call as "Quality Experience". If UX designer comes with one best solution for some product problem, a skilled tester with their insights, product knowledge, awareness around cross-functional dependancies can point out variety of ways in which it may fail. This does not mean a tester has to criticise UX solutions as such but early collaboration and exchange between two can help avoid lots of unnecessary research and rework.
On other hand, testers can borrow realistic information from UX's research which can help them design tests that matter and prevent themselves from straying into unwanted scope. Testers can borrow statistical data or interaction/persona based information from UX that can help them shape better scope for their tests. Well, this is indeed interesting topic and deserves deep dive. I will stop here on this one for now.
A skilled tester would make this collaboration happen and would bring out the best from both the worlds.
5. Friend in need for marketing and user care
This is another area of collaboration where skilled testers can add great value to solve team's problems beyond finding bugs. I have listed some of the possibilities where testers can help user care and get benefited in return and I believe same applies to their collaboration with marketing teams:
6. Alert mechanism when system is on the verge of collapse
Well, this is bit tricky but I would mention it anyway. By nature of their work, skilled testers can use their sharp observation skills to observe and understand people, situations and events around them and can draw inferences that can help identify potential risks, before it is too late. Retrospective meeting is a great place for skilled tester to raise flags for potential issues surrounding team dynamics or people issues they observed, in constructive way of course which is where their communication skills can come handy.
Testers are usually in touch with fellow testers in organisation from where they get information around what other teams are working on, their future plans, blocking issues etc and this information can be very well used to analyse the impact on their respective team's road-map , work in progress or work items that share dependancies with other teams.
A tester who is skilled enough to foster network and relations across teams, can certainly help bypass blockages when team needs it most.
So, those are some aspects in my opinion and experience, a skilled tester can contribute value to project team beyond their traditional tasks.
If you are to a hire new tester or would like to fairly evaluate/mentor tester in your team, I would suggest give these ideas some consideration. It's high time that we look at tester as someone beyond bug finder. They can do wonders for your team and can help to accelerate shipping of quality product, if allowed to use their full potential.
I shared what I think can be helpful. Feel free to chip in your ideas ....
Evenings... I have started to fall in love with them.
After a tiring day at work, as I walk back towards my little nest ... I meet them on my way.
“Them who?”, you may ask. They are these little birds that do this blissful chirping... and some of them blow that sweet whistle. It feels like they now know me very well and are happy to see me around. Makes me feel so special I tell you. And then there are these beautiful trees. Some small and some of them being huge. They all are now lush, fresh and blooming with colourful blossoms. A couple of them even greet me with maddening fragrance of their flowers... And again... there is this tiny but cutely flowing canal. A bit mild but yet lively sound of its water when it splashes against little rocks in its way... it hypnotises me, I must say. That very old wooden bridge right above this naughty canal...it asks me to talk with it for a while, everyday. Sometimes it tells me the sweet old stories of old couple living in a big farm-house nearby and also about those naughty deers that come there for drinking water and run away as fast as they come. The bridge says he gets angry on them for not even saying “hello” but never forgets to mention how much he loves them.
Then I meet these big farms, proudly showing their bright yellow Rapseed. It looks as if whole land is wearing yellow blanket with lush green skirting. And what should I say about this saffron Sun? I have never seen something as magnificent as him. He is brightest of them all in the sky but still down to earth. I guess I would hardly find something as humble as a setting Sun. Until last minute of his goodbye for the day, his grace and elegance do not fade by a tinge.
Today I stopped by the bridge and asked them all, “hey, can I become your friend? I feel so lonely.”
They all bursted into a laughter as if they knew I would ask it some day. And then one of them said, “Why not? But there is one condition!”
“What condition?”, I asked hurriedly.
“You need to be your own friend first, young man”, said the old bridge and everyone else nodded.
The confused me looked at the giant oak tree there. “Let me explain”, said the kind guy.
“All of us here are in harmony with each other. But before that, we are in harmony with ourselves. If you want to be one of us, you need to become one with your inner-self. The day you do that, you will find yourself among us.”
I believe I got it. What I have been seeking outside, must be deep down inside me too. Quietly thinking about it for a while, I smiled, felt like I got an answer I have been looking for. I thanked them and we parted our ways... with a promise to meet again tomorrow.
I can’t wait to become one of them someday! And be in harmony with myself first!
"I am software tester. My job is to break software." , said one student in my Exploratory Testing workshop. I asked him to elaborate and explain me his techniques to break the software. He was silent for moment and then said he did not know how exactly to answer that.
I further asked about the last defect he found and how did he find it. He could explain that to my satisfaction. Then I asked if the defect was already there or something that he did introduced it. Student realised where I was coming from and admitted that he did not break software, he only helped to uncover the software which was already broken. Then I asked him once again to explain his techniques to uncover those already broken points and he explained that to my satisfaction again without getting frozen in between.
One of the important lessons I have learned from James Bach and something that I make sure to propagate in my discussions with testers is that, we (testers) need to be careful of the vocabulary we use to describe our work because it indeed makes big difference. With this little change, I have experienced the change in me in terms of how I perceived things before with the use of established vocabulary and after starting to choose my vocabulary carefully. That's not it, I have witnessed that when you help people to realise the same in a constructive way, they also make sure they help others to realise it. And this "chain reaction" of propagating that gesture is in my opinion, an important part of contributing to the craft.
I came across this thought-provoking post written by Maaret Pyhäjärvi (influential colleague in testing community I respect and admire ) and some of the arguments she has made, made me ponder upon my own attempts and experiences.
In her blog Maaret says:
Instead of changing the vocabulary, I prefer changing people's perceptions. And the people who matter are not random people on twitter, but the ones I work with, create with, every office day.
I totally support the idea of helping to change peoples' perceptions. I have made those efforts and have seen that taking effect. The approach is very much in line with Weinberg's idea of influence and that has always been my first approach towards changing something. However, the results in this particular case in my experience have been short-lived. I found people to be coming back to their initial understanding which was primarily shaped by established vocabulary and every once in a while I had to discuss the same thing with them again. Can I say my efforts were paying off? I guess not really.
What was and is the problem?
I realised that people that I had helped change their perception (mostly programmers and non-testers) got back to old vocabulary because other testers and stakeholders they were working with, were totally unaware of what they were talking about and why. Those people found it very frustrating to explain others their rationale behind using different vocabulary and eventually they gave up. I remember of one programmer friend coming to me and saying that he felt silly and stupid because the tester he was talking to was totally clueless of what he meant. And he finally said if testers themselves don't care about what their vocabulary should mean, why should he? And he was right!
The problem is, testers who understand the problem with established vocabulary are very less in number as compared to an entire lot of project stakeholders who use it. And testers who make an effort and help others to change their perception are even less.
And this is why I personally see the problem with "living with" some established norms which need revision. I think it's a high time that we strongly disapprove of what we do not believe in. Because it badly affects all the efforts made by people who care. When we know about problem with things and still decide to live with them, our awareness about those problems becomes pointless or less effective if not.
It is not just about people around us
The other day I watched this humours video by AIB on mass technical recruitments where they pick two gardeners towards end of recruitment to fill their quota and say, "Let's put these two on manual testing. Who requires talent for that anyway?"
That was very difficult to digest for me but at the same time, I could not blame the producers of that video because our established vocabulary is not their problem. They just presented the widely established (and mistaken) perception of our established vocabulary.
If we as testers don't care enough about changing something wrong just because it is established, we are letting others shape wrong perception about our profession and that is a silent killer. One of the leading testing tool company recently tried to showcase "manual testing" as outdated, bad testing and proposed their tool that supports Exploratory Testing as a solution. Major part of our industry still considers testing = manual testing = bottle neck and hence thinks of eliminating testing all together. But in reality what they want to get rid of is bad testing only and not manual testing or testing as such. If it is not us then who else is supposed to care about these problems and make efforts to solve them? And I don't know how best we can stop it other than getting rid of the labels and classification which is adding a lot to confusion.
In my opinion, we are responsible for how we let others shape their understanding of us. And no matter how hard we try to do it with people around us, there will always be people beyond our scope who will undo what we do. If we keep collecting the karma of living with established vocabulary and do not make deliberate efforts to change it, it is most likely going to haunt us and our generations (if at all we survive).
It is now up to us whether to collect that karma or to cleanse it. Cleansing sounds reasonable to me but I am still looking for more options . What if we do both?
A passionate & thinking tester. Trainer & student of the craft of testing. Father, Foodie and dog lover. Chief Editor and Co-founder of Tea-time with Testers magazine.