Disclaimer: Purpose of this blog is not to undermine the importance of automation skills for testers but rather to highlight the lack of awareness in our industry for tester's usefulness in teams beyond typical testing and automation tasks.
Recently, I was invited to conduct a workshop around "Whole Team Testing" at Test Leadership Congress in New York. Participating in this conference has been a great experience, especially because I truly believe in the need for conferences dedicated to test leadership and career path for testers in changing era.
The discussions I had there with fellow testers, managers and directors of engineering/testing were interesting. Well, not only interesting but insightful and with lots of ideas, concerns, observations to ponder upon. Out of the things we discussed, what held my attention the most was the growing concern of evaluation of testers in Agile teams. The evaluation can be for hiring or for their overall performance in team.
What is the problem exactly?
If you analyse the trend from State of Testing report, it is evident that with increasing Agile adoption, centralised testing units are mostly getting dissolved and testers are now reporting to Dev/Team leads. With this change of structure, hiring new testers and evaluating those in teams naturally becomes the responsibility of the Dev/Team leads. And this is where things start to get interesting. How? Let's take a look:
But this situation is dragging us into bigger problem. We are giving too much importance to things that are just a part of big scheme of things that matter and contribute to software quality and teams' overall ability to ship quality software, faster and frequently.
Of course automation skills and other technical skills are helpful, rather important I would say. But that has become a norm now and what we really need at this point of time are tools to see beyond this norm. If we want excellent testers in team who can really add value to software quality, it's high time that hiring managers come out of their obsession of hiring testers based on the norms that they can most comfortably evaluate. Please, it is not about what we are most comfortable with but what Agile team really needs from a tester to ship a quality software.
Heavy emphasis on technical skills for testers is not my big concern, but lack of awareness around other things that testers must get evaluated against, concerns me the most. Purpose of this article is to highlight some of those areas (beyond typical technical skills) that I think managers/leads may want to consider while hiring new tester or evaluating those in their teams.
Why do we need testers at first place?
One may argue that in an era of AI and advanced automation where every check can be easily automated, why do we need human testers at all? That's interesting argument and before I explain why we are talking about it, I want you to have a look over the diagram below:
The diagram represents a system that we as a team operate into. I created it based on my experience as a tester so far and by interviewing some programmers I have closely worked with.
Looking at bunch of things mentioned there, you can figure out what all can happen (rather typically happens) when tester in team is not available. If you read the diagram carefully, you will notice that impact on finding and reporting bugs is just one segment that gets highlighted when tester is missing in team. Impact on writing new tests or automating them is another segment. But is that all you need testers for in a team? Certainly not!
The purpose of having testers in team is way beyond finding bugs, writing tests or automated scripts for that matter. If utilised to their full potential, testers in team can very well serve as a mode to protect your system from running into collapse or explosion mode (please read more about that in "How Software is Built" by Jerry Weinberg). How? It's simple. By providing system related feedback to the controller which is typically team lead or test manager in typical team set-up. The better you utilise tester for their abilities the better feedback you get from them which in turn can help you command better control over your system and protect it from collapse/explosion. You need tester in team to provide you with information that you as a stakeholder can use for making better decisions.
And when I say feedback, it can be anything from information about bugs, risks highlighting, asking context revealing questions, questioning user stories or decisions made, finding out historical data, highlighting third-party dependancies, sharing news about decisions made by cross functional teams, collaborating with other disciplines to create valuable assets, their observation around team dynamics, their predictions about possible failures and so on. And if you are lucky, they can also tell you some great things about quality of your product.
The point is, tester in your team can add far more value beyond finding bugs, if you allow them to contribute beyond their typical role. If not, you can always empower them to do so. Here are couple of things that in my experience testers can contribute to and against which you could evaluate them (for hiring or performance reviews etc):
1. Primary analyser of production logs and alerts
Production deployments are typically done by programmers who then closely follow the logs to see if there are any obvious issues created by latest build. Consider handing over this responsibility to testers. It is likely to serve multiple purpose:
2. Enhancers of your product's coverage for quality
It has very little value if tester in your team just sticks with usual acceptance criteria for the ticket and automates the stuff for you to top it. The real value a skilled tester can add is when they question the very product coverage you have in place and help your team see elements of product that matter from quality aspects.
A lot of elements matter when it comes to your product's quality. Functional acceptance criteria is just tiny part of it. A skilled tester would know about these elements and they would educate the team about same.
Check out SFDIPOT from HTSM by James Bach or simply have a look on mindmap below (thanks to Albert Gareev)
3. Advocates for Testability
Testability is one of the key area I would expect skilled tester in my team to help programmers and designers get it right. A skilled tester would know what makes product testable, how to evaluate it for testability and also advocate the team as in where and how they can improve it. Nothing beats the pleasure of testing better testable product.
Here is a heuristic for testability if that makes you curious. Out of various types of testability mentioned there, I would expect the tester to at-least know Intrinsic Testability and how to help their team improve it.
4. Better allies for UX peeps
Ever wondered what would happen if System Thinking meets Design Thinking? I firmly believe that challenges faced by testers and UX professionals are more or less same when it comes to ensuring better quality and better user experience.
A regular and close exchange between these two disciplines has tremendous potential to create a better user experience with enhanced product quality which I would call as "Quality Experience". If UX designer comes with one best solution for some product problem, a skilled tester with their insights, product knowledge, awareness around cross-functional dependancies can point out variety of ways in which it may fail. This does not mean a tester has to criticise UX solutions as such but early collaboration and exchange between two can help avoid lots of unnecessary research and rework.
On other hand, testers can borrow realistic information from UX's research which can help them design tests that matter and prevent themselves from straying into unwanted scope. Testers can borrow statistical data or interaction/persona based information from UX that can help them shape better scope for their tests. Well, this is indeed interesting topic and deserves deep dive. I will stop here on this one for now.
A skilled tester would make this collaboration happen and would bring out the best from both the worlds.
5. Friend in need for marketing and user care
This is another area of collaboration where skilled testers can add great value to solve team's problems beyond finding bugs. I have listed some of the possibilities where testers can help user care and get benefited in return and I believe same applies to their collaboration with marketing teams:
6. Alert mechanism when system is on the verge of collapse
Well, this is bit tricky but I would mention it anyway. By nature of their work, skilled testers can use their sharp observation skills to observe and understand people, situations and events around them and can draw inferences that can help identify potential risks, before it is too late. Retrospective meeting is a great place for skilled tester to raise flags for potential issues surrounding team dynamics or people issues they observed, in constructive way of course which is where their communication skills can come handy.
Testers are usually in touch with fellow testers in organisation from where they get information around what other teams are working on, their future plans, blocking issues etc and this information can be very well used to analyse the impact on their respective team's road-map , work in progress or work items that share dependancies with other teams.
A tester who is skilled enough to foster network and relations across teams, can certainly help bypass blockages when team needs it most.
So, those are some aspects in my opinion and experience, a skilled tester can contribute value to project team beyond their traditional tasks.
If you are to a hire new tester or would like to fairly evaluate/mentor tester in your team, I would suggest give these ideas some consideration. It's high time that we look at tester as someone beyond bug finder. They can do wonders for your team and can help to accelerate shipping of quality product, if allowed to use their full potential.
I shared what I think can be helpful. Feel free to chip in your ideas ....
During my ET/SBTM workshops, I have been often asked if it is possible to perform Session Based Testing in typical Agile/DevOps environments.
I think if tester knows how to perform SBT (especially with different session types), there should be no trouble in doing it regardless of the development methodology their team follows. (If you are unaware of typical session types in SBTM, then I recommend you to checkout this informative series by my friend Simon ‘Peter’ Schrijver.) However, I understand that probably, the fast paced way of working through sprints, shipping small chunks of software regularly, keeps everyone mainly focused around achieving the sprint goal. And for testers, this often means focusing only on ‘acceptance criteria’ and moving on to the next ticket, which is waiting to be tested and deployed. And that might force some testers to just forget every awesome technique they know and run behind getting “the stuff done”.
If you are a tester (sorry, Agile Tester) stuck in such situation and feel bad about it, then this post is for you. For a while, I too was stuck in similar situation. Not that I was not getting to perform ET in SBTM way, but I constantly felt something was missing and that there was still more I could do. A couple of important things were getting skipped off my typical sessions, and all I could do was procrastinate them. And that was bad, very bad! (yeh)
When I was done with feeling guilty about it, here is what I did (and I strongly recommend you too try it out). I created some more session types for myself some of which I perform twice a week and some I perform daily. And they have been helping me greatly so far. Well, what are they?
1. Critical Thinking Session
I know, a good tester should think critically at every possible occasion. But there is no harm if you dedicate a special slot for it. How often do agile testers spend time on reading through backlog and preparing their notes in advance so that they know what to ask, know what extra information they would need for certain ticket or prepare a risk list associated with some ticket, to make everyone aware of it before it’s too late?
A dedicated ‘Critical Thinking’ session aims to solve this problem. Schedule at-least two of such sessions, one before beginning of new sprint and one before your feedback/refining sessions so that you spend enough time for ‘critically thinking’ about stuff that would be soon on your plate. The sooner you prepare yourself with it, the better you will be able to shape your further sessions around actual testing of those items.
Medium sessions of roughly 60 minutes works great for me. Sometimes a short session of 45 mins is enough. The more you practice with Critical Thinking sessions and the more time you spend with applications you test, I guess you will eventually need lesser time for CT sessions.
In case you are wondering what I really mean by “Critically Thinking” about backlog items then please checkout “Mary Had A Little Lamb” heuristic by Jerry Weinberg or “Huh?Really?So?” by James Bach.
2. Monitoring Session
Monitoring production logs especially after deployments has benefits of its own. And if tester does it regularly then there is a lot they can discover through those logs.
If as a tester, you are not yet doing production deployments then I suggest you start doing it and make sure to check production logs (or logs on other environments for that matter). If for some reason, you do not own deployments then try to spend at-least one short session for monitoring your application’s production logs, every day. That’s what I mean by “Monitoring session”.
Because of such dedicated monitoring sessions, I have come across some illusive bugs that were hard to catch on testing environment. Production logs are also very interesting means to learn about different ways in which you can test your application against disfavored use or extreme use (refer HTSM by James Bach – SFDIPOT – Operations). Not only that, it can also help you identify some illusive integration level bugs which might not be caught easily otherwise.
Monitoring sessions can also help you identify technical bugs, errors or warnings, which might not be directly affecting the end user but still warrant attention and fixing. It’s hard to identify such issues in regular functional testing which usually has big scope of its own.
3. Bug-visit Session
This session is about visiting the bugs in backlog and going through them carefully. Sometimes, over the period of time, some bugs become irrelevant or can also become important to be fixed on priority. Revisiting those bugs helps take appropriate action on time.
If there are bugs logged by other teams or customer care team for example, then they can also serve as a great ‘test ideas’ that can be extended to other features of application. Information gathered through such bug-visit sessions can help you create your own risk-list or project specific heuristic (I have explained that in detail with HEEENA).
4. Test for Testability Session
I can't stress enough on importance of this session, especially in this changing era of Software Development. One of the key role a skilled tester can play in modern software development is of "advocate of testability". Checkout this heuristic for Software Testability by James Bach. Once you understand the dynamics of testability you will realise that the sooner you care for it (and advice where needed) the better testable product your team will create. If I am to give you an example, please check "stats for nerds" on youtube videos you watch. Would such information not help you if your product starts storing such information, that you as a tester can access easily and shape your tests based on that?
I suggest you create the checklist for Intrinsic Testability in particular and test your design right from the beginning against it. Please dedicate a compulsory session (short should be good to begin with) for testability even before you get the build for testing against acceptance criteria. In fact, you could also pair with programmer or PO when they work on user stories that will latter come to you for independent testing. This in tern will help you as a tester to better test the product for session/charter you care for (and you won't get lost in figuring things out to help you test better).
So, that’s about four additional session types I have created for myself to do even more effective SBTM in agile environment. I am finding them very useful and hope you too will benefit from them.
Feel free to get in touch if you have any questions or would like to discuss it further.
Header image credit - blog.xebia.fr
A passionate & thinking tester. Trainer & student of the craft of testing. Father, Foodie and dog lover. Chief Editor and Co-founder of Tea-time with Testers magazine.