Considering my experience and what I have been observing in the industry, there seems to be an increasing interest in the idea of Whole Team Quality. The idea itself is not new as far as I know but certainly, there seems to be more awareness and eagerness towards its implementation, lately.
Why is it needed and how does it help?
Well, if you are delivering a product as a team, it is natural that everyone who helps build the product is responsible for its quality, or is supposed to be. And for more on this I would urge you to read another article from me on this topic.
Where is the problem?
When I tried to figure out how different organisations and teams are going about Whole Team Quality, I realised that asking everyone in the team to test (or asking programmers to test) and automating as much as possible is what they consider Whole Team Quality to be.
I see several problems with that approach:
Sure I do support the idea of Whole Team Testing to help achieve Whole Team Quality but how do you go about it makes the big difference.
Over the last four years, I tried different ideas, did experiments in teams for succeeding with Whole Team Quality. I failed but I learned. I continued to try and eventually I would say I succeeded it in. Succeeded in achieving Whole Team Quality in a meaningful way. The way in which risks are found earlier (even before they manifest as a bug in the product) and the Quality is assessed/analysed/addressed/achieved on every level and by every individual in the team.
The solution that is working for us
Based on my experiments and learning, I would like to present the model and framework I have developed and am still experimenting with. It has given me and my team useful results so far and I would encourage you to try it too.
The model: QualiTri for three notions of Quality
Having deep philosophical discussions with Michael Bolton when he peer-reviewed my paper on Whole Team Quality, helped me formulate/conceptualise QualiTri. And this model further guided me to create the framework for its implementation.
Like I said before, focusing on the Product notion of the quality alone is not enough. To succeed with Whole Team Quality, it is equally important to understand Project and the People notion of quality. They are related and they do affect each other. That said, to deliver a quality product we got to be equally conscious about the project and the people notion of the quality.
The framework: Quality-conscious Software Delivery
The challenge was how to really go about implementing QualiTri model in a context. And thinking about it helped me formulate my goal to be to achieve the delivery of quality products by quality-conscious people using quality-empowering processes.
How to implement the 4E structure of QCSD can vary from context to context but below is how we implemented it in our team which has worked great for us so far.
Going further in detail of the implementation of 4E structure for QCSD framework would require a series of blog posts. It starts with creating awareness, convincing your team for the need of it, considering their inputs, evaluating the project context and creating the workflows/action items together with your team, and then committing for the efforts needed. It's a process that takes time. Plus it is highly subjective from project teams to project teams and their contexts. And hence I would rather stop here for now.
How do we know it worked?
Below you can see the Lead Time graph (solely for representation purpose) before and during an experimentation phase of QCSD in our team (based on the improvements we did in the processes, consciousness with which all people worked with and keeping quality of the product in mind).
I believe it was the first sprint in a long time, where we as a team finished all the tickets and pulled more, the so-called testing bottle-neck was minimal and the bugs reported that would make into the backlog or warrant some critical rework post-production were negligible.
Sure, this graph did not remain ideal all the time. Teams change, business contexts change too which affects the overall delivery and quality of the product we end up shipping. But if you know how to go about delivering a quality product by quality-conscious people using quality-empowering processes, I am almost certain you will do way more good than bad. And it's a win in my opinion.
See if you find it worth the shot. If you would like to borrow my help for consulting/implementing this idea for your team or organisation, it would be my pleasure. Just let me know.
I have been talking about Whole Team Quality via Whole Team Testing for a couple of years now. During my workshops, I am often asked if testing can only be extended for programmers in a team. Pretty interesting question it is and my answer is obviously "no". Though I usually explain in my workshops on how to extend testing to roles beyond programmers in a team i.e. for UX or PO roles, I realized that I have not given deep thought to it and to how exactly testing could be extended to other disciplines in a meaningful way.
I read books, discussed with my colleagues, did my research and the outcome has been what I would like to name as QX i.e. Quality Experience. If QA (read that as Quality Advocates) and UX professionals collaborate in a meaningful way, I firmly believe they can co-create a Quality Experience for everyone associated with the product.
So, what is QX after all?
QX stands for Quality Experience. For sake of understanding, you can call it a marriage between QA (read it as Quality Advocacy please) and UX. After some need-based discussions and interactions with my UX colleagues, I realized that we can achieve a lot more if we work closely together regularly. The key idea of QX is about facilitating the collaboration between QA and UX so that they can contribute to what I would call a "Quality Experience" of the product. This is both for the end-user and for those who build the product itself.
I believe that with some process optimizations, mindset enablers for testers as well as UX designers, and following some heuristics I have created, it is possible to kickstart the QX journey if that idea interests you.
But what's the need? Is QA alone not enough to cater to product quality (or UX alone not enough for better user experience)?
Well, I am afraid, it is not. Not at-least when you believe that Quality is value to someone (who matters) and when multiple stakeholders matter at the same time.
Let me explain. Imagine that a new design change required in a product is a revenue booster for the company but it is also likely to impact the user experience. Testers often end up with oracle problem in such situations and can not decide what their quality criteria should look like. Of course, the Product Owner can be consulted here for a final decision but that's not the point. We testers are in the information business, after all (yes, even if you follow the Modern Testing). I find it important for testers to be able to make a comprehensive information gathering and present that information to decision-makers so that they can make an informed decision based on that.
Now, if testers lack the tools and mindset to figure out how to go about solving such problems and gathering information that would matter, their job would be poorly done. And if you are still not convinced, I highly recommend you to read Weinberg's latest book around System Design Heuristics. This book could not have come at any other better time for me. I started reading it while working on the QX concept and it has given me some interesting insights to sharpen my thinking around this topic.
On the other hand, let's assume that a UX/Interaction designer has been given a problem to solve or has been asked to create a new solution for some product. How do they ensure they have gathered enough information to do that job right? How about historical incidents or say hidden technical challenges or simply put edge case scenarios, cross-functional dependencies, and so on? I believe that having this information at hand can greatly benefit how UX designers can approach the problem and solve that in a better way.
Therefore I think that an engagement between UX and QA can help both of them to perform their job even better. And hence QX seems to be a good way to go about it.
Well, how does it work?
Here are some ideas that I experienced to be working quite well. See if they work for you too?
1. Cross-discipline training for QA and UX
For successful collaboration, it is important that QA and UX understand each other well and that they speak and understand each other's language. More important is to understand the mindset with which they both operate.
I have experienced difficulties in understanding UX's point of view sometimes and my UX colleagues made the same experience. But a conscious effort made towards understanding each others' language helped us solve those issues and facilitating the collaboration thereby.
That said, we recommend that testers and UX designers start from understanding each other's roles and mindset. Attending cross-discipline training should help but if that is not possible, try doing "pairing sessions" at least.
2. Process changes or optimizations
A great deal of it depends on what kind of team set up you have. Some teams may have dedicated UX designers, some organizations have UX teams as a "lateral" service provider for different teams and some teams simply don't have UX people. Their designs are usually outsourced or made by engineers themselves. Some recommendations I would like to make here are:
1. Involve testers as early as possible and make them also part of the design process/discussions. UX will thank them for lots of useful info which might result in the faulty design if missed.
2. Early and frequent communication between UX and QA would help. Try "brainstorming" sessions for early design stages. Ask tester for hidden scenarios or technical hacks to go around things. Ask them for user complaints or known production issues surrounding the design under discussion.
3. Testers may perform focused UX testing and consult UX from time to time for their design-related findings. A "pair testing" session with UX expert can greatly benefit testers for more test ideas surrounding usability and human-computer interaction under different contexts.
4. Instead of creating a misinformed bug report surrounding usability and UX, testers can always consult UX colleagues first for feedback around their findings and use them as their oracles. Most of the time, UX people have information and insights (from their interaction with real users, test sessions they perform, qualitative and quantitive data analysis they do, etc.) that explain why that design is made a certain way.
5. Not every design change goes via the standard UX lead design process. Engineers sometimes have to make decisions that may result in a change in product design or impact certain product behaviors. Such changes can always be sent to UX designers for their feedback. The tester can play a big role in making this activity happen on behalf of the engineering team.
3. UX testing heuristics for testers
Mere exchange with UX colleagues without having proper knowledge of how to test for better user experience can be futile. Which is why I recommend testers to get good at UX testing too. By that, I don't mean typical usability testing or accessibility testing alone. After giving a deep thought to all possibilities involved, I have come up with the following heuristic for testers.
Keep in mind that it can also be used as a means to facilitate collaboration and have a better discussion with UX colleagues. Not everything might be applicable for all the contexts. Choose those that fit best in your context. Here you go:
Problem -To come up with relevant test ideas, Testers and UX must be on the same page in terms of understanding the problem they want to solve with the proposed design. UX often get first-hand information from the PO about the problems, which testers sometimes are lacking. Trying to understand what problem UX wants to solve with their solution, can open up lots of possibilities for testers and confine them to think in the right direction. It would also spare them from coming up with the right questions and perform better impact analysis of the change.
User Needs -
This is highly subjective and might vary from project to project. The key idea is to understand what User Needs are getting addressed through the design and if you as a tester can foresee any challenges/impact of that. This can also be very well made as a part of 'understanding the problem' part. It does not matter how you do it, but what matters most is that you know what aspects of user needs you are dealing with.
User vs Business Needs
Finding Balance - Solving the Oracle Problem
When you as a tester are unable to decide if the proposed solution is good for the product but bad for users and vice versa, use some of the methods below to make your decision-making more concrete and practical. Based on the configurations of teams, there may be no dedicated UX expert to cater to these needs. It is therefore advisable for testers that they wear UX hats and find the balance by asking the right questions before it is too late.
Plenty of efforts could be saved if testers too are involved in design/UX decisions early where dedicated UX expert is absent or designs have been made by third party services etc.
What Must Not Change?
Whatever we ultimately do, what are the things you don’t want to be changed? (from System Design Heuristics by Jerry Weinberg)
Introducing design changes in an existing product that are targeted at a specific goal by UX designers, can harm things that are not meant to be affected by that. I highly doubt if there are deliberate efforts made to analyze this regression impact on the design level itself. This is why, if a tester asks this question right in the early phase of a design change, it is likely to save lots of rework for the later.
“If we don’t start that way, it’s all too easy to lose track of the unchangeable.” - Jerry Weinberg.
What are the visible and invisible parts of the product that are impacted by this change? And what is the impact?
Running out of test ideas to decide if the solution is perfect? Pour some creativity in.
Exactness, Intuitive and Counter-intuitive Design
Most of the time, testers are more focused on the technical and functional aspects of the product so much that they unknowingly tend to ignore looking for obvious problems. A deliberate attempt needs to be made to look at the product like an unexperienced user to see if they will understand what we expect them to. To do this:
Well, I wish I could work on this and develop this idea even further. But for the benefit of time, I would stop here. Maybe I would get back to this again when I have more ideas. But in the meantime, feel free to comment below and share more ideas if you like. And do not forget to tell me how you find the QX idea so far.
And, I recommend you to read System Design Heuristics by Jerry Weinberg. The book gave me lots of ideas to ponder upon.
The more I think about events and people around me, the more I have started to believe in higher calling and celestial hints. I have been wanting to write about Jerry and as I started to pen my heart down finally, I learned that it’s Jerry’s birthday today. Could that be a coincidence?
Happy birthday Jerry! This time, accept it with tears in my eyes and lots of gratitude.
2018 has been a year of losses and realisations for me. I lost people that I once held close to my heart and spirit, I lost opportunities that could have helped me contribute better to the community. But the realisation that has come after these losses has been an eye opening experience.
One of such persons that was very close to my heart from years, that I lost this year has been Jerry Weinberg. Yes, The Jerry Weinberg.
It’s been almost four months now and I am still finding it hard to believe that Jerry is no longer there with us in this world, in his physical form. After Jerry’s demise, a lot of my friends and colleagues have written about his impact on their lives and how he helped them transform their lives. Reading that all has made me feel even more emotional. The realisation of being close to someone for years, who has transformed many lives in many ways, makes me feel so very special and sad at the same time.
I really don’t know if anyone else has received so much from Jerry, as much as I feel I have received from him. But I consider myself truly fortunate for him coming in my life and transforming me bit by bit, month by month and year by year into a better person and software professional.
I still remember the day when I wrote Jerry asking for his permission to publish one of his article in Tea-time with Testers’ earlier edition. It was February of 2011 when I wrote him for the first time. I shared our first issue with Jerry so that he could make a decision about his contribution.
To my surprise, Jerry admired the work we had done with our first edition and happily agreed to contribute his work. ‘Testing without Testing’ was his first article that we published.
Jerry liked our project for multiple reasons. He admired our will to make meaningful contribution to software testing community, the efforts we were taking by collaborating with many testing experts in the world and the value that he saw we were creating with this project. Moreover, Jerry liked that we mentioned our ‘team’ as our ‘family’ and when he said he would love to be part of it, my joy had no bounds.
And since then, Jerry Weinberg became part of Tea-time with Testers family. We felt like we got an angel to guide us in times to come. And Jerry proved every bit of that feeling to be true.
I have lost the count of how many times we communicated with other, over these seven years. But each time, I communicated with Jerry, my respect, admiration and love for him grew more and more. With each issue we published, Jerry gave us feedback and helped us become better and better. His feedback around my editorial that sited Kipling’s ‘If you can….’, is still fresh in my mind. He told me that had that poem hung on his wall for quite some years of his life. Don’t know why but I felt more connected with Jerry after that.
Our collaboration brought me very close to Jerry and I never had to think twice asking for his opinion and support in different initiatives we took over years. State of Testing survey has been one of such project where we closely worked with Jerry. The webinar we did with him and Fiona Charles after that is still fresh in my memories. To check if everything is fine before the webinar, I made Jerry a phone call and that was for the first time, I got to hear his voice. Firm, but it had warmth of its own kind. Darn, just felt like I heard him again. And I have goosebumps on my hand as I am writing this further.
Interesting part of my exchanges with Jerry has been that, once in a while, along with professional work that we were doing together, he shared about himself, his choices, his likes, anecdotes, his take on things happening around and of course his advice on variety of topics we discussed. It never felt like I had not met Jerry in person. There was different pleasure in getting to know him on personal level, bit by bit… as if I was reading story he was writing about himself and there were so many interesting chapters waiting to be shared. I kept wanting for more and Jerry never disappointed me there.
We used to talk about dogs sometimes, German Shepherds especially. The dog breed I suppose Jerry liked the most and me too. He once gave me pleasant surprise by adding picture of me and Victor in his Pinterest collection around GSDs. It’s wonderful collection. Don’t forget to checkout if you are GSD fan like both of us.
I think I can keep writing about Jerry that I got know for days to come. Through our personal collaboration and amazing books Jerry has written, he stays with me every now and then. Sometimes when I get stuck on some concept from his book, I feel like writing him an email and there he will be, explaining me things in a way I will never forget. I wish I could do that forever. This realisation is hard. Feels like I am waking up from some dream, for it never happened that I emailed Jerry and he did not reply.
Meeting Jerry in person was on my list of ‘things to do at-least once in a life’. We were planning to meet in person this year. Every time in past when Jerry wrote me that he was keeping unwell or he had doctor’s visit planned, my heart used to beat faster. And then he used to make me very happy again telling he was doing good and there was nothing to worry so much.
It was bit different this year. I had my flights booked to meet him June and Jerry suggested that I don’t make any reservations for he was unsure if he would be around by then. I did not know what say and even how to respond to that. I cancelled my flight tickets for I never disobeyed Jerry. Not sure why but with that particular exchange, Jerry left an everlasting impression on me. In Hinduism, we have this belief that great people can know of their time to leave the earth, well in advance. I think Jerry was one of such great persons.
His outlook and acceptance towards all phases of life made me feel awaken and enlightened. My that exchange with Jerry made a very positive spiritual impact on me.
I admit, I am yet to meet someone like Jerry who has been so brave and open about all possibilities of life, including the time to say good bye. I’m yet to meet someone so legendary yet so down to earth, someone who can explain mysteries of galaxy as simply and effortlessly as if it is nursery rhyme meant for kids, someone so generous and kind and someone who has transformed lives without even meeting people in person.
I wish I could meet you at least once in my life, Jerry. But not getting to meet you physically makes me firmly believe that you were an angel sent for many of us from heavens. You have touched me and transformed me in many ways and have made me realise my true potential. Moreover, your belief in my abilities and guidance you offered me over years, has made me feel so proud and special about myself. You taught me the power of compassion, you taught me how to stay unaffected by situations beyond our control, you taught me the importance of paying it forward. All through your life you have led us all by your example. I wish I could ask you what did you see in me that made you invest in me. But for whatever reasons you chose me, I will be grateful to you for that forever. And I will try my best to ‘pay it forward’.
Good bye and happy birthday once again, legend!
A passionate & thinking tester. Trainer & student of the craft of testing. Father, Foodie and dog lover. Chief Editor and Co-founder of Tea-time with Testers magazine.