TALES OF TESTING
  • BLOG
  • QCSD
  • Tea-time with Testers
  • Press, Publications & Talks
  • About
    • Testimonials
    • Photos

TALES OF TESTING

​​  Testing, Leadership, Software and more...
QUALITY CONSCIOUS SOFTWARE DELIVERY
About me

Quality Experience(QX): Co-creating Quality Experience for everyone associated with the product.

3/7/2020

0 Comments

 
Picture
I have been talking about Whole Team Quality via Whole Team Testing for a couple of years now. During my workshops, I am often asked if testing can only be extended for programmers in a team. Pretty interesting question it is and my answer is obviously "no". Though I usually explain in my workshops on how to extend testing to roles beyond programmers in a team i.e. for UX or PO roles, I realized that I have not given deep thought to it and to how exactly testing could be extended to other disciplines in a meaningful way. 

I read books, discussed with my colleagues, did my research and the outcome has been what I would like to name as QX i.e. Quality Experience. If QA (read that as Quality Advocates) and UX professionals collaborate in a meaningful way, I firmly believe they can co-create a Quality Experience for everyone associated with the product. 

So, what is QX after all?

QX stands for Quality Experience. For sake of understanding, you can call it a marriage between QA (read it as Quality Advocacy please) and UX. After some need-based discussions and interactions with my UX colleagues, I realized that we can achieve a lot more if we work closely together regularly. The key idea of QX is about facilitating the collaboration between QA and UX so that they can contribute to what I would call a "Quality Experience" of the product. This is both for the end-user and for those who build the product itself. 

I believe that with some process optimizations, mindset enablers for testers as well as UX designers, and following some heuristics I have created, it is possible to kickstart the QX journey if that idea interests you. 
But what's the need? Is QA alone not enough to cater to product quality (or UX alone not enough for better user experience)? 

Well, I am afraid, it is not. Not at-least when you believe that Quality is value to someone (who matters) and when multiple stakeholders matter at the same time. 
Let me explain. Imagine that a new design change required in a product is a revenue booster for the company but it is also likely to impact the user experience. Testers often end up with oracle problem in such situations and can not decide what their quality criteria should look like. Of course, the Product Owner can be consulted here for a final decision but that's not the point. We testers are in the information business, after all (yes, even if you follow the Modern Testing). I find it important for testers to be able to make a comprehensive information gathering and present that information to decision-makers so that they can make an informed decision based on that.  

Now, if testers lack the tools and mindset to figure out how to go about solving such problems and gathering information that would matter, their job would be poorly done. And if you are still not convinced, I highly recommend you to read Weinberg's latest book around System Design Heuristics. This book could not have come at any other better time for me. I started reading it while working on the QX concept and it has given me some interesting insights to sharpen my thinking around this topic.

On the other hand, let's assume that a UX/Interaction designer has been given a problem to solve or has been asked to create a new solution for some product. How do they ensure they have gathered enough information to do that job right? How about historical incidents or say hidden technical challenges or simply put edge case scenarios, cross-functional dependencies, and so on? I believe that having this information at hand can greatly benefit how UX designers can approach the problem and solve that in a better way. 

Therefore I think that an engagement between UX and QA can help both of them to perform their job even better. And hence QX seems to be a good way to go about it. 

Well, how does it work?

Here are some ideas that I experienced to be working quite well. See if they work for you too?

1. Cross-discipline training for QA and UX 

For successful collaboration, it is important that QA and UX understand each other well and that they speak and understand each other's language. More important is to understand the mindset with which they both operate. 

I have experienced difficulties in understanding UX's point of view sometimes and my UX colleagues made the same experience. But a conscious effort made towards understanding each others' language helped us solve those issues and facilitating the collaboration thereby.  

That said, we recommend that testers and UX designers start from understanding each other's roles and mindset. Attending cross-discipline training should help but if that is not possible, try doing "pairing sessions" at least.   

2. Process changes or optimizations  

A great deal of it depends on what kind of team set up you have. Some teams may have dedicated UX designers, some organizations have UX teams as a "lateral" service provider for different teams and some teams simply don't have UX people. Their designs are usually outsourced or made by engineers themselves.  Some recommendations I would like to make here are:
1. Involve testers as early as possible and make them also part of the design process/discussions. UX will thank them for lots of useful info which might result in the faulty design if missed. 

2. Early and frequent communication between UX and QA would help. Try "brainstorming" sessions for early design stages. Ask tester for hidden scenarios or technical hacks to go around things. Ask them for user complaints or known production issues surrounding the design under discussion. 

3. Testers may perform focused UX testing and consult UX from time to time for their design-related findings. A "pair testing" session with UX expert can greatly benefit testers for more test ideas surrounding usability and human-computer interaction under different contexts. 

4. Instead of creating a misinformed bug report surrounding usability and UX, testers can always consult UX colleagues first for feedback around their findings and use them as their oracles. Most of the time, UX people have information and insights (from their interaction with real users, test sessions they perform, qualitative and quantitive data analysis they do, etc.) that explain why that design is made a certain way. 
5. Not every design change goes via the standard UX lead design process. Engineers sometimes have to make decisions that may result in a change in product design or impact certain product behaviors. Such changes can always be sent to UX designers for their feedback. The tester can play a big role in making this activity happen on behalf of the engineering team. 
 
3. UX testing heuristics for testers 

Mere exchange with UX colleagues without having proper knowledge of how to test for better user experience can be futile. Which is why I recommend testers to get good at UX testing too. By that, I don't mean typical usability testing or accessibility testing alone. After giving a deep thought to all possibilities involved, I have come up with the following heuristic for testers.

Keep in mind that it can also be used as a means to facilitate collaboration and have a better discussion with UX colleagues. Not everything might be applicable for all the contexts. Choose those that fit best in your context. Here you go: 

Problem -To come up with relevant test ideas, Testers and UX must be on the same page in terms of understanding the problem they want to solve with the proposed design. UX often get first-hand information from the PO about the problems, which testers sometimes are lacking. Trying to understand what problem UX wants to solve with their solution, can open up lots of possibilities for testers and confine them to think in the right direction. It would also spare them from coming up with the right questions and perform better impact analysis of the change. 
  • What product problem are you testing the UX solution against?
  • Is the problem simple to understand or has complexities within itself? Can you break it down further? Ask for more information if that helps to understand the problem better. 
  • Rule of Three (thank you Jerry Weinberg) - If you can’t think of at least three ways the design could fail, you haven’t tested the design enough. What to do about those identified potential failures is another thing. But it would be worth to identify them and at-least discuss those possibilities with UX or PO. (Pro tip - Don't overdo it)

User Needs - 
This is highly subjective and might vary from project to project. The key idea is to understand what User Needs are getting addressed through the design and if you as a tester can foresee any challenges/impact of that. This can also be very well made as a part of 'understanding the problem' part. It does not matter how you do it, but what matters most is that you know what aspects of user needs you are dealing with. 
  • What user needs have been used to design the solution?
  • Do you understand those user-needs in the context of a given problem and your product feature?
  • Based on your product knowledge, do you find the user-needs 'selected' suitable for the designed solution?
  • Is there anything (information around business/technical/cross-team constrains/special scenarios etc.) that invalidates or challenges the user-needs that are taken into account?

User vs Business Needs 
  • What is the goal of the solution provided? Is it for ease of business for products or for improving user experience?
  • Is the change in your product feature adversely impacting other parts of the application (other teams) in terms of User Experience or KPI? - Inform the stakeholders
  • Can you borrow supporting data/statistics from UX/PO to understand their rationale better? - e.g. Why this screen size and why not this? Because we have only so and so % of users using that. And thus the impact is minimal. 
  • Does an attempt of "ease of business" compromise "user experience"?
  • Does improving the "user experience" goal impact business/product KPIs?

Finding Balance - Solving the Oracle Problem 

When you as a tester are unable to decide if the proposed solution is good for the product but bad for users and vice versa, use some of the methods below to make your decision-making more concrete and practical. Based on the configurations of teams, there may be no dedicated UX expert to cater to these needs. It is therefore advisable for testers that they wear UX hats and find the balance by asking the right questions before it is too late. 

Plenty of efforts could be saved if testers too are involved in design/UX decisions early where dedicated UX expert is absent or designs have been made by third party services etc.

What Must Not Change? 

Whatever we ultimately do, what are the things you don’t want to be changed? (from 
System Design Heuristics by Jerry Weinberg)

Introducing design changes in an existing product that are targeted at a specific goal by UX designers, can harm things that are not meant to be affected by that. I highly doubt if there are deliberate efforts made to analyze this regression impact on the design level itself. This is why, if a tester asks this question right in the early phase of a design change, it is likely to save lots of rework for the later. 

“If we don’t start that way, it’s all too easy to lose track of the unchangeable.” - Jerry Weinberg. 

Impact Analysis 
​

What are the visible and invisible parts of the product that are impacted by this change? And what is the impact?
  • GUI process flow : For the end-user and for internal users
  • Impact on user's feelings - Happy? Overwhelming? Confusing? Irritating? Unexplainable?
  • Impact on cross-functional teams ( talk to them early and discuss for a possible solution)
  • When the design is independent of user-needs/user data
      • Basic Rules
      • Usability Ergonomics
      • Heuristic from Jacob Nielsen
      • FCC CUTS VIDS 
    • UX Checklist (chrome extension)

Creativity

Running out of test ideas to decide if the solution is perfect? Pour some creativity in. 
  • Consistency with Comparable products/competition - Is this solution consistent or competent with comparable products/solutions in the market?
  • Inspiration from various fields of study and domains for ideas (not correctness) - 
    • Philosophy 
    • Social Science 
    • Gender, Age and Geographical studies 
    • Medical domain
    • E-commerce Domain
    • Fashion Domain
    • Others?

Exactness, Intuitive and Counter-intuitive Design

Most of the time, testers are more focused on the technical and functional aspects of the product so much that they unknowingly tend to ignore looking for obvious problems. A deliberate attempt needs to be made to look at the product like an unexperienced user to see if they will understand what we expect them to. To do this:
  1. Check text/labels of your menu items, action buttons, information tables, and even terms and conditions - Are they obvious and intuitive? Are they confusing users? Is there more than one possible meaning? 
  2. Is the 'interaction' between components intuitive and obvious? What are the abnormalities? How are they different from common use conventions? 
  3. Are the terms you are using to explain things/menus/actions confusing? Can they be understood by the typical user easily? Are there any cultural constraints that might backfire? How do we avoid misunderstandings? 

Well, I wish I could work on this and develop this idea even further. But for the benefit of time, I would stop here. Maybe I would get back to this again when I have more ideas. But in the meantime, feel free to comment below and share more ideas if you like. And do not forget to tell me how you find the QX idea so far. 

And, I recommend you to read
 System Design Heuristics by Jerry Weinberg. The book gave me lots of ideas to ponder upon. 

Until then...
QX for Testers by Lalit Bhamare
0 Comments

Evaluating and Empowering Testers in Agile Teams

9/10/2018

0 Comments

 
Picture

​Disclaimer:
Purpose of this blog is not to undermine the importance of automation skills for testers but rather to highlight the lack of awareness in our industry for tester's usefulness in teams beyond typical testing and automation tasks. 

​Recently, I was invited to conduct a workshop around "Whole Team Testing" at Test Leadership Congress in New York. Participating in this conference has been a great experience, especially because I truly believe in the need for conferences dedicated to test leadership and career path for testers in changing era. 

The discussions I had there with fellow testers, managers and directors of engineering/testing were interesting. Well, not only interesting but insightful and with lots of ideas, concerns, observations to ponder upon. Out of the things we discussed, what held my attention the most was the growing concern of evaluation of testers in Agile teams. The evaluation can be for hiring or for their overall performance in team. 

What is the problem exactly?

If you analyse the trend from State of Testing report, it is evident that with increasing Agile adoption, centralised testing units are mostly getting dissolved and testers are now reporting to Dev/Team leads. With this change of structure, hiring new testers and evaluating those in teams naturally becomes the responsibility of the Dev/Team leads. And this is where things start to get interesting. How? Let's take a look:
​
  1. Dev leads are usually excellent programers with high passion and inclination towards typical programer mindset. And this is very natural. 
  2. Neither all of the leads come with prior experience of hands-on testing nor all are well versed with testing mindset and philosophy which is key differentiator between  testers and programmers. The way programming excellence comes with deep understanding of programming concepts and ample hands-on experience of coding, it is quite natural that in order to understand testing better (so as to evaluate people for testing skills) one has to have equivalent expertise or at-least deep understanding of the subject. And this is something our industry is still lacking, unfortunately. 
  3. But despite of this all, teams still need testers and they must be hired and evaluated. What happens in such situations? Testers get evaluated based on the scale and criteria, that hiring managers (programmers at heart) are most comfortable with.  Yes, read that as automation and programming skills. This is quite natural, mostly happens unconsciously and I won't blame it on them entirely.

But this situation is dragging us into bigger problem. We are giving too much importance to things that are just a part of big scheme of things that matter and contribute to software quality and teams' overall ability to ship quality software, faster and frequently. 

Of course automation skills and other technical skills are helpful, rather important I would say. But that has become a norm now and what we really need at this point of time are tools to see beyond this norm. If we want excellent testers in team who can really add value to software quality, it's high time that hiring managers come out of their obsession of hiring testers based on the norms that they can most comfortably evaluate. Please, it is not about what we are most comfortable with but what Agile team really needs from a tester to ship a quality software. 

Heavy emphasis on technical skills for testers is not my big concern, but lack of awareness around other things that testers must get evaluated against, concerns me the most.  Purpose of this article is to highlight some of those areas (beyond typical technical skills) that I think managers/leads may want to consider while hiring new tester or evaluating those in their teams. 

Why do we need testers at first place?

One may argue that in an era of AI and advanced automation where every check can be easily automated, why do we need human testers at all? That's interesting argument  and before I explain why we are talking about it,  I want you to have a look over the diagram below:
Picture
The diagram represents a system that we as a team operate into. I created it based on my experience as a tester so far and by interviewing some programmers I have closely worked with. 

Looking at bunch of things mentioned there, you can figure out what all can happen (rather typically happens) when tester in team is not available. If you read the diagram carefully, you will notice that impact on finding and reporting bugs is just one segment that gets highlighted when tester is missing in team. Impact on writing new tests or automating them is another segment. But is that all you need testers for in a team? Certainly not!

The purpose of having testers in team is way beyond finding bugs, writing tests or automated scripts for that matter.    If utilised to their full potential, testers in team can very well serve as a mode to protect your system from running into collapse or explosion mode (please read more about that in "How Software is Built" by Jerry Weinberg). How? It's simple. By providing system related feedback to the controller which is typically team lead or test manager in typical team set-up. The better you utilise tester for their abilities the better feedback you get from them which in turn can help you command better control over your system and protect it from collapse/explosion.  You need tester in team to provide you with information that you as a stakeholder can use for making better decisions.

And when I say feedback, it can be anything from information about bugs, risks highlighting, asking context revealing questions, questioning user stories or decisions made, finding out historical data, highlighting third-party dependancies, sharing news about decisions made by cross functional teams, collaborating with other disciplines to create valuable assets, their observation around team dynamics, their predictions about possible failures and so on. And if you are lucky, they can also tell you some great things about quality of your product.  

The point is, tester in your team can add far more value beyond finding bugs, if you allow them to contribute beyond their typical role. If not, you can always empower them to do so. Here are couple of things that in my experience testers can contribute to and against which you could evaluate them (for hiring or performance reviews etc):

1. Primary analyser of production logs and alerts 

Production deployments are typically done by programmers who then closely follow the logs to see if there are any obvious issues created by latest build. Consider handing over this responsibility to testers. It is likely to serve multiple purpose:
  • Time spent on back and forth of sending build for testing and deployment can be greatly reduced. Once tester is done with their testing, they can directly promote the build for deployment. 
  • Participating in deployment process can help tester understand underlying infrastructure and can offer better technical overview of how those dependancies can affect end to end testing and delivery of your product. This understanding can also help them better analyse the logs and use those for technical investigation of observations made. 
  • Over the period of time when enough insights are acquired, using their sharp observation skills, familiarity with known issues and ability to quickly analyse the gaps, testers can help identify root cause of problems, functional and business impact  as they read the logs and draw inferences from those. 
  • Using the failures and errors via production logs, testers can derive more tests for future and also identify risk areas that were hidden until given point of time. 
  • Participating in production deployments and monitoring can certainly foster better collaboration and pairing between programmer and testers which has benefits of its own. 

2. Enhancers of your product's coverage for quality

It has very little value if tester in your team just sticks with usual acceptance criteria for the ticket and automates the stuff for you to top it. The real value a skilled tester can add is when they question the very product coverage you have in place and help your team see elements of product that matter from quality aspects.

A lot of elements matter when it comes to your product's quality. Functional acceptance criteria is just tiny part of it. A skilled tester would know about these elements and they would educate the team about same.  

Check out SFDIPOT from HTSM by James Bach or simply have a look on mindmap below (thanks to Albert Gareev)
Picture
3. Advocates for Testability 

Testability is one of the key area I would expect skilled tester in my team to help programmers and designers get it right.  A skilled tester would know what makes product testable, how to evaluate it for testability and also advocate the team as in where and how they can improve it. Nothing beats the pleasure of testing better testable product. 

Here is a heuristic for testability if that makes you curious. Out of various types of testability mentioned there, I would expect the tester to at-least know Intrinsic Testability and how to help their team improve it. 
Picture
4. Better allies for UX peeps 

Ever wondered what would happen if System Thinking meets Design Thinking? I firmly believe that challenges faced by testers and UX professionals are more or less same when it comes to ensuring better quality and better user experience. 

A regular and close exchange between these two disciplines has tremendous potential to create a better user experience with enhanced product quality which I would call as "Quality Experience". If UX designer comes with one best solution for some product problem, a skilled tester with their insights, product knowledge, awareness around cross-functional dependancies can point out variety of ways in which it may fail. This does not mean a tester has to criticise UX solutions as such but early collaboration and exchange between two can help avoid lots of unnecessary research and rework. 

On other hand, testers can borrow realistic information from UX's research which can help them design tests that matter and prevent themselves from straying into unwanted scope. Testers can borrow statistical data or interaction/persona based information from UX that can help them shape better scope for their tests. Well, this is indeed interesting topic and deserves deep dive. I will stop here on this one for now. 

A skilled tester would make this collaboration happen and would bring out the best from both the worlds. 

5. Friend in need for marketing and user care 

This is another area of collaboration where skilled testers can add great value to solve team's problems beyond finding bugs. I have listed some of the possibilities where testers can help user care and get benefited in return and I believe same applies to their collaboration with marketing teams:

Picture
6. Alert mechanism when system is on the verge of collapse

Well, this is bit tricky but I would mention it anyway. By nature of their work, skilled testers can use their sharp observation skills to observe and understand people, situations and events around them and can draw inferences that can help identify potential risks, before it is too late. Retrospective meeting is a great place for skilled tester to raise flags for potential issues surrounding team dynamics or people issues they observed, in constructive way of course which is where their communication skills can come handy. 

Testers are usually in touch with fellow testers in organisation from where they get information around what other teams are working on, their future plans, blocking issues etc and this information can be very well used to analyse the impact on their respective team's road-map , work in progress or work items that share dependancies with other teams. 

A tester who is skilled enough to foster network and relations across teams, can certainly help bypass blockages when team needs it most. 
​
So, those are some aspects in my opinion and experience, a skilled tester can contribute value to project team beyond their traditional tasks. 

If you are to a hire new tester or would like to fairly evaluate/mentor tester in your team, I would suggest give these ideas some consideration. It's high time that we look at tester as someone beyond bug finder. They can do wonders for your team and can help to accelerate shipping of quality product, if allowed to use their full potential.

​I shared what I think can be helpful. Feel free to chip in your ideas .... 


0 Comments

    Author

    A passionate & thinking tester. Trainer & student of the craft of testing. Father, Foodie and dog lover. Chief Editor and Co-founder of Tea-time with Testers  magazine. 
    Setting up this blog to share my experiences whilst doing what I do... 

    Categories

    All
    Agile Testing
    Community Of Practice
    DevOps Testing
    Editorials
    Estimations
    Exploratory Testing
    Heuristics
    Hiring
    Interviews
    Leadership
    Lean Coffee
    Life
    Meetups
    Mindmapping
    People
    People And Processes
    Philosophy
    Problem Solving
    Problems Surrounding Testing
    Project Management
    Testing And Programming
    Testing CoP
    UX And Testing
    Whole Team Quality
    Whole Team Testing
    XING

    Archives

    January 2021
    September 2020
    July 2020
    October 2018
    May 2018
    October 2017
    June 2017
    May 2017
    April 2017
    December 2016
    September 2016
    August 2016
    June 2016
    January 2015
    September 2014

    FREE subscription

    Picture
    Tea-time with Testers
    Your Email:

    RSS Feed

© Copyright | Lalitkumar Bhamare | All rights Reserved.
Contact  About  Home
  • BLOG
  • QCSD
  • Tea-time with Testers
  • Press, Publications & Talks
  • About
    • Testimonials
    • Photos