Never let a metric go unsupervised. Interview with James Bach
With this blog post a1qa starts interviews series with famous persons of software testing and quality assurance industry. James Bach honored us to be the first one. He is the expert who speaks at prestigious conferences, and an ardent trainer who strives to bring benefit to his students.
James Bach is the person for whom Testing is not only a job … it`s his life. Previously working for Apple and Borland; now he is leading Satisfice Inc., a teaching and QA consulting company. James is the one who is behind of Exploratory Testing concept and the Context Driven School of Testing. His testing blog breaks all records of popularity over the globe. We are delighted to host James in our blog; see James`s blog.
a1qa: James, thank you very much for being our guest. The last post in your blog was about measuring testing quality. It’s not a secret that a developer and project manager usually want to know how good their code is. And test activities often result in test report from responsible QA engineer with a quality analysis and a quality estimates. How do you estimate quality? Is that verbal, numeric attribute or instinct feeling for you?
James Bach (J.B.): All reasonable people assess quality of software the same way: weighing the evidence in their minds. It’s what juries do, it’s what voters do, it’s what political and military leaders do, it’s what everyone does when assessing anything complicated and important. Anyone who tries to specify software quality substantially in terms of numeric attributes is either joking, incompetent, or a fraud. Management is primarily responsible for weighing the evidence about the product. There are no metrics that can take this responsibility away from management. If they attempt to shift their responsibility onto metrics then responsible people must resist that.
My job as a tester (I prefer not to say “QA”) is to collect, arrange, filter, and present the evidence that management needs. I might use metrics as part of that evidence. What metrics I choose is a decision that is completely dependent on the situation. However, I use metrics with skepticism. Numbers are too attractive to people who want decisions to be simple. I never let a metric go unsupervised into the hands of management. And I almost never use any metrics associated with test cases (there are a few, rare exceptions), since they mean nothing. I never reduce quality to any set of numbers.
a1qa: What are your expectations with regards to QA and trends you foresee in testing? Will test automation ever replace manual QA?
J.B.: Skilled testers have always been a tiny minority. Now, at least, there is good international community to support the growth of skilled testers. But the community of serious testers is under constant attack from different quarters. Some Agilists are bored by testing and want all testers to be programmers. Many old factory-style testing consultants are threatened by the new wave of intellectual testers, which is why they have conspired to create the new ISO testing standard that was obsolete long before it was even officially created. Certain large companies famously deny they even have testers.
I think that the days of poorly skilled testers who follow test cases has largely passed. Those people will get less and less work.
As for automation, well testing cannot be automated. There is no such thing as automated testing. I realize that your company publishes blog posts that speak of automation, but that’s not testing– that is fact checking. Unless fact checking is guided and designed and managed by people who know how to test, it will remain an expensive waste of time. Today, most of what is called test automation delivers very little value, relative to the cost.
I am a tester. Of course I’m interested in facts, but I also have the skills to analyze and interpret evidence. I know how to design experiments. Those skills cannot be automated, although I do use tools to help me test.
a1qa: QA to Dev ratio as 1 to 3: do you believe in it? What’s your advice on QA team size? What factors do you consider when setting up a team?
J.B.: Tester/Developer ratios don’t mean anything.
My advice is to hire or train fantastic testers until the critical testing work is getting done quickly enough. I generally start by hiring one tester and go from there. This is the same way developers are hired.
One factor that matters to me in building a team is diversity. I want different kinds of people, with different temperaments and educational backgrounds.
Another factor is service. Testing is a service role. People who test should embrace that. Testers, like lawyers and doctors, are not in charge of their clients’ lives. Instead they serve them. A tester who doesn’t enjoy being of service should go into some other line of work. A tester should be thinking “how can I help?”
a1qa: Who would you recommend to consider QA as their career? How would you recommend to train newcomers and senior engineers? How to learn new if you are really long in this sphere?
J.B.: I would not recommend any career to anyone. It’s like recommending that two strangers get married. They should fall in love, first. I like testing. It’s good for someone who likes to solve puzzles and work on different things each day.
I cannot quickly tell you how to train testers. I’ve worked many years on that, and there is too much to say. I will make one suggestion: do not get ISTQB certification. It’s a waste of time and it merely gives money to a corrupt organization.
a1qa: Looks like there is no an event with Agile, Scrum, or Kanban being discussed. So Agile for you is a silver bullet or pop trend? What is your strategy in selecting approach?
J.B.: Agile is a fairly vague collection of aspirations, habits, and expectations. Skill, motivation, and social relationships are what make projects succeed.
I’m a problem solver. My way of selecting an approach is to understand the problem and my role and my available tools to solve it. Then I do what solves that problem. It’s called the Context-Driven way.
a1qa: Thanks a lot, James, for sharing your insightful views! Obviously, one post is not enough to discuss all topics of the interest. Hope we will have an opportunity to talk more and meet in the future.
Reach James Bach on Twitter.
