Interview with Adam Knight: how much of a Sisyphean task is in software testing?

Adam Knight is a passionate tester eagerly contributing to the testing community. He is an active blogger and constantly presents at such testing events as Agile Testing Days, UKTMF and STC Meetups and EUROStar.

Hi Adam, can you please tell us about yourself, your current occupation and expertise?

For the last couple of months I’ve been working as an agile consultant establishing Product Owner role in a company called River. It specializes in employee and retailer engagement solutions.

Prior to that I spent a number of years working on a database product called RainStor, where I was responsible for testing, technical support and technical documentation as well as performing many of the day to day product owner duties.

You have a considerable experience in testing. How has testing changed since you started your career?

The main change that I see is the visibility of software testing outside of the organization that testers work in. When I started working in IT, testing was very much perceived as something done by people drafted in from the business, rather than necessarily being a valuable and skilled occupation in its own right.

The testers were often very isolated individuals or teams within their organizations, relying on strict formal doctrine and rather dry reference books. The advent of social media has had a massive impact on testers in allowing the creation of groups and forums in which people can discover and communicate with other people doing similar jobs.

In response to this question I could talk about changes such as new techniques, new automation methods and new tools, however people have been creating tools to solve testing problems for years.

What has really changed the face of testing more than anything else is our ability to share these tools and ideas more easily with others.

Some experts say that in future almost all manual testing should be automated. What do you think about this trend of increased automation?

I’m a great believer in automation. I don’t believe that an agile approach to development is possible without some level of test automation. The use of such approaches does, however, need to be combined with an appreciation of the information that the automation provides you with.

Essentially most automation is a detection system, alerting if some unexpected outcome arises from a check performed after a series of actions. Somewhere a human is still required to design relevant alerts, understand the implications of those alerts being triggered and take the appropriate course of action should that happen.

One of the principles that I hold to with test automation is that the automation should do more than just perform the checks. The automation provides an opportunity to gather information on the activity as it is executed which can help a human to investigate and diagnose should the checks throw up an unexpected result.

Is it possible to achieve 100% test automation?

If you don’t do any other tests than automated ones, and you exclude a huge amount of human critical activity from your definition of testing, then you could argue that it is possible.

It would be a weak argument however given the fragility of the assumptions around it.

No matter how much automation you create, the process of creating automated checks is a human sapient and often exploratory testing activity which in itself is not automated.

Something that is apparent to anyone working in testing, but difficult to explain to some outside of testing, is that 100% testing is not possible.

Even on the simplest of software applications there will be an infinite range of possible tests that we could perform. The amount of testing we therefore undertake is some finite subset of this, and 100% automation would only ever be 100% automation of this subset.

A great number of organizations are currently adopting agile methods and DevOps principles. How will testing change in the nearest future?

Roles are blurring. I think that we are moving away from the idea of well-defined roles. Both the agile methods and DevOps that you mention have in them an expectation of individuals with a range of skills and capabilities, rather than viewing development as a factory production line with a series of very separate roles owning different stages of the process.

I’ve never been particularly good at seeing future trends, however I see testing and product ownership responsibilities merging, with testing providing the information to allow the product owner to make educated decisions.

What are the main benefits of agile testing?

I would suggest that a really good question would be ‘what are the main benefits of agile to testing?’

Agile approaches have huge benefits to testing. The short timeboxes mean that it is hard to accumulate a huge ‘ball of mud’ development over the course of many weeks before handing it to the testing team and so testers are able to test incrementally on smaller pieces of development, making both discovery and diagnosis significantly easier.

The main benefit of an agile approach for testing in my opinion lies in the point at which testers are included in the conversation.

The “Three-Amigos” is a phrase that encapsulates these benefits. The fact that we have a phrase in common use that exemplifies the need to have testers involved in early conversations on the scope and value of new features, is huge.

How do you see the ideal testing process of future?

I think that far too much time is spent by testers on simple functional testing. I think that the ideal testing process of the future, or even today, involves the developer taking responsibility for the testing of the functional adherence of their developed software to the model that they are working to.

The role of the specialist tester can then be about validating how this solution model maps to the problem domain, what situations or events could cause the solution to fail, what environmental factors could result in the solution to fail, what software factors could cause the environment to fail.

I called my blog “A Sisyphean Task” because many folks perceive software testing as such, performing the same activity over and over again just as Sisyphus was doomed to do rolling a rock up a hill in Greek mythology. Where the interest, and value, lies for me is when we have sufficient confidence in our basic functionality to start exploring the deeper characteristics of the software and its operating environment. That’s where the really interesting testing happens.

What sources of information can you advise testers, who want to develop their practical abilities and technical skills?

For years I’ve recommended looking to other roles in the organization to learn skills from. I’ve learned a huge amount from DBAs, System Administrators, Security experts, network guys and support teams, not to mention developers.

People also need to be proactive. Most of the new practical skills that I have developed in the last few years have been the result of spending my own time researching interesting new projects.
I learned Ruby scripting through developing a SQL generation tool in my evenings. I learned Java in my own time to create a multi-threaded test harness.

If you see a need, take some time to research a solution yourself. Then take it to your manager and demonstrate the value of what you have done. Not only will you develop new skills but you won’t do your career progression any harm either.

Thanks for sharing your viewpoint with us.

If you want to learn more from Adam, visit his blog.