Blog

Encouraging exploratory tests. Interview with Alan Richardson

Exploratory Testing requires each tester to learn how they currently test, identify how they can improve how they test, and how they can apply their unique skills and experience in their exploratory testing. It is a uniquely personal approach and as such ‘mindset’ offers a far better word to encourage ownership of the testing process than methodology.
8 January 2015
Interviews
The article by a1qa
a1qa

Alan Richardson has more than twenty years of professional IT experience, working as a programmer and at every level of the testing hierarchy from tester through head of testing. Author of the books “Selenium Simplified” and “Java For Testers”. Alan also has created online training courses to help people learn Technical Web Testing and Selenium WebDriver with Java. He works as an independent consultant, helping companies improve their use of automation, agile, and exploratory technical testing. Alan posts his writing and training videos on SeleniumSimplified.com, EvilTester.com, JavaForTesters.com, and CompendiumDev.co.uk.

Alan RichardsonCROP (AlanCrop)– Do you agree that exploratory testing is more a mindset than a methodology, as James Bach and Cem Kaner say?

– I do agree that it is more a mindset than a methodology. I can’t point at any mandated ‘rules of exploratory testing’ document in the practices of QA outsourcing and say “only if ye do such that is written here shall ye be of the Exploratory”.

Exploratory Testing requires each tester to learn how they currently test, identify how they can improve how they test, and how they can apply their unique skills and experience in their exploratory testing. It is a uniquely personal approach and as such ‘mindset’ offers a far better word to encourage ownership of the testing process than methodology.

Methodology suggests that ’some other’ authority knows best. Exploratory Testing, with its emphasis on learning from the actions and testing that you personally take, can’t fit into that schema.

Those testers I know that excel at Exploratory testing, do so, because they have made it their own. They have looked around at the information that other people have shared about exploratory testing. They have tried it. They have analysed what worked for them and what didn’t. They have adjusted. They have incorporated techniques, approaches, analogies, and skills into their testing that they recognised as valuable to them. And they continue to experiment with, and explore, new approaches and nuances.

Exploratory testing is a learning by doing approach. Therefore we should expect everyone doing it to learn something different and approach it subtly (or sometimes radically) differently.

When I talk to testers I very often talk about taking responsibility for their testing. And by that I mean owning their approach, so that they can justify it using their words, and from their experience, and from the experiments that they have conducted during their testing. I think that is how you build experience, attitude, learning, belief systems; and all of that, I can see falling under the banner of ‘mindset’.

– Selenium WebDriver is a tool that immensely accelerates testers` work. Still, what are the main difficulties that users face while applying the tool?

– I think that Selenium WebDriver offers testers the best open source browser automation library available today. And that when a robust set of domain specific automation libraries have been built by the team, it can allow team members to rapidly automate the functional flows through their application. And that allows teams to build sets of methods which they can use to check the application for unexpected deviations in behaviour. They can also rapidly build up adhoc code which exercises the application until the application achieves a specific state that we can manually test from. The team can also more easily add data into the system, and generally automate flows through a web application.

Common difficulties that people have when they start using Selenium WebDriver is viewing it as a tool. I view Selenium WebDriver as a library, and like any other library that we use when programming, we have to know how to program to be able to use the functions in the library. So programming knowledge is required. And that can prove a difficulty for people.

The Selenium eco-system does provide tools; like the Selenium IDE, or Selenium Builder. Which allow recording of actions with the automatic generation of scripts. But unless you know programming, you don’t know how to effectively tailor those scripts, and wean yourself off the recording tool.

Other difficulties people have, relate to the web technology itself, simply because there is a lot to learn if we really want to master automation.

Because we use Selenium WebDriver to automate the browser, we have to understand some of the technology associated with how browsers work; the DOM, CSS, JavaScript, etc. We have to understand how the specific pages we are testing work, so that we know how to synchronise effectively with the state of the page so that our automation works consistently.

So I find that, over time, to effectively work with Selenium WebDriver, we have to learn: programming, CSS selectors, HTML, DOM, dynamic HTML population, synchronisation strategies, browser developer tools, software design, continuous integration, environment maintenance, etc. (the list could go on).

I don’t think these learning topics are unique to Selenium WebDriver. I think they are important for any browser automation. But I think some automation technology and tools use a sales tactic of pretending that their users will not need this learning, but they will.

You can achieve good and effective results with minimum levels of knowledge in these areas, but to really make your automation work, you may have to learn things that other testers, may not immediately need to learn.

I could continue to list many more things that people find difficult because I do receive a lot of emails from people learning Selenium WebDriver and the issues they are having.

Very often someone on the web has already faced, and identified a workaround to, their issue. So sometimes the difficulty people have is that their google-fu isn’t quite effective enough to find the existing answers out there on the web that will help them.

– You do a lot of webinars. Is it your pass

– I need to do more webinars. I don’t do enough of them.

I like to share experience. Partly because I learn when I do that, because I have to codify it in a different way to communicate it to others. Partly because you can reach more people over time, with a webinar, than through a conference talk.

A conference talk will reach several hundred people, at a single event, and that is it over, done. I might release the slides and a blog post, but it doesn’t have the power of the presentation behind it.

With a webinar, and online training, I can reach thousands of people. And they can encounter it at the point when they are ready for it.

We also give the viewer more control: they can pause it, rewind it, or skip sections if I’m wittering on about something unimportant to them. Webinars and videos put more control in the hands of the viewer, than in the hands of the presenter, and that allows the viewer more control over their learning process.

Also the commenting systems online give people the opportunity to think about questions, ask them whenever they want to, and have more of a discussion than the conference format often allows.

Webinars also allow me to ‘demonstrate’ stuff more than I can in a conference talk, and I think it is very important for people to see things in action. Because it shows what is possible, and that can help change a belief or a mindset about what ‘testing’ means, or about what is acceptable in testing. And they can model the behaviour shown and achieve the same result.

And I guess I’m passionate about that.

– Will it be beneficial for the workflow, if team combines exploratory testing with agile techniques?

– Exploratory testing allows us to test at speed, often with minimal information, it encourages learning through investigation and conversation, we can collaboratively make decisions about which risks to explore in detail and which to perform less work on. Every Agile process I’ve worked in, has required the associated testing process to have those attributes.

I have worked on projects where people have expressed the opinion that the Acceptance Criteria on a story is good enough to assess the completed code against. So the team then build confirmatory Acceptance Automation – possibly using BDD, or ATDD, or by building Gherkin feature files or xUnit tests to cover the criteria. And when they all run green the team consider the Story done.

This is a confirmatory process.

We also need a questioning process. “Did you mean X or Y?”, “Did you only mean X?”, “What if Z happens?”, “What about…?” etc. etc.

Some of those questions may occur during the discussions as the story is coded, and some of the answers may even be added as Acceptance Criteria.

But I’ve seen too many stories where questions were not asked.

Even when the questions were asked, the answers weren’t written as acceptance criteria, and so the confirmatory automation did not cover them, and sometimes the functionality didn’t cover the criteria effectively, but no-one knew until the software was released.

An exploratory testing process can ask those and other questions.

But the main reason I think we have to combine exploratory testing with agile techniques is because all testing is exploratory. So if you want to actually have testing on your Agile project, you have to explore.

– Alan thank you for sharing your ideas and expereince. we hope to talk to you again.

More Posts

10 March 2020,
by a1qa
6 min read
Dedicated team model in QA: all you should know about it
Check on everything you should know about when to apply, how to run and pay for a dedicated team in QA.
Interviews
QA consulting
Quality assurance
30 September 2019,
by a1qa
4 min read
“Every team member is responsible for software quality”: interview with Head of QA at worldwide media resource
We continue talking about unsurpassed software quality. Consider how to make QA more efficient using shift-left and continuous testing.
Interviews
8 December 2017,
by a1qa
4 min read
a1qa: one-stop shop for first-rate QA services
Dmitry Tishchenko, Head of a1qa Marketing and Pre-Sales Department, answers the questions of The Technology Headlines. 
Interviews
Quality assurance
17 August 2017,
by a1qa
4 min read
From requirements specification to complex business analysis: interview with a1qa head of BA
Check how we at a1qa converge business knowledge with IT skills to deliver maximum value. 
Interviews
QA consulting
1 August 2017,
by a1qa
4 min read
Interview with head of a1qa test automation center of excellence
Dmitry Bogatko on how to manage the in-house Center of Excellence delivering value to the company's projects. 
Interviews
Test automation
19 August 2016,
by a1qa
4 min read
Interview with Adam Knight: Big Data exploratory testing
It is not so much to say that I find exploratory testing necessary. Rather I would say that I found it in my experience to be the most effective approach available to me in testing the business intelligence systems that I have.
Big data testing
Interviews
5 August 2016,
by a1qa
5 min read
Interview with Adam Knight: how much of a Sisyphean task is in software testing?
I’m a great believer in automation. I don’t believe that an agile approach to development is possible without some level of test automation. The use of such approaches does, however, need to be combined with an appreciation of the information that the automation provides you with.
Interviews
Test automation
20 July 2016,
by a1qa
4 min read
Good testing is about asking right questions: Intereview with Thanh Huynh
Software testing is not just to confirm things, it’s a process to explore, exercise the system to discover potential problems. You can achieve that by asking good questions to the system under test, yourself, your customers, your product owner, your manager, your colleagues, etc.
Interviews
20 June 2016,
by a1qa
5 min read
Interview with Lisa Crispin: whole-team approach to quality
To succeed with delivering valuable software over the long term, the whole team must take responsibility for quality, planning and executing testing activities. Our mindset has to shift from finding bugs after coding to preventing bugs from occurring in the first place.
Agile
Interviews

Get in touch

Please fill in the required field.
Email address seems invalid.
Please fill in the required field.
We use cookies on our website to improve its functionality and to enhance your user experience. We also use cookies for analytics. If you continue to browse this website, we will assume you agree that we can place cookies on your device. For more details, please read our Privacy and Cookies Policy.