A1QA Blog

QA Core for Executives

Learn Saying ‘No’, You Сan’t Do All Good Ideas. Interview with Steve Rowe

Steve Rowe today is a Principal Test Manager at Microsoft.

What is amazing hе has worked at this global company since 1997 i.e. 17 years (!) which is so rare in the modern world of software testing.

Started on the DirectXMedia team testing multimedia components in IE4 he is currently responsible for Windows Runtime in Windows 8 test development. Previously Steve has also worked on Media Center, media pipelines (DirectShow and Media Foundation), DVD, video rendering, and audio.

In addition, Steve is a curious tester who keeps popular blog presenting his “Ruminations on Computing – Programming, Test Development, Management and More…”. A1QA is glad to discuss challenges and lessons learned by Steve during his long testing career.

Steve Rowe
A1QA: Steve, since you’ve been working for Microsoft for the last 17 years you must be the right person who understands the development and role of software testing in big global corporations like Microsoft. So, please express your opinion if the situation has changed during last decades and testers become more influential in overall software development process?

Steve Rowe (S.R.): The role of the tester has changed a lot since I began. When I started, most testing was done manually. Development would implement some feature and then we would go pound on it by clicking buttons or writing scripts. At some point we realized how expensive it was to have people pushing the same buttons every day. For software like Windows, the testing doesn’t stop when we ship. Windows XP was actively supported for over a decade. That means someone pressing the same buttons every day for 10 years. So we had to innovate.

We moved into a phase where we automated the button clicking. Software Test Engineers gave way to Software Development Engineers in Test. Testing became less about exploring the software and more about writing tools to exhaustively cover the surface. This is an expensive proposition up front, but the cost can be amortized across the life of the software. This had the side effect of distancing the tester from the experience of the user. It was easy to write software that was correct, but hard to use. We had to innovate again.

The newest phase of testing seems to be about reconnecting to the user. The internet allows a real time connection with users. No longer do we have to try to envision what a user might do, it is now possible to observe what they are actually doing. Data science and machine learning can provide a lot of insight about what is working well and what needs to be fixed. A/B testing can be used to help determine what solution works best. Websites and services figured this out first. Big, packaged software is late to this party, but is starting to come along. I suspect testing will do much more in this direction over the next several years.

A1QA: This is interesting you work for one employer so long which is not so common in the modern testing world. What impresses you the most in Microsoft? What is the most important lesson you have learned while working there?

S.R.: Microsoft was one of the earlier companies to really embrace testing. They developed the SDET role and really supported it. At some companies test or QA is relegated to a second-tier status. Not so at Microsoft where we had an equal seat at the table. The willingness to embrace testing, to put some of our best and brightest on the task, and to provide a full career path for test is one of the things that impresses me about Microsoft.

As for what I’ve learned. One of the most important lessons is the power of saying ‘No’. Carefully consider what you can do. Plan for it. And then commit to only that. There are too many good ideas to do them all. The criteria for accepting new work can’t be, “Is this a good idea?” Most of the time that answer will be yes. Instead it has to be, “Is this better than what we’re already doing?” If it is, take off the lowest priority items of your plate and add this new thing. Always clear space for the new work. You won’t magically be able to find time in an already full schedule.

Overcommitting may feel like the right thing to do. It may feel like it will get you ahead because that is what your manager wants, but over the long term, it will hurt you and the product. You’ll work really hard and still ship late. It is best to set realistic expectations up front.

A1QA: As a software tester you are working with Windows 8 generation, which includes thousands of new APIs from new UI controls to new ways to interact with the devices. How was Windows 8 different from other projects you’ve worked on and what has been your biggest challenge that you had to solve?

S.R.: I’ve worked in and around Windows for my whole career at Microsoft. I started working on DirectXMedia and DirectShow which were the multimedia APIs in Windows 98. For Windows 8 we created a brand new type of application, the Windows Store app, and a whole new API surface for it called Windows Runtime. The magic of Windows Runtime was the ability to write an API once, in say C++, and then have it automatically projected into three different languages: C++/CX, C#/.Net, and JavaScript. This means three times the surface to test. This projection was free. As in free puppies. It didn’t cost much up front, but it was a lot of work.

The challenge was figuring out how to test all three projections without hiring 3x the number of testers. We did this through better policies and better tooling. We determined what kind and how much testing needed to be done in each language. We wrote tools that could scour the API surface and catalog it and other tools that could determine what had been tested already and what had not. We wrote tools that leveraged the metadata and automatically wrote tests for the APIs. We had a dashboard that could show us the status of every API in the SDK so we could find the weak points and shore them up.

Thanks for being our guest and for sharing your viewpoint. We will be glad to see you and talk to you again.

Follow Steve on Linkedin and read his blog.

Share this: