Opening

We often hear the words ‘great culture,’ and it is emphasised in many QA companies’ websites and job specifications.

We all want a great culture, but do we actually understand it enough to maintain or improve it?

In this article, I will be sharing all of that.

What is culture of happiness?

I like to think of it like the foundation of a house — without a strong one, there will be no stability to it, and it certainly will not last. It is the key element by which everything else is added, such as a kitchen, bathroom, bedrooms, study, and garden, to name a few.

Culture takes place whether you want it to or not. It is the nucleus of a QA company and is in large part created by the founders and all employees — not by their words alone, but more so by their actions.

Strategy is key in business, but without the right culture, any strategy will fail.

Culture is led by beliefs, attitudes, and practices that people are exposed to when they interact with your business.

Culture, or lack of it, is infinitely more important and could kill your strategy before it even gets off the ground.

Strategy and culture need to mutually support each other. A bit like in a car — no matter how good the parts are, without wheels, you will not be moving very far.

Why does culture matter?

Think back to your past roles that you have enjoyed the most — I’m sure the reason was due to an amazing culture. The opposite applies for roles which you did not like.

There is evidence that good cultures create a feedback loop that often self corrects bad strategy, and also that good strategy gets consumed by bad culture.

From my experience, a better culture helps to create a higher chance of great delivery output — take a happy environment generally, there will be learning, and all the software testing team will be moving in the same direction.

I have seen this in action, and for me it was due to the crew being synchronised and working to meet a common goal. If a team works together, there’s a bigger chance to meet our objectives.

What impact does bad culture have?

A bad culture normally creates fear in the QA team; thus, they generally will be scared to innovate and will make more mistakes. Another big impact — people will not want to work within the company in the future. Or ever come back.

People do not leave the company; they leave the culture.

QA culture also hits revenue; the best performing software testing teams generally are those performing the best long term. In my view, squads who have a bad culture and are still successful, will only maintain high performance in the short term, or they may not actually be meeting their maximum output.

What should your company’s QA culture include?

Remember that culture is more than just creating a great place to work and some words in your mission statement.

The culture should determine what is encouraged, discouraged, and  acceptable.

It includes:

• Adaptiveness to change — in business, we need to be flexible to adapt quickly.
• Innovation — making great new software products to keep ahead of the competition.
• Risk tolerance — how much risk are we willing to take?
• Decision making — being brave to take decisions, even when they may fail.
• Efficiency — ensuring that we are efficient in our output.
• Customer focus — how is our culture driving what a customer wants?

Culture starts at the top of a company, and each layer below should be aligned with this.

What can you do to improve your company culture?

• ‘Check the pulse’ and understand your culture.
• Foster conversations and ensure it is a safe space to talk.
• Map your values and purpose.
• Celebrate success together.
• Culture is created by behaviours that you tolerate.
• Change starts at the top — company leaders show how they care about culture.

A leader recognises they are ‘a voice’ around the table, not the ‘voice.’ They allow others to say how culture can be improved and also listen to any concerns within software testing teams.

For me, companies that are culture conscious outperform industry benchmarks. What makes a business unique is its culture and it is often used as a competitive advantage.

I like to think of culture as a tailwind for a travelling plane, it pushes a plane forward and if stars align, you will make it to your destination faster, therefore, it adds hidden value.

Closing

As a leader, we can all impact the culture of the company and move the lever. Though, it takes patience and perseverance.

The world is changing, and the same goes for technologies. You can hardly imagine the software that doesn’t need improvements and updates, can you?

But how to manage constant modifications confidently while delivering a competitive IT product? And that’s where continuous testing comes in to support businesses in boosting apps quality and winning the high market race.

In the article, let’s discover…EVERYTHING about continuous testing: perks it provides for companies, reasons to adopt it, and 6 core steps to make to smartly integrate it into business processes.

Shaping a complete picture: why do companies need continuous testing, and what do they get in return?

These days, we are witnessing a mass-scale adoption of flexible methodologies. The 15th State of Agile Report indicates that 94% of surveyed companies are practicing Agile while 74% of respondents are implementing DevOps. What are the reasons behind these high rates? Constant changes. In the market, across the software development process, and end-user behavior patterns. And Agile and DevOps perfectly assist in addressing all of them.

However, businesses want a bit more than just producing a top-notch IT solution. What if the goal is to deliver high-quality products at a fast pace with innovations at the core? Then, it’s essential to integrate continuous testing (or CT, for short).

The common question is, “How can the company understand that it’s time to adopt continuous testing?” Here, there are 3 critical factors to bear in mind.

Factor 1. Working in a highly competitive market

For several decades, the world has been in the constant flow of technological change, with many industries experiencing disruption — from streaming companies to hospitality and automotive. With more organizations applying AI, IoT, AR/VR, and other technologies to boost customer experience and hold the attention of picky end users, remaining competitive has come to the fore. Adopting CT helps test early, often, and faster, delivering quality software.

Factor 2. Following regular release cadence

Amazon deploys every 11.7 seconds, and other companies attempt to reach the same results. Continuous testing, in its turn, allows keeping up with this pace, significantly reducing the testing cycle time and helping deliver high software quality at speed. At really high speed!

Factor 3. Striving to enhance software quality

To increase the odds of success in the market and deliver IT products that attract end customers, companies continuously do their best to improve software quality. CT allows them to establish a holistic quality assurance process by fulfilling tests throughout the entire SDLC, from the stage of early planning to deploying to the production environment. This means meeting the high software soundness, performance, and security.

It’s true that before introducing a particular service, companies need to know exactly what values it may bring to them. Let’s take a look at the 5 strongest benefits of continuous testing.

1. Early mitigated risks. Timely feedback throughout the SDLC assists in promptly identifying blocking defects and eliminating them before the preparation of code.
2. Smarter software release. Within Agile and DevOps, companies build an extremely flexible software development ecosystem with continuous release cadence.
3. Effective QA workflow. Continuous testing allows shifting left and right at any SDLC stage while performing required QA activities at the necessary time.
4. Boosted user experience. Timely performing test activities and conducting test automation throughout the SDLC keep the software code under exceptional supervision and prevent fault leakage. So, it helps improve CX, delight end-users with high-quality new functionality, and avoid business decline.
5. Better teams’ integration. When evaluating the quality state from the very start of the project, all project members share QA value (as the time of siloed QA teams has passed). This streamlines effective collaboration, minimized downtimes, and high-grade code in production.

Building the 6-step plan of continuous testing

Now, let’s proceed with the steps to make to introduce CT with confidence.

Step 1. Conduct exploratory testing

Yes, test automation is an excellent choice for those wishing to check whether code changes haven’t affected existing functionality. But test automation alone can’t truly define if the features live up to set expectations.

It’s vital to perform exploratory testing as a first step in assessing the quality of a product. It helps identify issues that are hard to notice during other software testing approaches, simplify the cooperation of the cross-functional team, and find issues before introducing automation.

Step 2. Prioritize risks and perform test design

Modern software testing solutions are characterized by increased complexity due to multiple innovations at their core. And often, the costs are too high to allow even a single defect to leak into the production environment as the software may be embedded in a medical device intended to save someone’s life, for example.

So, to detect issues quickly, it’s critical to define business risks, prioritize them, and create the right test coverage that you’ll be able to gradually increase to reach 90% and more.

Step 3. Introduce test automation

Test automation works miracles when it comes to improving QA efficiency and accelerating time to market without compromising on software quality. The latest World Quality Report shows that it also provides businesses with reduced operational costs (47%), enhanced detection of defects (49%), mitigated risks (51%), and many more.

Source: World Quality Report 2021-22

However, to get the greatest value from test automation efforts, it’s significant to do 3 things: define the right toolset based on your business needs and budget, teams’ capabilities; set up a stable approach to have real-time visibility into the current quality level; expand the coverage over time.

Step 4. Prepare the infrastructure for CI/CD

CI/CD pipelines are a core part of DevOps practice as they help boost project velocity, minimize manual efforts, and reduce the probability of errors during integration and deployment stages.

To release reliable software, companies should prepare their infrastructures for CI/CD by organizing pipelines, integrating them into the communication channels, and performing service virtualization.

Step 5. Establish procedures (related to processes based on minimizing business risks)

By following a sequence of steps, companies standardize procedure flows. It minimizes the probability of defects slipping into the production environment and decreases the QA budget required for their fixing.

For example, before rolling out a new feature, it’s vital to define business risks and create test coverage, consisting of step-by-step procedures. They include defining the scope of unit tests, component tests, and finally, end-to-end tests. It also allows easily tracing the roots of any problems by analyzing them during retrospectives.

Step 6. Measure progress

It streamlines determining the progress and getting insights into which parts of the implemented solution may work better and improve them. Here, you may analyze the influence of your actions, reanalyze the risks, and conduct a retrospective to achieve greater success.

Wrapping up

So, what do we have? The key factors for adopting continuous testing are working in a highly competitive landscape, following regular release cadence, striving for software quality enhancement.

About the benefits. With CT, companies mitigate risks early on, provide smarter software releases, enhance QA workflow, boost user experience, get better teams’ integration.

And that’s not all. Companies may follow a 6-step plan to implement it smartly and confidently.

Reach out to a1qa’s experts to get a personal consultation on adopting continuous testing.

Within growing popularity amid end users, AR/VR have emerged as mediums providing new ways to implement a range of business solutions and helping modify both online and offline shopping experiences.

What to consider to smoothly introduce AR/VR innovation into retail business? This is where forward-thinking retail players rely on software testing while preventing possible issues like customers’ outflow, revenue decline, unstable IT products, and many more.

Let’s take a closer look at the infographic that shows what digital reality tendencies such organizations launch and how they navigate the challenges through AR/VR testing.

You can download the infographic here.

Final note

The future has already come in the retail industry within AR in-store navigation, digital displays, smart dressing rooms, and VR shops. However, companies still have hesitations on implementing AR/VR by experiencing high expenses and time-consuming app support.

Is it a race for being trendy or it is a real need to move on to the next level of doing business? You decide.

To keep up with customers’ expectations and introduce high-end technologies with confidence, organizations apply AR/VR testing. Alongside ensuring impeccable quality and stable operation, QA helps enhance CX, increase conversion rates, and reinforce end users’ loyalty.

Need support in introducing AR/VR technologies into retail software? Get hold of the a1qa team.

In line with a highly competitive market, businesses have their eye on digital experts to effectively promote their brand, turn to programmers and QA specialists to deliver a top-notch application under tight deadlines.

Cooperating with QA vendors, organizations not only get an experienced team, but also reap numerous benefits: concentrate on core activities, optimize costs, increase operational efficiency, and much more.

However, besides brought success, it shares potential risks and responsibility for achieving goals in delivering flawless software.

The process of choosing an entrusted partner is difficult, even when you have a solid plan for further activities. In this article, I will share tips on how to pick an infallible QA provider and how to start this prosperous cooperation.

Is it worth onboarding the distance working team?

Every organization possessing an IT solution asks the same question. “Should we hire in-house QA engineers and provide hands-on technical training, or transfer tasks to an outsourcing team that integrates into the process and delves into the software hallmarks?” There is no right answer, as it all depends on the peculiar requests of a company.

• If it aims to nurture an internal department, the decision will be made in favor of its in-house team.
• If the goal is to reduce costs on creating and maintaining a novel direction, it will turn to an independent vendor.

Collaboration with a QA provider brings a formed group of specialists that are selected according to business needs. Therefore, it saves costs and time on human resources.

Outsourcing a dedicated team allows setting non-key processes (like software testing for a bank, for instance). Also, QA specialists’ commitment and narrow-focused competence push the envelope to new initiatives.

Since the organization has an IT solution, its development process divides into cycles and workload distribution is likely to be unbalanced. Sometimes QA engineers have to wait for the next build so that they stay out of work. After a while, they are overwhelmed with tasks and run out of time to perform all the necessary tests. Outsourcing helps companies flexibly control the amount of work and onboard specialists on demand.

Moreover, even if you are confident in your software product, occasionally it needs an independent assessment to identify the shortcomings and improve quality.

Selecting a QA vendor

The provider’s investigation lays in the heart of the successful partnership. Try to figure out all the necessary information. It might be the size of a company, its market experience, portfolio, and much more.

What is more important, pay attention to the official website. It is rather unreliable to trust a company, which can’t establish a satisfactory operation of its own digital space, right?

You may also have a look at the reviews from its previous clients on the independent platforms, like Clutch. It helps form a more comprehensive picture of the vendor.

Don’t forget about a portfolio. There you can find success stories from diverse spheres and make sure that the organization has experience in similar projects.

Many outsourcing companies also offer pilot projects so that a team demonstrates its skills in practice. It is the most suitable option for clients to check flexibility and team scaling on demand to assess the results and decide on further cooperation.

Integrating the outsourcing QA team

Once a company has chosen a QA provider, the next step is to think over a team integration in the current project infrastructure. Continuous communication is a silver bullet. It might be more effective to use shared online tools: file storage, messengers, video conferencing platforms, etc.

Identifying the responsibilities of each member helps everyone know what to do and whom they can contact in case of an emergency. To synchronize processes, regular meetings can be established.

Knowledge transfer is one of the keys to business success. However, many companies do not have a clear plan and holistic insight into it. a1qa’s experts suggest the following algorithm at the start of cooperation:

1. Compose a plan. The order of joint actions should describe the business process that is delegated to a remote team. So, you can detect potentially vulnerable areas that need close attention from the side of a QA provider.
2. Ensure minimal impact on business-critical operations. Knowledge transfer shouldn’t disrupt business activity. If a company shifts the responsibility of a process to an outsourcing team, then in-house specialists should be fully involved in their core charge.

How to provide efficient communication

Both customer-company and vendor-organization should work together and find an optimal cooperation format. Members can communicate in a real-time mode, every day, every two weeks, once a month, or less frequently.

Accustomed platforms and messengers are suitable for solving simple or non-urgent tasks. It is convenient to keep in touch with each other within one or two tools when all the information is concentrated in one storage. Such communication can be supplemented by weekly or short daily calls.

Retrospectives summarizing interim results mark the end of each sprint in Scrum. The client’s dedication helps form a clear vision of current priorities, and the team receives timely feedback.

When leveraging Scrum or SAFe, companies should adhere to the following principles:

• Incrementally plan participants’ actions
• Maintain continuous communication
• Meet deadlines
• Scale up and down the team on demand.

Scrum planning can take a few hours, while the cycle in SAFe usually lasts a couple of weeks with several phases of planning, verifying, and configuring.

Unlike high-level team consistency in SAFe, Scrum allows multiple teams to work together at a low dependency level.

Estimating the remote team’s productivity

Before jump-start working, you should clearly understand your project business goals, while by tracking KPIs, you can see met targets, realization stages, and remaining tasks to derive the valuable outcome.

Testing equipment also makes a difference. For example, mobile app testing requires real devices to simulate end users’ actions.

Partnership with the outsourcing team: core concepts

Shared stake in success with delegated responsibility is the cornerstone of project performance. For instance, a QA team sets such transparent processes that the client receives up-to-date information about the status of issues and new defects weekly or daily via video conferences or email. This adaptability helps build trust and mutual understanding.

Besides, many customers value QA specialists’ intent on delving deep into the business context and learning the product features.

It is great when a client provides everything required for high-quality work, maintains the relevance of information sources, and informs about any internal changes in the company.

In conditions of geographical distribution, an experienced provider adapts to the client’s time zone. Depending on their requirements and business goals, QA engineers can adjust work schedules to fit the customer’s mode.

Cooperation with a remote team is a craft where the all-important point is a joint work of a customer and a QA provider.

a1qa knows how to tweak it to perfection. If you still doubt to onboard an outsourcing team, feel free to get hold of the a1qa’s experts.

The forced changes in the global context due to the spread of the COVID-19 virus have affected many areas of our lives. Right now, we are observing the onset of a global crisis that is turning our understanding of digital space upside down.

Take the promotion of home self-isolation as an example. It has stimulated a growing interest in online services. In the USA alone, Internet traffic has increased by 20%. Unfortunately, not all websites and mobile apps were prepared for such a large-scale online migration.

We still have time to rectify the situation. Here are five key lessons the businesses need to learn during this pandemic to somehow achieve the planned outcomes.

The need for digital transformation

The demand for digital technologies for business is growing by leaps and bounds because web users expect fast and accurate fulfillment of their requests.

Machine learning, artificial intelligence, the Internet of Things help improve customer experience. Software solutions based on these technologies are significantly more appealing for users than the rest. Here are just a few examples of why this is the case:

• Virtual assistants save user time because they can answer simple queries.
• Smart ecosystems make life more comfortable as they can synchronize different devices and can be controlled remotely.
• Big data solutions bring the user more satisfaction because they make generic software become personalized.

An unfinished digital transformation has limited the potential of a business at a time when everyone has massively migrated to the online space. After all, poor-quality or the absence of digital products has pushed users to find quick alternatives in conditions of increased competition in the network.

Attention to data management

Сollection, storage, and analysis of data help companies get a comprehensive picture of their customers. Effective data management is the basis for the development of IT systems.

Big data has become a corporate asset that helps build a development strategy, make informed business decisions, and streamline internal processes.

How does it work in practice? Data management allows you to create the optimal offering for each individual user, and this customer focus results in loyalty. Working with data has allowed companies to form more unique selling points.

If the company does not collect and analyze data or does not control the quality of information, the risk of receiving erroneous results with regard to compiling a business strategy will grow.

Flexible business model

Another lesson from the epidemic is the importance of building a flexible business model that allows you to transform processes and operations quickly if necessary.

Benefits were afforded to those companies that followed the digital-first concept. The main idea behind this approach is the preference for digital interaction channels and platforms. Such companies who have taken their first steps toward realizing this already possess a margin of safety.

To become a digital-oriented company, it is important to enlist the support of a reliable vendor. For example, choosing an outsourcing company can help you set up the continuity of working activities.

Outsourcing teams specialize in remote collaboration, so they already have the necessary experience in creating effective interaction processes. Contacting experienced IT service providers allows you to build a more flexible and productive business model.

The other advantage is risk diversification. The current situation has clearly shown the global chain of processes. The pandemic has also displayed how simple it is to break this link in a chain reaction. It is already impossible to navigate building a solely offline business because it can fall victim to recession in the conditions of a prolonged quarantine. It is necessary to formulate different formats and approaches for the development of operations that can interchange each other.

After the pandemic, businesses should constantly ask themselves the following questions:

• How can we reduce the impact of a sharp decline in demand for my product?
• How can we attract attention to the product in conditions of increasing competition?
• Why should a user continue buying my product even with a decrease in his/her income?

Focus on customer-facing solutions

A key component of this strategy is the focus on user satisfaction achieved via utilizing all customer touchpoints. Understanding the importance of customer needs is a significant business investment that affects process design.

The first step to ensuring personal interaction between the user and the product will be provided by the UX. After all, the higher the comfort of the end users when using your software product, the more likely they are to become regular customers and brand advocates.

An additional factor in creating customer-facing solutions is performance testing. These quality checks allow not only evaluating the performance of the product under the expected loads but also to look at it through the eyes of the user. The timely elimination of system bottlenecks helps introduce a more attractive and competitive product in the market.

Who might need this? Large corporations are a potential target. The global increase in traffic has affected even the giants. In early March, Netflix and YouTube asked their users to lower the quality of the video to reduce the load on the servers.

Process automation

Optimization of business processes can significantly save resources, and the key to this is automation. The attraction of self-regulatory technical means has shown their strength during the pandemic. Indeed, the automation of some tasks helps ensure that the production processes are not interrupted.

Test automation, for instance, allows a series of regular operations to simulate user activity. Individually designed tests can be re-run when testing functionality.

Moreover, test automation makes it possible to conduct checks at the optimal time while reducing the load on the server. Currently, a decrease in human involvement in this process is allowing business continuity.

Сonscious work on errors can help us actually implement the lessons of this pandemic and prevent possible difficulties in the future. It is worth remembering that the current predicament is only a phase that will end someday.

At present, we can learn several important lessons, revise our development strategies, and realize how to maintain an attitude of optimism in these difficult days.

Get a free consultation with the a1qa engineers on your software quality issues.

Founded in 2003 and based in Lakewood, a1qa consists of an 800+ strong independent software quality assurance providers’ team and delivers full-cycle testing services as well as particular testing types along with QA consulting, test automation, and many others. Furthermore, the company also holds a dozen offices and testing labs in Europe and is looking for geographic expansion to assure more significant cost savings to the clients.

With more than 1,500 successfully developed projects in its portfolio, a1qa has already served more than 800 global companies across multiple industries, including telecom, IT and software development, BFSI, healthcare, etc. Moreover, the professionals at a1qa cater to them with the values that adhere to the highest professional standards and improve the clients’ businesses. Through these exceptional values, QA professionals also create an unrivaled environment for extraordinary people.

View a1qa’s GoodFirms’ profile to know more about its guiding principles and other robust characteristics.

Starting with the interview, Nadya Knysh narrates her primary role as USO Managing director at a1qa and also the idea behind the commencement of the business along with the other parameters.

The primary responsibility of Nadya is to run a1qa’s operations in North America, manage adherence to the chosen development strategy, and ensure that professionals add value to clients’ businesses.

Moreover, Nadya also mentions the idea behind the commencement of the company by saying the following:

The professionals at a1qa help enterprises protect themselves from software failures in terms of time, money, and reputation, which can occur due to required bug fixing before the software goes live.

Coming to the a1qa’s most flourishing services, Nadya proudly mentions that “We believe in long-term collaboration and facilitate its development. At a1qa, we think that a partnership brings more value than a client-vendor relationship model.”

The professionals at a1qa enable a software testing process that approves the development journey of a particular project of the clients’ business. With a1qa’s engineers, the clients’ business team will be able to focus more on new traits rather than fire-fighting bug issues that arise from new builds.

Besides this, the team at a1qa applies both domain-specific and client-oriented testing methods and tools to guarantee that the delivered software products are of reliable and consistent quality. The QA specialists hold technological expertise that enables them to create the best-fit combination of infrastructure, tools, and resources based on the project goals.

Moreover, a1qa also offers 24/7/365 service to compress the lifecycle of the development process. Thus, backed by the team of professionals with narrow specialization in QA and software testing that solve the most challenging issues endows a1qa to tap into the list of the top software testing companies in the USA at GoodFirms.

The below-displayed review and the scorecard are confirming the quality of the testing services provided by the QA engineers at a1qa.

After giving a brief overview of the testing services rendered by the QA engineers at a1qa, Nadya also elaborates on the outstanding service, i.e., full-cycle testing. It is one of the core QA offerings catered to the clients globally. With over 16 years of experience, the professionals have worked out a holistic full-cycle testing solution to give the clients the most reliable end-results.

The QA engineers pick up manual, semi-automated, and automated testing services to assure that both front-end and back-end elements of the application run correctly as initially designed. Moreover, the testing veterans also reveal bottlenecks and breaking points in the application, evaluate current and projected data and user loads, and attune the software and hardware elements. Thus giving improved functionality with more excellent reliability to get better performance, the testers’ team of a1qa burgeons amongst the top QA testing companies at GoodFirms.

Having read the brief description of the testing services by Nadya, one can also have a look at the detailed interview at GoodFirms.

About GoodFirms

Washington, D.C. based GoodFirms is a maverick B2B research and reviews firm that aligns its efforts in finding the top software testing and QA testing companies delivering unparalleled services to its clients. GoodFirms’ extensive research process ranks the companies, boosts their online reputation and helps service seekers pick the right technology partner that meets their business needs.

About the author

Anna Stark is presently working as a Content Writer with GoodFirms – Washington D.C. based B2B Research Company, which bridges the gap between service seekers and service providers. Anna’s current role lingers to shape every company’s performance and key attributes into words. She firmly believes in the magic of words and equips new strategies that work, always in with ideas, something new to carve, and something original to decorate the firm’s identity.

Here at a1qa, we understand how difficult it can be to mediate your software’s quality while also fostering growth for your business. That’s why we’re dedicated to providing independent QA and testing services, so you can focus on enhancing your IT solution, thus, boosting your customers’ experience.

In light of our outstanding achievements in the software testing industry, we’ve been named a top technology partner on the Clutch 1000! This is a comprehensive directory that highlights leading companies in the B2B space, verified by Clutch’s research.

Out of 160,000 vendors listed on the platform, we fall in the top 1000 vendors, which puts us in the top 1 percent. We’re number 459 on the list, one out of only six application testing firms in the world, and one out of sixteen vendors features from Denver, Colorado.

We’d like to thank our customers for participating in client interviews on our behalf to help us achieve this award. They ranked our services on the basis of the quality of service, attention to project deadlines and time constraints, and overall cost as in value for their monetary investment. In a general reflection of those scores, we’ve been given an amazing five out of five stars! We’re so happy to be meeting our clients’ expectations for top-notch quality and assurance.

For those who might not know, Clutch is a B2B market research firm that employs a unique rating methodology to compare companies across various sectors. We’re also highlighted on both of Clutch’s sister sites, Visual Objects and The Manifest. The Manifest, a business data and how-to site, recently featured us on their list of top software testing companies.

We thank once again our customers and the Clutch team for making this award possible.

Please drop us a line if you’d like to enhance the quality of your software solutions with a1qa today.

As the holiday season is coming, many of us are planning to go travelling, visit  friends and families, go shopping, eat out or buy food to cook at home, or simply enjoy other seasonal pastimes in full swing. It may surprise you, but most of these pastimes are aided now by sophisticated pieces of software.

Here’s why we at a1qa want to take an opportunity to give sincere thanks to all QA and software testing pros. They work behind the scenes to provide us with high-quality software so that all of could rely on them for holiday fun, safety, stability, and convenience.

For example, if you’re one of the 50,9 million people who will take a 50-plus mile drive from home on November 23, the American Automobile Association reports, you are likely to use a GPS navigation app to find your destination. You may also download an app to find the lowest gas prices or find auto repair facility if needed (hopefully not, but who knows?)

Or maybe you’re one of the nearly 29 million Americans projected to fly during the 12-day Thanksgiving travel period. Your navigation through an airport will be much more convenient with the specialized app in your smartphone. Travelling with children? Install sandbox-style games or puzzle apps to keep them occupied – and thinking in the skies.

Purchasing meal for the holiday dinner? Point-of-sale software in most stores will speed up the checkout. You also can save a few bucks on Thanksgiving dinner by finding coupons for various Thanksgiving stuff. (By the way, the American Farm Bureau came up with another reason to be thankful for – a 16-pound turkey this year costs $22.38, which is a 36-cent decrease per turkey compared to last year.)

Prefer shopping online? Analysts report, Cyber Monday will remain the No. 1 day online sales and will generate $4.50 billion in e-commerce sales. Shopping online becomes easier and safer thanks to thorough and timely testing, of course.

And the last but not the least. You’re likely to use your smartphone to check emails, text you dearest and nearest, make calls. Or maybe you’ll be the one who will call the Butterball Turkey Talk-line experts to tips on how to cook the main part of the Thanksgiving meal.

It’s very easy to take the flawless quality of these apps for granted. But we’re thankful to our colleagues who do their best to ensure the quality of the apps to detect all the issues that may ruin the holiday.

Happy Thanksgiving, everyone!

For those of you who’ve missed our previous post, let’s remind that a1qa has been engaged in the project of testing the new payment system for more than two years. There was nothing unusual about the project. Until one day, the customer gave us the challenge to arrange a demonstration of the first product release. Our team knew the product perfectly, so we immediately agreed, although the task required additional preparations.

The product was to be demonstrated to the customer’s team that knew and understood the requirements, but had only a vague idea of testing processes and terminology.

Acceptance testing challenges we faced

Customer representatives and tech geeks seemed not to see the product through the same glasses. And that became the reason of the first challenge.

What was important for the customer? To make sure that all processes function correctly from start to finish. As for us, our team of functional testers split the processes into small blocks and tested each block separately and in great detail, sometimes losing sight of the product as a whole.

Additionally, we lacked a clear mapping of test cases and business requirements. That was the second challenge. Test cases were based on technical documentation developed by system analysts and architects. Therefore, it was difficult to find the approach that would help demonstrate the expected result to the customer representatives.

a1qa approach to demonstrating first release

Criteria for successful acceptance of the first release there were very loyal: the implementation of 30% of the requirements, pass-rate – 80%, and the absence of critical defects in the implemented functionality.

Taking those criteria, a small number of demonstrated requirements (about 180), and the lack of complete business processes implementation (only some modules were released), we decided to use the following approach:

• We selected the realized requirements, which had to be demonstrated.
• For each requirement one or more test cases were prepared. Test cases were aimed at demonstrating that particular requirement.
• Each test case assigned to the requirement was a standard test case with test data, a precondition, a detailed description of the steps and results. However, we focused on the description of the test case purpose and underlined the connection between the test case and the requirement as clearly as possible.

Thus, the implementation of each requirement was demonstrated by the test case, and the correctness of implementation was confirmed by passing the corresponding test case

Test cases were prepared as a separate Excel file, and attached to the Acceptance Testing (AT) document.

How was AT held?

AT was held by the a1qa engineers in the presence of several customer representatives. a1qa was in charge of demonstrating the product and executing test cases.
The commission included 10 people – representatives of top management and employees, who had to manage the payment system and monitor it.

The technical equipment of AT wasn’t sophisticated and consisted of a projector to show the information about the product under test and a monitor to display the Acceptance Testing document.

The AT process was organized as follows: the demonstrating requirement was named, the test case was described, and each action was commented. The result of each check was documented. During the demonstration there were many questions, comments, and proposals for improvement. Comments were recorded and questions were answered. If we had no answer for a question, the question was recorded and passed to responsible specialists.

The demonstration generally proceeded smoothly. All 230 test cases were passed, and therefore all 180 requirements were implemented correctly. The first release was successfully demonstrated.

However, we weren’t overly optimistic. There were two more releases ahead and it was advisable to start preparing for the next AT as early as possible. Moreover, after the first AT, we had a lot of ideas about improvements for the following demonstration.

Second release: new approach for Acceptance Testing

The main point that we had to improve in the approach was the “Test Methodology” section. Why wasn’t the approach used for the first release appropriate? The thing is, all test cases covered different modules, and didn’t demonstrate the system as a whole. Thus, it wasn’t evident that all processes were covered, and there was no sense of the integral system.

At the first release, the number of implemented requirements was not so large. To demonstrate the final product it was necessary to use a new approach.

Finally, we had to work on the terms used in the test cases. They were understandable for functional testers and other IT professionals, but not for the customer representatives. Therefore, it was necessary to use terminology of business requirements and prepare test data close to the real ones.

Solution found

We decided to separate the test documentation used for functional testing from the AT one. So we arrived at the idea of preparing separate scenarios for demonstrating the system through business processes – end to end (E2E) scenarios. One more activity was to prepare test data.

What is the reason for using E2E scenarios and what is their advantage? Naturally, there are some projects on which acceptance can be carried out using usual test-cases; however it was not our case. Within the framework of our project, the E2E scenarios were more applicable than test cases due to the following factors:

• Reducing the number of test artifacts by grouping several requirements into one scenario. This allowed demonstrating all the implemented functionality faster. By the way, the demonstration of the first release took us five days. Keep in mind that only 180 requirements were implemented.
• Demonstrating the product through complete business processes, close to the real scenarios of working with the system. The customer got an opportunity to test the product in conditions close to real ones.

Formulating ideal scenario

The idea of using business scenarios defended itself completely. Working together with the customer, we were able to develop a certain approach to preparing scenarios, and decided on a list of requirements for this type of test documentation.

Based on the experience of previous AT and the wishes of the customer, we provided the following structure of the test scenario:

Title. A short and clear title of the scenario helps to understand the aim of the check.

Test Case Goal. It was necessary to describe the purpose of this scenario.

Description. This part describes how the scenario should be passed, using business terminology and speaking the customer’s language.

Constraints. On large projects, one of the affected modules might be not implemented or implemented partly. But this is not an excuse to skip demonstration of the functions associated with it. For example, notifications of a certain type aren’t generated or some interface elements are displayed incorrectly.
It’s not that important. The main thing is that you, as QA engineers, have found and submitted these defects. This is what should be described in the Constraints section. This will help avoid the customer’s disappointment.

Test Data Description. On our project, this section was appropriate, since various technical test data were used to describe the scenarios, which were difficult to relate to the business components. Therefore, in this section, we described as detailed as possible what all elements of the database using business terminology.

Correlation of Steps and Requirements. Since one of the AT goals is to prove that requirements are implemented, it is necessary to specify which requirements we demonstrate passing the scenario.

Test Scenario Steps. This part consists of three components:

1. The actions description with the help of business terms. Technical terms are also applicable, but only in exceptional cases. The main point is to use terms that are understood by the customer.
2. Expected result (again using business terms).
3. Method of verification. Here we can use a technical terms to describe technical implementation of previously stated business requirements.

Conclusion

Our team has come a long way from AT test cases to complete E2E scenarios that show the system from the business point of view.

To sum up, let’s note the following points:

• Not every test case is suitable for AT.
• Learn how to write E2E scenarios and your work as a tester will be invaluable.
• Do not be afraid to take on new tasks. If we do only what we know, we will never become better.

Good luck on your projects!

We’ll be happy to answer any questions you might have.

Let’s face it: missing bugs is frustrating. It will be even safe to say a missed bug is the worst nightmare of any tester.

Why so? There is a myth and misconception that testers should hunt all bugs. Testers are viewed as goalkeepers who are the first to blame if there is any bug leakage to the production.

Yes, missing bugs is annoying. But there are reasons behind it. Let’s look at the most common ones and offer solutions that can help testers fix the situation and never let bugs appear in the final version for the same reason.

#1. The pesticide paradox: with the course of time test suites wear out

Almost 20 years ago Boris Beizer, American software engineer and author, formulated the Pesticide Paradox:

Plainly speaking, it means that when tests that are launched hundreds of time, they stop being effective. As a result, a number of bugs that are introduced into the system are not detected by existing tests get to end users.

What can be done?

Never assume that you can build an ultimate test suite that will detect all the bugs in all product versions. What you should do to ensure your tests are working well and perform successful testing is to keep track of the product changes, review and update your test suites regularly.

#2. Lack of time to test that area

It’s not a rare case when software testing team gets under time pressure and has to opt either to burn the midnight oil working overtime, or skip some tests. Even if you are diligent enough to select the first option, you will certainly be in a hurry. And it’s quite natural: when people are in a hurry, they overlook things and miss the bugs, even the most obvious ones.

What can be done?

If the deadline is fixed, communicate with your manager to decrease the scope of testing and analyze risks. Prioritizing testing is also a great plan here as you can suggest skipping low-risks area and focus on the business critical functionality. Tell your manager or any stakeholders what you can test, and what you don’t have time to. Also, inform on the risks entailed.

Be transparent and never hide the issue from the customer hoping that the bugs won’t reach end users.

#3. Missing the most obvious bugs

You can’t even imagine how often the bugs that seem to be right in front of your eyes are missed! And they are the most annoying ones. Testers miss them because they get accustomed to them while looking at the app under testing. It may also happen when a tester is too focused on another task and switch off the “bug hunting mode”.

What can be done?

Practice multi-tasking and attention to details. Try to put yourself in the end user’s shoes and click the application from scratch.

#4. Requirements documentation is improper

At times, the root cause of the missed can be found on the earliest stages of testing projects, when testing itself hasn’t started yet. And it’s about poor requirements documentation. If the documentation doesn’t cover all usage scenarios, testers will not cover these scenarios when testing.

It’s very important that both requirement documentation and test cases prepared should be complete and clear, covering all functions and user scenarios. From our experience, we’ll say for sure that it’s less costly to prepare comprehensive documentation than fixing a bug on late stage of the development.

What can be done?

While establishing proper communication with the customer and asking proper requirements documentation is the responsibility by test managers or business analysts (depending on your organization workflow), there is something you can also do to avoid miscommunication errors. Make sure you know the expected system behavior before getting down to testing. Once the requirements change, make sure you’ve reviewed and updated your test suites as well.

#5. The bug was discovered and reported, but it was too costly to be fixed

Testers are responsible for providing developers and stakeholders with relevant information on the system quality. However, it’s not their responsibility to decide on the developers’ work or product’s release.

Actually, there are many factors to decide on going live (much more than bugs presence). And at times it makes sense to ship the product even with minor bugs in it.

What can be done?

Report all the bugs to the developers in a clear and timely manner. Before the release, provide the stakeholders with the most comprehensive feedback you can on the system functionality, performance, usability and security.

#6. This area is intentionally left blank

It’s up to you to fill it: add any other reason for missing bugs.

Welcome to the comments!

By Anton Trizna, Head of the a1qa Business Analysis TCOE, and Elena Goropeka, senior business analyst at a1qa.

The terms “business analyst” and “system analyst” are regularly misunderstood and used interchangeably. But in fact, these are two different positions with different duties and set of skills. We’ve decided to learn how these two positions differ and what commonalities they share in what relates to QA consulting specifics.

First, let’s take a look at the commonly applied definitions.

BABOK (Business Analysis Body of Knowledge, a table book for business analysts) says that there are many job titles that may perform business analysis: business architect, data analyst, business consultant, process analyst, requirements engineer and system analyst, etc.

BABOK also outlines that the business analyst should ensure that the delivered solution will enable the company to produce the expected outcome. And the solution doesn’t need to be an IT-system. From this perspective, we can assume that the business analyst is a general role applicable to the group of professions working with business requirements to ensure the achievement of the set goal.

Very often analytical roles in IT are divided according to the main sphere of the knowledge area: whether it’s IT specifics or customer’s business domain.

Where does the analyst fit in?

At a1qa, we have adopted the following differentiation:

• The business analyst will use business analysis methodologies to gather the customer’s requirements and check them for possible challenges to produce a high-quality solution.
• The business analyst in IT is the analyst who will solve customer’s problems by proposing to develop and implement certain IT-systems.
• The system analyst is responsible for defining technical aspects of the developed IT system, the platform, integration means, and developed system role among the company’s products.

Business analyst’s main focus is to identify customer’s needs and justify the necessity of the project implementation. Typical tasks performed by business analysts include:

1. Discovering customer’s needs and problems.
2. Designing of the project scope.
3. Functional and non-functional requirements elicitation.

In the last stage, a system analyst may already come into play. However, the performed duties will vary. A BA won’t consider the implementation platform and technologies and will pay much attention to the customer’s objectives and preferences. At that rate, the gathered requirements should be measurable, clear and correct. An SA will choose the most appropriate technology and platform to meet all functional requirements.

At times, the platform and technology may be specified in advance. If so, the primary goal is to correlate functional requirements with the chosen software means, adapt them in accordance with the platform terminology and interaction interfaces to ensure proper developers’ work.

After addressing all the requirements, analysts start consulting development and testing teams. A BA will present the requirements from end-user’s perspective, while an SA will put emphasis on the platform.

Highly professional business and system analysts will possess the following knowledge and skills:

Following the division of the areas of responsibilities, business and system analysts will deliver different sets of documentation. A BA will create the vision and scope document, widely accepted business requirements document and software requirements specifications.

An SA will present the concept of IT solution and indicate the platform on which the system is to be developed, technologies, programming language and interaction interfaces.

Summing it up, in practice, it may be hard to differentiate between both roles as they may intercross on the project.

The titles themselves don’t matter a lot. What is really important is that all employees should be aware of the duties both specialists perform and what problems they are expected to solve.

Do you know that IT sphere is considered to be one of the most rapidly developing industries? The latest IT trends are changing as fast as the readers’ attention. a1qa team has analyzed the posts that caught the greatest interest and compiled the list of TOP 5 most popular topics.

1. Agile

Agile is called the world’s most popular innovation engine that is why it will never lose its popularity.

The Introducing QA to Agile Team post by Svetlana Pravdina contains practical advice on how to implement QA to Agile teams. This article would be of special interest for those who would like to introduce QA to Agile team, but is confused or scared to face radical challenges.

The article Applying the Agile Manifesto to Mobile Testing by Nadia Knysh states the factors you should take into consideration if you want to follow The Agile Manifesto Principles in testing mobile apps. It provides you with practical tools to implement these principles The Agile Manifesto proclaims 4 key values and 12 that have been adapted for managing a variety of business and IT-related projects.

2. Test automation

Modern IT world requires more and more software QA engineers and test automation. The article Software Testing Needs More Automation by Aleksander Panchenko presents the results of global market research study.

Part 1 tells about the general directions in IT and automated tests results. Learn what future specialist forecast for test automation in particular, and IT world in general in Part 2.

3. Interview

Have you ever heard anything about Alan Page? He is the lead author of books about Software Testing and a software tester with nearly 20 years of experience, who has published a collection of essays on test automation. Read the interview Test Specialists are Essential Members of Software Teams to find out why Alan thinks that “the trends of more frequent releases and wide scale development of web services and apps dictate that the future of testing will change”.

One more prolific specialist is Eric Jacobson, a tester with 14 years of experience in IT industry. Read about Anti-Bottleneck testing and Eric’s attitude towards trivial bugs in A Story by an Anti-Bottleneck Tester post.

Lloyd Roden, a Developer, Test Analyst and Test Manager without any doubt can be called a cross-functional specialist. In the How to Challenge Complexity post Lloyd says why he names himself a Jedi Tester and why he is “a firm believer in Exploratory Testing”.

4. Security testing

Today DDoS/DoS attacks are so ubiquitous that almost every day a new case is highlighted. But do you know when the first DoS attack was registered and what was it like? The article DDoS/DoS Attacks: Experience-based Insight by Aleksey Abramovich points out the modern motives for carrying out DDoS/DoS attacks and defines categories of attacks depending on their target. Find general information in Part I. Read Part II, if you need more detailed information on the subject matter.

5. Mobile testing

What should you know when choosing automation testing tools and test approach to mobile testing? What factors can influence your choice? Find the answers to these and other questions in the Automated Testing of Mobile Applications post by Dmitry Tishchenko. It is obvious that automated tests are gaining popularity and automation of mobile application testing is decreasing testing costs.

As you can see a1qa always stays in step with the latest IT trends and posts valuable information. But there’s still a lot to be done. a1qa holds by what readers say. Comment this article and express your interests. What topics would you like to read about?

The article by Alexander Panchenko, Deputy Head of Complex Web QA Department, and Olga Demeshko, QA Engineer.

Everyone counts

If you have decided that testing is your call, – the easiest way to get the basics of the profession is to enroll in testing course. Preferably, take a course at the QA company which you would like to join afterwards. Although the theoretical basis is mainly the same, different companies apply principles which may vary significantly.

We want to emphasize – there is no need to become a tester first and then to retrain for a software developer. Of course, both jobs deal with software products, but in a completely different manner.

Developers are to create, whereas QA engineers are to criticize and crack the code. In fact, they have different mindset. Only software QA consultants engaged in automated testing are rather close to development, although even this area has its own specifics. If you have really decided to become a developer, then try to take a developer course.

On the contrary, previous work experience is useful. Some companies recruit professionals for specific projects. Having contracts for testing internal wages systems in their pocket they will be glad to welcome accounting or HR professionals on board.The same applies to foreign languages, marketing or even construction – QA engineers test applications for a variety of industries.

No panic!

Now, you have successfully graduated from the training course and you got the job, right? If your first task on the project turns out to be too difficult, do not panic!

The description might be quite confusing for a person with no experience. But even if you got the impression of transliterated Chinese, do not despair. In any case, you are surrounded by experienced colleagues, and Internet is full of suggestions. The main thing is not to be afraid to ask questions. It’s not a shame not to know something. Much worse is to send the results not understanding the task.

If, on the contrary, the task seems too simple, do not accept greater responsibility than you can bear. Excellent performance in a focused specialization will characterize you better than a million of unrealized promises. So, do not try to prepare testing documentation, execute tests urgently, etc. A clear understanding of processes and terminology comes first.

From a newcomer to a top manager

The first thing you will have to face is testing the basic functionality and business logic. A common mistake of young QA engineers is to check an interface disregarding basic functions. The interface is important, with no doubt, but the number one task is to control the correct performance of application’s main functions.

Does it make sense to visit an online shop with a stylish interface, if a client is not able to register on this website? No matter how attractive the interface is, if the application is unable to fulfil the main objective, it is useless.

Along with the main work a tester should continue exploring operating systems, computer networks, virtual machines. The more you learn, the more skills you obtain, the faster you are growing in the profession. The conclusion is obvious, if you want to become a qualified expert in software testing – constant development is required.

Read the first part of the article here.

The article by Alexander Panchenko, Head of Complex Web QA Department, and Olga Demeshko, QA-engineer.

A QA-engineer is a highly demanded profession nowadays. The research made by Forbes this year proclaims that this job has been recognized as the second happiest in the US. CareerBliss lists a QA-engineer salary range of $41,000-$71,000 and up to $91,000 for senior QA engineers. This article provides information on who can embark on a career in software testing, what talents are necessary for this profession and what should you know to work for a QA company.

Job for meticulous ones

Let’s determine basic factors for being a QA engineer. A university diploma is essential but not critical since there are lot of training courses giving necessary knowledge and basic skills. The main characteristic that identifies the “inborn” tester is curiosity. Those, who in their childhood were really passionate about dismantling a PlayStation, and penetrating into a washing machine to see what it has inside, may be sure they can be good testers!

If you’re wondering how the every-day-life devices are constructed, your friends call you a scrupulous person and you don’t leave the work just because you are bored – then Software Testing is a good fit for you. Despite the meticulous character that testers have, do not think that QA is only routine and mechanical work. Certain creativity is always expected from a QA engineer. Intuition and great imagination are indispensable parts of the profession.

Apart from that your manager would expect some flexibility. If you know a set of standard checks and possible errors, you will not be able to work in testing for a long time, that’s obvious. Just try to recollect the way computers looked like when you were a schoolboy (schoolgirl) and note how little kids operate cutting-edge Tablet PCs today. Testers need to be prepared for continuous learning. New applications release, new operating systems and gadgets should be immediately taken into account.

Test… a pencil!

So, you have decided that this profession might be interesting to you. How to start? You can get a general idea from special literature and Internet forums. But in this case you risk getting lost in different opinions and viewpoints.

To find out what you will have to face, the senior testers offer to conduct an experiment – to test a pencil, a chair, a cup of coffee – anything that helps to a standard software testing approach. The aim is to be creative enough and to use another angle while doing your job.

The same principle works for mobile applications testing. It is important to check all possible actions that “a pencil” user might execute. But do not forget about the appropriateness of those checks. While the case when a pencil falls from a desk is close to reality, its cutting with a chopper is clearly not. If you manage to come up with a lot of scenarios of such unusual testing you have successfully passed the first exam to go into the profession.

Read the second part here.

If you missed the first part of the article read it here.

It would be unfair to ignore the risks that testers face working side by side with developers. For me, the description of the defects became the most serious risk.

Sometimes, testers try to save their time and efforts by discussing defects directly rather than receiving detailed written reports. That said, as new developers and testers join the team, having clear descriptions is important to get them up to speed.

Initially, I even tried to adjust the testing strategy depending on the specific functions of the developer. However, that was not correct, as sooner or later, it leaded to missed defects.

The first time I found myself in the epicenter of a developers meeting I felt like I was in another country.

But, as it always happens in the real life, after a while their language didn’t seem so foreign.

In the beginning I appeared at such meetings by chance (all meetings were held near my workplace), but it soon became clear that my presence at the general meetings was quite useful for the project.

Being a part of those meetings I understood every single detail and often was valuable in developing the testing strategy. Moreover, I informed team members how the code or its part would be tested, that helped to prevent defects proactively.

The abovementioned communication rules are pretty clear and simple. However, these obvious steps are rarely executed when the team is distributed across different floors, different cities and countries.

All too often, IT project participants forget that developers and testers have the same goal: to deliver high-quality software. No matter how talented its participants are, the project will never run smoothly if the team doesn’t strive for the same result. Providing a common goal helps establish a team environment and clear lines of communication. Everyone will benefit —software developers, testers, managers, customers, and the company in general.

The article by Anastasia Kotsevich — a QA Team Lead at a1qa —  was published on Stickyminds. Read the full article here.

The article by Anastasia Kotsevisch was published on Stickyminds. You can also read it here.

Anastasia Kotsevich is a QA Team Lead at a1qa with 3 years of experience in Testing and Project management and  more than 6 years of experience as an IT-coach. She used to manage the team of more than 10 people and  specializes in projects with complex business logic,  financial technical analysis in corporate ERP-systems. Anastasia is also a frequent speaker at international QA conferences and the local QA Academy.

I have a question for testers: Have you ever tried working in the same room with coders? I expect the majority of responses to be “no”. It’s really no surprise, considering testing is most commonly performed in a separate location.

That’s why, when I faced an opportunity to work such an environment, I was hesitant. I wanted to take on this task but, to be honest, I was afraid. I didn’t have to move to another city or country, or even another building — I just had to go one floor up — but I expected to be like Alice in Wonderland, falling through a rabbit hole into the strange world of coders.

I halfway expected to see Supermen controlling computers with the power of thought and correcting defects like cracking nuts. I was very much afraid that they would find me silly — or, on the contrary, very smart. I wasn’t sure which one is worse.

I felt that I entered a hostile environment; I thought they hated me from the get-go. I can’t say I blamed them. After all, my job was to disrupt coders’ quiet lives by finding bugs and issues in their code. Who would appreciate that?

The first point in my plan was to understand the project clearly in order to not ask stupid questions and annoy the coders with “under-bugs”. However, that group of projects seemed extremely complicated, and the field — financial analysis — was unfamiliar to me. I decided to ask the developers for help.

Unfortunately, I did not get any assistance from the coders because they weren’t any more familiar with the business logic than I. But making an effort to “speak their language” when asking for help made it easier to become a part of the team.

Looking back, I realize that it was the moment we discovered the first rule of the teamwork: try to sort it out together! The earlier you understand this, the better code you will make up and the more effective tests you will get.

The second point of my initial plan was to provide a flawless description of defects. I was pretty confident that well-described defects wouldn’t be too annoying for programmers. That is why I wasn’t hesitant to share the first bug I found, clearly describing fields and algorithms, providing informative attachments, etc.

Imagine my surprise when my new colleagues started to ask questions regarding this “ideal” defect. At that moment, I discovered one of the most important things: I learned that developers and testers perceived defects differently. While testers might think they knew what field would be of the highest interest to coders, they were wrong. Each developer focused on a different part of the defect, often unaware of the fields presented in the description.

Read the second part here.

Decoding the voice

It’s important to consider that testing voice recognition clients differs from any other type of testing. Unlike testing a regular mobile application, the tester has an endless scope of data that could be entered. If you want a good client, you do not limit a person to just 10 words the system recognizes. Modern voice recognition clients should decode as many commands as possible, presenting developers and testers with a very challenging task. But still, even the best voice recognition system doesn’t guarantee correct decoding 100% of the time. The job of the developers and testers is to make the correct decoding percentage as high as possible.

If we talk about voice typesetting, the stage where mistakes are most likely to occur is the decoding stage. Let’s look at the architectural level of the process, when sounds captured by the microphone are going through frequency analysis. First, the voice is converted into graphical wave motion; then it is transformed into characters that build the word. The searching tool in Google Chrome mobile version, and predictive typing on mobile phones and tablets are good examples of this process.

However, it gets more complex when you deal with multi-functional applications, where the voice recognition system consists of two stages. First, the client decodes the voice and forms the whole phrase. Second, a complex algorithm switches on and starts analyzing each word separately and the whole phrase together. This is where the biggest amount of mistakes occur. Those voice recognition systems are pretty bulky, so they are installed on servers, while the mobile device has just a small client to record the voice, send it to servers and receive the commands back to perform them. To optimize testing and the fixing of bugs on the server and the client, mistakes should be strictly differentiated.

Testing the client: with female voice and in the pub

The way we speak and the way we pronounce words – these are the types of factors that have an impact on voice recognition systems. The voice pitch and timbre could be recognized by the system differently. Also, every person has his own voice speed. This should be taken into consideration while working on choosing testing scenarios. It is recommended to choose a quality assurance engineer with average pitch, timbre and voice speed. Ideally, the same functions are tested with both male and female voices. In testing a client for a foreign language, it’s good to have a tester able to speak without an accent, so you don’t end up like the guys in this video clip.

The tester should forecast different environments; it’s not enough to make a test just for a quiet room. Noisy streets, crowded pubs and public transport – the voice client should be adjusted to decode the human voice anywhere.
What else can undermine voice client performance? If technical support, such as headsets, Bluetooth and other accessories don’t function correctly, the client can fail in accomplishing the task at hand. The need for an instant and reliable connection challenges developers and testers to diminish the impact of Internet connection quality. It also helps if the tester emulates other user scenarios, such as playing music on his phone, incoming calls and other interruptions.
It’s not so easy to imitate a user while testing voice recognition clients. However, this is the very case when the “do like a user does” approach is the key to success. An experienced tester can think of many users’ testing approaches to ensure high quality of the final product.

Currently at its peak of popularity, voice clients still have a huge niche in which to be developed and adopted. This gives developers a lot of room in improving current software and creating new ones. At the same time, it is a great responsibility to be involved in this process. Every tester should keep in mind the millions of people using the voice recognition software they have tested and improved. Using correct approaches and optimal strategies in testing will allow every user to be satisfied with the communications channel you have enabled for them.

Read the full version here. 

The article was published in RCRWirelessNews.

The article was published in RCRWirelessNews.

Voice clients are already installed in mobile devices, computers, TVs, washing machines, elevators and cars. If you are interested in learning more about the story behind maintaining human-machine communication, then read on.

Progress demands

We are witnessing a growing tendency in embedding voice recognition systems in all industries – from tablets to cars. Such a trend could be explained by our hi-tech crazed modern society. The trend is also encouraged by media, cinematography, marketing, TV and the Internet.

But if you look deeper, you can find the roots of humans’ aspirations to start talking to machines in their anthropology. Applying voice recognition systems is natural for people, and whatever feels natural makes the processes of achieving these things simpler. Historically, mankind used to exchange information through interpersonal communication, where verbal communication was it. Before the time when people were writing letters, texting and sending messages by e-mail, the way they exchanged information was via verbal communication only.

The main advantage of voice recognition systems is that the user doesn’t need to develop any new skills to manage the program. In comparison, consider how when becoming a computer user a person should learn computer literacy – how to hold the mouse and how to type on the keyboard. When managing the machine by his voice, however, the person doesn’t need any special skill to pronounce the command. Another important aspect is that many tasks could be completed just with a help of human voice. Then the voice recognition system is applying itself to many components of the system, while the person doesn’t need to switch different interfaces.

Limits stimulate development

The target audience of voice recognition systems is quite diverse. There are groups that are in urgent need of having such an option: people suffering from disabilities and people driving their cars, for example. For many disabled people, voice recognition software is the only way for them to interact with the outside world independently. As for car drivers, they have been pushed to start using voice recognition systems. With security requirements getting stricter all over the world, car manufacturers have started installing voice systems to avoid the official ban on talking while driving.

As a result of these demands, most mobile manufacturers are also developing and implementing voice recognition clients. While the concept of the system is similar, the quality of products varies. The voice client quality is becoming a real advantage for potential phone buyers, since they consider it a serious issue when choosing a new device.

Read the full version here.

You can read the first part of the article here. The article was published on Engineers Edge.

Shortcuts for managing

The administration of the Linux host (where your web application runs) requires frequent job and process managing activities. A few must-knows are listed below.

To interrupt a job, use a shortcut Ctrl-C. When you need to resume a job, use a shortcut Ctrl-Z. The command fg restarts the job, while bg places a job to the background, allowing you to perform additional tasks at the same time. Also add an ampersand (“&”) to your command in the end of the string to start it in the background.

When you need to view currently running processes, run “ps.” While all jobs have unique process IDs displayed in the first column of the output, rest assured there are some more useful options here to modify its result view.
If you need to end a required job, run “kill/killall” followed by the process ID or process name (kill 22064; killall java, for example).

The grep command will help to find a specific job you might need. It is an efficient search tool with a large scope of configurations (for example, ps -aux | grep java). The ps returns the list of all processes. The grep filters the list according your search criteria.

Installing new software

What should we expect when installing new software in Linux OS? This can be a challenging task among former Windows users. Usually it can be done by following these methods: installing RPM packages, installing DEB packages or installing from tarballs (esp. source code).

On top of that, when starting working with Linux, you should always keep in mind software repositories, which provide storage for packages (both source and binary) accessible via Internet to install any required software on your computer. It’s up to you whether to use a certain repository or create your own. See examples for two of the most popular utilities: YUM in files repo in the directory/etc/yum.repos.d/ and APT in file /etc/apt/sources.list and in the files in the directory/etc/apt/source.list.d/.

Types of software

As for software testing itself, there are basic instruments for testing Linux applications you will definitely need. Most of these solutions are applicable to the majority of Unix-based systems and are console-based, which makes them easier to automate.

There are three types of software in Linux: Core (Kernel), User applications (userspace level), and Core + User applications. Core applications include the core itself, the kernel modules and user space level for kernel control (meaning the / proc and / sys interfaces). Since the kernel itself is written on C and ASM, C is the preference for testing. Usually these are small test kernel modules, checking some functions or module with different parameters + script.

Based on many years of testing experience, it is recommended you avoid using one module that checks the entire “feature.” This is why many modules are used to check each of the functions separately. Also keep in mind that you have to check all possible functions return codes.

User applications can be considered any application running on Linux. However, if the application is written in Java, you’ll need to own Java, at least in order to make sure that the program is working.

Core + User applications are the most popular to be used in Linux. If you are dealing with this type of application, it means the core driver provides low-level communication with any device and the user program.

Testing tools for Linux

Since all Linux tools are either present in any distributive or can be freely downloaded, Linux is a convenient OS both for programming and testing. Basic tools for testing Linux applications are as follows:

• GCC – Gnu C compiler. To test the compiler, you can use gcc, which is a Basic C, C++ compiler for Linux. Its website has special tests. If you compile the -g option, you can debug the program with GDB.
• BASH. The BASH shell is also included in each distribution. It is very useful for writing scripts.
• expect is also present in each distributive. It is a simple but quite handy syntax tool command language (TCL).
• expect-perl ? expect-python (pyexpect) – libraries expect for scripting languages perl and python.
• gdb – Gnu Debuger. This is a standard C/C++ debugger. If you’ve never used it, we advise you to get acquainted with this tool. Use kgdb for kernel.
• ltt – Linux Trace Toolkit. If your Linux core supports LTT, you can view the active processes/system calls in the current process.
• import ? gimp – can be used for taking screenshots for testing graphics applications.
• minicom is a program for manual testing. If you want to automate the console, it is better to use the expect (or in conjunction with the “cat” and “echo”, or just open / dev / ttySx as file – sometimes the second option does not fit).
• ltp – Linux Test Suite Page [ltp.sf.net] is a very useful collection of tests. It includes tests of file systems, system calls, etc.

Among other common tools, it would good to mention such as netperf (utility to verify the network performance), ircp, irdump, openobex (utilities for infrared checking), and telnet, ssh (a remote shell). If you need to enter same commands frequently, you can use expect, which is available in any distributive). A more detailed comprehensive list of tools commonly used for testing the various components of Linux can be found here.

Hackers’ security distribution

Linux also has its own distributions for testing. Backtrack-Linux.org is a good example of a specialized Linux distribution that has just one purpose – to test your network, devices and systems for security vulnerabilities. The last version of Backtrack was released in August of 2012.

Backtrack all started with earlier versions of live Linux such distributions such as Whoppix, IWHAX, and Auditor. As it’s said at Offensive Security, after years of development, penetration tests, and unprecedented help from the security community, it evolved to what is now known as a GPL-compliant Linux distribution built by penetration testers for penetration testers with development staff consisting of individuals spanning different languages, regions, industries and nationalities.

Backtrack consists of more than 300 security open source tools and utilities. While there are many commercial programs available, many security professionals prefer BackTrack tools. The interesting thing is that BackTrack is also popular among hackers because of its anonymity; when installing this distribution, you don’t have to register.

Many security practitioners use BackTrack to perform their security assessments. BackTrack is an open-source, Linux-based penetration testing toolset. It makes performing a security assessment easier, because all of the common tools that you need are all packaged into one nice distribution and ready to go at a moment’s notice. As with other Linux distributions, BackTrack is supported and developed by a community of users that range from skilled penetration testers in the information security field, government entities, information technology experts, security enthusiasts and individuals new to the security community.

Conclusion

The above should provide a good overview of some of the basic Linux tools, singularities, process management, specific limitations, etc., that are vital for quality assurance services involving Linux. However, this is just the tip of the iceberg when it comes to Linux, the most stable, efficient, safe and legal operating system ever.

The article was published on Engineers Edge.

Even though Linux has a relatively small percentage of desktop users, that small percentage must be provided with well-developed and tested software. That means testers all over the world should be ready to fulfill any customer whim, including testing Linux-based applications. This is where the following tutorial comes in – to help prepare for this scenario.

The great battle of Linux and Windows

Unlike the majority of operating systems, Linux is a free one. It does not require any license to purchase it and can be downloaded at no charge. A lot of available software is developed for Linux, so a user doesn’t experience any inconvenience when choosing Linux over Microsoft.

The main difference between Linux and Windows is the superior flexibility Linux provides. While Windows has the same settings for all users, Linux settings and configurations can be easily adjusted to fit each user’s preferences. This is why every user has a unique system, which can’t be said about Windows. It is these standard Windows settings that most PCs have that make Microsoft PCs more vulnerable than Linux systems.

Being a stable system, Linux is also well-known for its extremely high security. Despite many attempts hackers have made to break the system, Linux has managed to remain secure.

Another important Linux characteristic is the productivity it supports. If you run two identical programs on two identical computers with the only difference being the OS (Windows or Linux), you’ll find the Linux OS operates faster. Consider the statistics; more than 95 percent of supercomputers are operated by *nix, and a significant number of servers are run on Linux distributions.

Linux standard base (LSB) testing

Unlike Microsoft, Linux doesn’t have hundreds of hired developers and quality engineers to maintain the quality of software produced for its users. Regardless, the community of Linux volunteers has found a way to underpin long-term compatibility guarantees and comprehensive compatibility testing.

Together, the Linux Foundation and the Institute for System Programming of the Russian Academy of Sciences are putting huge resources toward developing new tools and technologies to break through LSB testing challenges. These resources, known collectively as the LSB Testing Framework, include such components as Linux Application Checker, Distribution Testkit (DTK) Manager, AZOV Shallow Test Development Framework, T2C Normal Test Development Framework and UniTESK Deep Test Development Framework.

Also, a great number of paid-for tools are developed for testing software that runs Linux distributions. Now we can move on to the technical differences testers should be aware of when it comes to Linux.

Introduction to *nix

To distinguish between two operating systems we’d like to share some hints to help avoid their specific singularities. We suggest every novice Linux user start with the “isman” command, which displays online manual pages for specified commands. If you put “man ls,” for instance, this will return info regarding the command you may want to learn: Name, Synopsis, Description, Options.

Pay attention to the fact that command names, paths and file names are case-sensitive. For example, “test.log” and “TEST.log” could be different files in the same directory.

Spaces were originally used for pointing multiple arguments of the command, so if you use them in file names within the terminal (console), that will cause incorrect behavior. Therefore, you should use underscore or CamelCase (PascalCase) instead (e.g. “test_log” or “testLog” instead of “test log”).

The “mv” command should be used if you need to rename a file: mv test.tar.gz temptest.tar.gz. That will change the name of “test.tar.gz” to “temptest.tar.gz.”

Overwritten or modified files couldn’t be restored to their original state in Linux, since this OS doesn’t have an “undo” function. The same thing happens if you need to restore a file that was deleted earlier. Linux has neither a “Trash” nor “Recycle” bin. Moreover, you have no chance to restore deleted files and folder using standard tools in most Unix distributions. This is why you should be careful when working with Linux. You have to make sure you delete files you really don’t need; specify unique parameters for the rm command. For example: rm -i test*.txt (user will be prompted). Finally, the alias command will help to reconfigure the rm command call if you really care about data loss (which most of us do).
You should always keep in mind shortcuts: current directory (.) and parent directory (..). Never miss them and do not run: rm -r .* This command will delete the parent directory (the expression matches “..”).

Be aware of the autocomplete function for command or file names if you work within a console: type a few first characters of the name and press the Tab key.

If you need access to recent command history, you can use up and down arrows on the keyboard to browse commands you previously ran.

What are the limits?

Next, let’s explore path types as well as name length limitations in the *nix OS. It’s best to start with common terms.

There are two types of paths: absolute and relative. An absolute path is the location of a file or directory from the root directory (top level): e.g. /var/log/protocol/log. Relative path means path related to the current directory (pwd). For example, you are located in /var/log and you want to go to the directory /var/log/protocol/log/. You can use relative path here, so apply: cd protocol/log/.

As for limitations applied to folder and file names in *nix, there are 256 characters for a name and 1,024 characters for an absolute path (these limitations should also be checked during the test of your web application).
When working on Linux, you cannot log in as the root user (technically, the top level user or administrator), which is either prohibited or impossible due to an unknown or hidden password used as part of the security policy. At the same time, most of daily routine administrative tasks require administrator permissions: web app start/stop, database restarting/cleaning, new build deployments and so on.

To complete those tasks, you have another solution: use sudo commands (requires a password as well – stands for super user do). Just use sudo followed by the required command to perform activities with so-called super user permissions: sudo apt-get install shellutilities.

In the next part of the article we`ll continue discussing testing specifics of Linux OS.

First, let’s have a “helicopter view” on what comparative test provides and which cases are appropriate for back-to-back in Telecom industry.

Most commonly, comparative testing is used in the case of the full replacement of OSS/BSS solution, which is absolutely crucial for any Telecom business. Rarely, new version of existing OSS/BSS is installing and requires verification.

Comparative testing checks two identical OSS/BSS with the same input data in order to reveal incorrect data processing.

Three main goals to use comparative tests

1. Detection of functional defects in Rating and Billing systems;
2. Detection of migration errors when user data transferred from one system to another;
3. Configuration defects identification (setting tariffs in both systems).

The general workflow for comparative test (proposed by the author while implementing projects in Telecom) is shown at the picture below.

As you can see, two identical systems are required for the testing process: the master (“Golden”) and the test system itself.

Sometimes the input data for “Golden” system cannot be accepted by the test system without additional pre-processing. Then, test data must undergo further conversion phase for compatibility with the test system. After getting results from both systems, they have to be compared – the essence of comparative test – that is, as a rule, an automated process implemented independently from environments. The final step is an empirical data analysis activity that is performed by the tester.

When we analyze the efficiency of comparative test regarding OSS/BSS solutions on real Telecom projects, we must have a critical view and list some of the shortcomings:

• The test involves a lot of empirical work, which is difficult to automate (final stages of divergence analysis are performed manually).
• During late iterations of the test, a lot of records with numerical discrepancy that cannot be considered as defects can be revealed. These records must be filtered from comparison
• Sometimes the test indicates good results while in reality the situation is bad. For instance, during the collection of final statistics, two critical defects might neutralize each other when one of them increases and the other decreases the final amount.
• Completely relevant comparative test should be performed for at least one full billing period (about one month), which is not usually the case as the test lasts less than 30 days.

As for the billing systems comparative tests requires achieving a sufficiently high level of coincidence of the output results for both systems (for instance, 99.99%). Achieving this result is a bit tricky, due to the following factors:

• Different rounding of floating-point data in two systems;
• Incorrect development of mapping tables (service A in the system 1 corresponds to service B in the system 2);
• Different order of records processing in two systems may lead to divergence in tariff discounts application;
• Peculiarity of some data types in database (example – rating results for call forwarding);
• Dynamism of the master (“Golden”) environment. In the real life, clients are frequently changing tariff plans, phone numbers, SIM-cards, contract statuses etc. It’s almost impossible to synchronize all these changes in the test system;
• Confirmed changes in business-logic, that is the differences in behavior between the two systems envisaged by Telecom operator requirements to the new system;
• Uncertainty and indistinctness of the system requirements;
• Inability to implement the requirements of the system under test.

All these factors lead to numerical discrepancies in the results, which may be up to 25% from reference values.

Nevertheless, based on my experience I would advise implementing back-to-back testing due to the many advantages that this method provides:

• High level of test coverage for the system;
• The opportunity to get additional metrics of the system quality;
• Availability of implementing migration and configuration tests, not only functional tests;
• High automation level of the testing process.

This is confirmed by the picture below illustrating results of several testing iterations on a real Telecom project.

These statistics contains three metrics:

1. Defects – the number of functional, integration and configuration defects in the test system
2. Discrepancy in records, % – the percentage of data units which gave different output results, relative to the originally loaded amount
3. Discrepancy in amounts, % – the ratio of the total discrepancies amount between the master and the test system relative to the total amount of charges in the master system

Conclusion

We can note high efficiency of this type of test in relation to OSS/BSS. A good practice is to use a back-to-back test together with other traditional strategies for testing, since they are not mutually exclusive and able to detect defects of different classes of the same functionality. You should also keep in mind that an adequate strategy must be developed for back-to-back tests, taking into account all advantages and shortcomings presented in the article.

The article was published at SoftwareTestingMagazine.

We continue the topic of design defects. In case you missed the first part of the article, you can read it here.

Graphic description

There is another group of graphic defects connected not to usability, but mostly the application functionality itself.

Among them: FPS rate (frames per second). Is it good enough for high-performance games? Does the application display properly both in portrait and landscape orientations on the screen?

This group of possible defects is a must-know for any mobile QA engineer since they are official mobile development compliances.

The most useful and known among them are iOS Human Interface Guidelines (iOS HIG) and Android User Interface Guidelines (Android UIG).

These documents could help provide serious arguments in discussions with developers about potential “won’t fixes.”

Recommendations presented in these docs are not strict rules, especially for Android, but they are really useful in terms of user interaction and pretty stable for a long time with minor changes, which is a good sign that they are truly functional.

Below you can find a selection of such recommendations concerning design and a small part of the whole list. For the full version please see the original documents:

• The app should not contain depictions of gratuitous violence (iOS)
• The app should not contain materials advocating against groups of people based on their race, religion, disability, gender, age, etc. (iOS)
• The app should not contain sexually explicit or erotic content, icons, titles or descriptions (Android and iOS)
• Use illumination and dimming to respond to touches, reinforce the resulting behavior of gestures, and indicate what actions are enabled and disabled (Android)
• UI elements should have 8dp spacing (Android)
• Touchable UI components should be laid out along 48dp units (Android)
• Custom UI elements should not be used for a standard action (iOS)
• UI must be optimized both for Retina and non-Retina displays (iOS)
• UI elements from different versions of iOS should not be mixed (iOS)
• The app must not contain materials or services that facilitate online gambling (Android)

THESE ARE just a few examples of usability and graphics recommendations that are presented in the official documents for iOS and Android – easily accessible when needed to state your case with developers during “won’t fix” discussions.

If nothing else, remember this: even if the application is extremely good in terms of the idea, if fonts are too small to read, colors are too bright, backlighting of important areas is absent, and other usability challenges exist, users will not get past the exterior to experience the gem on the inside.

The article Why app design defects should not be “won’t fix” was published in Mobile Marketer online edition.

Are you sure that users will love the application that you are going to produce? After all, this is the key for commercial success. Unfortunately, sometimes, even great ideas fail due to bad implementation. How do you prevent this from happening?

The answer is proper testing before the product hits the market. At the same time, however, when it comes to usability and design issues detected by a tester, it is quite common to hear things such as, “Not too serious,” “No time to fix” or “Too expensive!”

It is important to avoid the “won’t fix” reaction from the development team in the event that the tester finds a significant defect. Below are some tips to help ensure applications’ quality and that all defects found will be fixed.

Views on use

First of all, let us talk about usability of a mobile application. No, we will not touch the usability testing in its full and heavy sense – with a lot of users for evaluation and scrupulous analysis of the target audience.

Rather, I mean the base layer of usability checks – the layer that the functional tester is able to perform himself or herself without investing a huge amount of time and money.

The key things are, of course, clear logic and simplicity. This applies to all kinds of applications – games, office suites, enterprise software, everything.

Is the application intuitive enough to use it easily right after the installation? If no, does the application have clear help, FAQs, tips, other useful FTU (first-time user) resources? Are these resources easily accessible?

These are the questions that the tester should answer while implementing usability tests.

If we consider games, it is important to examine the power balance and energy saving system.

For any type of mobile organizer, it is essential to find out if users are able to see the most important events and functions at the starting screen. They will also want a quick and easy way to add a new note to the organizer.

Listed issues are based mostly on the functionality of the application. In fixing them, you could help to keep users from abandoning the application – considering it illogical or useless.

Also important and connected to usability is an application’s graphics – how the design corresponds to the application’s purpose.

Would you find it appropriate to use all the colors of the rainbow for highly secure government applications? Certainly not. This is unnecessary and would look ridiculous.

That is why it is so important to use testing methods while evaluating application design. In doing so, you will most likely need several devices with different specs.

Texts and objects used in the applications should be noticeable enough and stand out against the background color.

If the font is difficult to read, this may lead to increased user’s annoyance.

You also need to get a negative response to the following questions to ensure that the application is good in terms of design.

For example, is the text too small for the 5-inch device screen? Are there too many targets or icons on the screen? Does the application respond correctly to standard gestures – for instance, “pinch in” is commonly used by mobile devices for zoom-out?

In the next post we`ll cover the issues of graphic defects.

The article Why app design defects should not be “won’t fix” was published in Mobile Marketer online edition.

The Raspberry Pi is a credit-card-sized single-board computer developed in the UK by the Raspberry Pi Foundation to promote studying basic computer science at schools. Raspberry Pi was first introduced as a prototype in late 2011 as a tool to teach and learn programming.

Most buyers, once they get their hands on the new RPi, follow the getting started instructions on the Raspberry Pi site running recommended Raspbian proprietary OS. Kano OS is a fork of Raspbian OS, a Debian Linux Distro. It’s an operating system designed for simplicity, speed, and code learning, targeted at new Raspberry Pi users. It dynamically adjusts the Raspberry Pi’s clock-speed when load reaches 100.

Still, there’s a wealth of other operating systems available on the market. But the more alternatives, the harder to choose. Thus, the preferences should be defined and then compared. QA consulting comes to help with this – below, we provide our benchmark on the three operating systems for Raspberry Pi based on our comprehensive testing.

How the benchmark was performed

We selected three Operating Systems Raspbian OS, KANO OS, Pidora OS and conducted different benchmark tests to define how performance characteristics of these operating systems varied.

21 different tests were run, and the results were compared and analysed based on the benchmarks` characteristics.

The following characteristics or measurements were considered:

• Each test was launched in the same system state.
• No other functions or applications were active in the system unless the scenario included some activity running in the system.
• Launched applications used memory even when they were minimized or idle, which could increase probability of garbling the results.
• The hardware and software used for benchmarking match the production environment.
• Three identical boards were used with Kano OS Beta 1.0,2 , Pidora 2014 (Raspberry Pi Fedora Remix, version 20) and Raspbian Debian Wheezy (version January 2014) operating systems, installed on SD cards.
• Benchmarks were launched via commands in Terminal, where real-time activities and results were displayed.

Benchmarks used

Below is a list of all Benchmarks used and the information on successful or failed launch.

Final OS benchmark score

Not all operating systems had success in launching some of the benchmarks and several benchmarks were not objective (PSTree, Top, HardInfo, sysv-rc-conf), so the score can be considered an approximate one.

Overall, Kano OS outperformed Pidora OS and Raspbian OS.
The measurements are approximate and are not 100% scientifically correct. Still, we intended to get a rough idea of how the systems perform. The performance benchmarks and the values shown here were received using particular well-configured and carefully installed systems. All performance benchmark values are provided “AS IS” and no warranties or guarantees are given or implied by a1qa. Actual system performance may vary and is dependent upon many factors including system hardware configuration, software design, and configuration.

Full report with test results data and benchmarks descriptions can be provided upon your request.

As a continuation to my former post, Testing Models: How Did You Ever Live Without Them, this post will detail the most effective testing model tools: Microsoft Test Manager, JIRA Zephyr and HP Quality Center.

The table below provides an at-a-glance comparison of these test tracking systems that can be used when delivering both manual and automation testing services.

Below is a more detailed description of each of the above test tracking systems.

Microsoft Test Manager

Microsoft Test Manager (MTM) simplifies testing of a developed application. It saves testing plans results on its Team Foundation Server (TFS). If you don`t need all Microsoft Test Manager functions, use Team Web Access to plan and launch tests.

• Exploratory testing – Allows writing down actions during test execution without pre-planned steps.
• Performing manual tests – This function allows displaying of the test case on the screen side during test performance. You can automatically record actions, make screenshots and perform other diagnostics to include in results and errors reports.
• Test configuration: Specification of testing platforms – You can develop several website versions for different hardware or software configurations.
• Collection of extra data for diagnostics in manual tests – This function allows you to record the event logs, data IntelliTrace, videos and other diagnostic data during the test.
• Windows Store applications testing – Applying Microsoft Test Manager installed on a separate device can help to collect diagnostic data and screenshots during test performance.
• Plan application tests in Microsoft Excel or Microsoft Word – This function allows you to use Microsoft Excel for editing of testing plans and synchronizing with them.
• Testing in lab environment – This function allows collecting diagnostic data from servers during test performance. Testers can manage functioning of server/computers and can quickly set up new test configurations including usage of virtual machines.
• System tests automation – MTM allows interconnecting testing methods in code to imitate manual tests and regularly repeat them. You can also automate whole process “build-deploying-testing”.

JIRA Zephyr

If your company currently uses Atlassian JIRA, JIRA Zephyr would be the obvious choice. JIRA is a commercial product licensed for operating on a local server or accessible as a remote application.

• Full integration with JIRA – You can create test cases, test plans and testing reports applying only the JIRA system. All team members can be involved in the testing process. Developers get easy access to test cases. In the event changes are made to the test model, they get immediate notification. The system provides quick access to testing reports for managers, clients and the development team.
• Similar configuration with development projects – As soon as a project is created in JIRA, the testing team can launch the process of test cases and test plans development. Unlike other systems, you don`t have to create new users, components, or iterations for testing tasks, which saves time.
• Simplified search of tests and entities – You can create test plans with saved filters, or set up a search of a certain group of test cases. Testers can create infographics using standard JIRA means and make them available for the entire team.

HP Quality Center

HP Quality Center includes five interrelated modules providing process continuity:

• Management – used for registration of OS releases planned for testing. Release entity can have child entity “Cycle” indicating testing cycle.
• Requirements management – module of requirements creation. Each requirement typically corresponds to one “requirement” entity. One entity can have several sub-entities, and there also can be several types of entities.
• Test plan – module of testing plans development. You can generate a testing hierarchy compatible with a requirements hierarchy. A detailed description along with the expected system state can be included in test cases.
• Test lab – module used for combining tests in sequences and setting conditions for test launch. Depending upon the success of previous tests, you can schedule tests to launch or do it manually.
• Defects management – applying the module, you can track the system of defects, and it is integrated with other modules; thus, you can register the defect in requirements, in test plan or in test lab modules.

Considering the above, we can make the following conclusions:

JIRA Zephyr is the preferred option for those who already use JIRA, as these systems are fully integrated. The advantage of this system is a convenient means of visualizing and collecting metrics and a relatively low price compared to similar products.

MTM is a very powerful tool, especially combined with Team Foundation Server. It has a number of unique features, allowing users to automate system tests, and conduct exploratory testing with simultaneous search of defects and the creation of test cases.

HP Quality Center is a professional product for control of test documentation. Its main advantage is a separate unit for monitoring requirements. It allows users to visualize and make transparent the coverage of requirements test cases; and easily manage the requirements for large and complex systems; however, the cost of the product is high.

Chances are you have not heard of Social Payment Media (SPM), and for good reason. This is a new term. But the trend behind the term – the integration of social media and payment systems – is growing.

Facebook recently announced the development of its own payment system. The company is close to obtaining approval from the Central Bank of Ireland to start a service that would allow users to store money on Facebook and use it to pay and exchange with others, including mobile payments.

Apple is getting close to releasing mobile payment services as well, illustrating the trend: the divide between payment systems and social messengers is fading fast.

Why is this happening?

With the onset of mobile digital communications, people’s behavior has changed dramatically. Data transmission time has reduced, making it easier to organize and monitor business and personal activities. The interaction between people has reached a new level. At first, the mobile message traffic share did not exceed 30 percent of the total communication services’ scope, but with continuous technology developments, the mobile Internet is flourishing. Even popular online business domains strive to mobilize their businesses: E-Commerce => M-Commerce.

The uprising of e-trade has led to a considerable boost of payment services from pioneering online payment systems, such as PayPal and Authorize.net. As a result, today, everyone can get what they need without leaving home; the “pay and be paid online” principle has been established. For example, anyone with a bank account (credit or debit card) can create an eWallet account and pay online for goods and services, using both cash and its electronic equivalent. Furthermore, often the exchange of e-money is more profitable due to partnership programs run by different payment systems.

Simultaneously, highly interactive social media platforms have launched. Social media seriously change the way people communicate in the modern world. It’s no wonder this medium is becoming more and more dependent on mobile technology, and deeply connected to instant messaging tools.

As is the case with social networks, popular instant-messenger offerings like WhatsApp and Viber are becoming an essential part of our smartphone or tablet desktop. The use of push-notification technology has given new life to the development of mobile messengers. As a rule, they are very simple but have comprehensive functionality; they transfer different types of images and videos, and even geo-location parameters. The only thing missing is mobile payments. To place this last piece in the puzzle, we must connect the link between social media, instant messengers and payment systems. In doing so, we get the fastest and safest method to transfer money, providing that mobile apps testing has verified there are no security breaches.

What does the future hold for social payment media?

I foresee the near future will bring a merging of mobile messenger-identification passwords and electronic wallet IDs. Not only will this allow data transfer, but it will significantly assist in quickly and easily transferring money and e-payments. This will expand the functionality and turn the messaging channel into a cash transaction pipe. Thus, e-Wallet could be linked directly to a mobile phone number (not a nickname, username and password), which makes the overall system much more safe and efficient. SPM is a completely new domain – an alliance of payment systems, social messengers and networks – with a bright future.

Typically, testing needs proper documentation explaining where, what and how to test. If testing of standard software needs checklists or test scenarios, OSS/BSS testing definitely requires a testing model.

What is a testing model?

It is a set of all possible scenarios connected with business requirements and stored in test-tracking systems. In the testing model test, scenarios are usually described best of all. They include all the necessary scripts, links to the documents and requirements.

Naturally, testing won’t go through absolutely all tests every time; it will only go through those that are required for that particular stage.

What provides such a detailed testing model?

First, requirements get changed when new functionality is implemented. Specific requirements and test cases allow organizations to easily define what they need to test in the near term and what to leave for the regression. Beginning this process with this step makes it less likely to miss a defect and significantly saves time needed to prepare for the test.

Second, the test cases can be used to train new engineers who need to support the billing system. Executing test cases with good content extends their knowledge of the system and its business processes.

Third, during the testing process, the cases described in detail can be performed not only by test engineers, but also by CSRs. When simultaneously developing OSS/BSS and executing test cases, it also aids in saving time and budget.

No matter how good the system is documented, organizations will benefit even more from knowledge and documentation of testing model implementations. Stay tuned: I’ll discuss what tools are most effective in my next article.

In previous articles we discussed, how to define the quality rate, evaluate testers` capacity and covered bug description method. All these things concerned tester`s work, but a tester is the one who detects a bug, while a developer fixes it. So this time I would like to cover some bits of a DEVELOPER`s niche on the QA consulting map. Taking bug statics as a basis, I want to tell about the indexes that reflect developer`s work efficiency.

First one is about defect lifecycle continuity. The index comprises “defect processing period”, “defect fixing period” and some other elements.

When running these metrics pay special attention to the fact how long it took to fix the defect and which step was most intense. Remember to track the index changes and put them in to the diagram, like this one, for example.

Defect lifecycle

The method of data collection resembles the one I described in the first article of QA metrics for managers, when told about defects having status “functions as designed”.

When you start calculating, you`ll see that index increases due to the defect lifecycle longevity. Considerable can be observed on stages like processing, assigning and queuing.

The other one index is the percentage of Rejected bugs. Defect is considered to be rejected, when a tester gives it status of a non-fixed one after checking. That means that a cycle has to be renewed: report to a developer, re-fix, re-check. The time is actually wasted for additional communication. On large scale projects such things cause additional expenditures, which is quite tangible.

Having exercised multiple projects, I noticed that 10% is an acceptable value for the index; try to keep to that limit. It is better to calculate the index after every release, especially on a large scale project, when more than two builds is released per week.

Remember to add the index in every report, to keep the tester and developer teams informed. Doing this you stimulate the team members to bring down the index percentage and define the positive work dynamics.