Although the main topic of this article is crowdsourced testing, it is essential to clear the air about crowdsourcing in general, as it forms the basis of crowdtesting as a concept and its characteristics. Hence, here we first discuss what crowdsourcing is and its brief history. Then, we connect the concepts of crowdsourcing and crowdtesting, and how the benefits usually obtained by the power of the crowds apply within testing.
What is crowdsourcing?
The term crowdsourcing was coined in 2006 by Jeff Howe in his article for Wired magazine called Rise of Crowdsourcing. A few years later, Howe also wrote a book called Why the power of the crowd is driving the future of business that elaborated on his points made earlier in the article and expanded the topic further in detail. To this day, this book remains one of the most recommended books when it comes to the topic of crowdsourcing.
However, since 2006 the Internet has not stopped growing in the slightest, further flourishing every year, and yielding innovation within technology and the ways people use it. The emergence of crowdsourcing and its success is mostly attributed to the so-called “Web 2.0” era, which also allowed it to further develop in the following decade.
In general, a divide in crowdsourcing definitions can be clearly seen.
Jeff Howe, considered by most as the ‘founding father’ of the field, defined the term as ‘ways in which the power of the many can be leveraged to accomplish feats that were once the responsibility of a specialized few.’ He highlighted how the crowd is not only wise but also creative, talented, and stunningly productive.
Several years later, the industry very much needed a less ambiguous definition. Daren Brabham gave such a definition in 2012 in his book, appropriately named Crowdsourcing.
The shorter version we are listing here goes like this:
Crowdsourcing is a type of participative online activity in which an individual, an institution, a nonprofit organization, or company proposes to a group of individuals of varying knowledge and number, the voluntary undertaking of a task. The undertaking of these tasks of variable complexity and modularity, in which the crowd should participate, bringing their work, money, knowledge, and/or experience, always entails mutual benefit.
The important distinction here is that there is an organization that proposes the task, and the interplay between this organization and the crowd yields mutual benefit.
What is not crowdsourcing?
To showcase what this distinction means in practice, it is critical to distinguish pursuits that do not count as crowdsourcing per se. Contrary to common belief and media articles, the following examples are not considered crowdsourcing.
What is crowdsourced testing?
It is finally time we talk about what crowdtesting is how it fits within crowdsourcing. Although lengthy, the paragraph above gives an important introduction to what crowdsourcing is and what its characteristics are. These aspects of crowdsourcing apply to crowdtesting and lay the ground for the tendencies and benefits of the latter.
Given the definition above, it is clear how testing, organized in a certain way, can be considered as a crowdsourced activity.
This certain way of testing organized to utilize the power of crowds is what Buglance provides as a platform - a platform where companies and teams submit their software applications and the community of Buglance tests and reports bugs.
Firstly, this is considered as a crowdsourced activity as there is an organization, in this case, a company or a tech team that submits a task for the crowds. The tasks in general terms entail making sure that the application at hand does not have defects, or more realistically, has as few defects as technically possible.
Secondly, bug-free software and the uncovering of defects of an application is the common goal for both parties: the company and the crowd.
This activity also leads to mutual benefit. Obviously, the company benefits from finding bugs in the submitted software, and tech teams within the company can get to fixing bugs as soon as they are submitted.
The individual testers, however, get financial rewards based on the complexity of bugs or issues they have reported. Buglance has set up the system and its pricing in a way that makes it financially appealing to both companies and the tester crowds. The company pays a certain amount, according to the pricing plan it has chosen and gets thousands of testers essentially for the price of one. We will expand on this point later in the article.
And testers, on the other hand, spend as much time as they would like on a problem and get financial compensations based on the difficulty of the problem and the amount of effort required.
The platform has shown that a win-win scenario is more than possible, and its model has proven to work extremely well. By now, the tester community of Buglance has grown in numbers to 50,000 testers from more than 150 countries.
More importantly, in just a year, the community has reported more than 250,000 bugs in software applications submitted from all over the world.
To showcase how crowdtesting enables efficiency and how the numbers mentioned are even possible, we will outline some characteristics that crowdsourcing in general, and crowdtesting in particular, have.
Note: it is also important to distinguish between crowdsourced and outsourced testing. Softwaretestinghelp.com has a great article about crowdtesting, which showcases differences in these approaches.
Most significant crowdtesting characteristics
We are going to list some characteristics crowdsourcing brings to testing, as well as some additional ones, which are by-products of such approaches.
1. Crowdtesting enables great scalability
It makes sense that in organizing the crowds to perform a task, more often than not, the number of people involved has to become relatively large. The scale with which the problems are solved enables organizations to effectively tackle issues without the need to hire thousands of people. Individuals from all over the world can start engaging with an issue at hand and offer their solutions and opinions on it.
When it comes to testing, scalability means a lot more than just the number of people involved.
Usually, a software application gets tested in a variety of areas. Those might include usability testing, security testing, functional testing and others. With crowdsourced testing, there is no need to do the testing sequentially and/or partially due to time and resource limitations.
When an application is submitted to Buglance, the preferred testing type for each test cycle can be selected during the setup. This way, the application can be tested within those types in parallel by hundreds of real users. For example, in a car rental mobile application, a group of testers could be reporting bugs on the GUI, while another group is testing the payment process in the app.
2. The testing process is considerably fast
High speed is another attribute of crowdtesting. Due to the number of people involved and the fact that all activities are performed independently and online, tasks get completed at an extremely high pace.
In regards to a software application, testing at such speeds can lead to many benefits. Especially when small and medium-sized companies are trying their best to deliver their product on time, and quality is of utmost importance.
Another positive consequence of the speed and crowd’s involvement is that by outsourcing their testing, teams can focus on the development of the product and its new features.
Instead of coming up with test cases, setting up the tests, and running them, software developers can effectively work and fix reported bugs.
Here in Buglance, we often find tech teams very surprised when they start seeing bug reports within the first few hours of submission. As a result, the whole process of setting up and testing itself is usually cut down by ten times.
Get free ebook about our crowdsourced testing path
3. Diversity in testers and testing environments
Diversity might be one of the most defining characteristics of crowdsourcing. It is usually said that the Internet and crowdsourcing have broken down stereotypes of expertise, proving that innovation does not necessarily come from officially qualified experts.
Since the qualification is not any longer a necessity for anyone to take part in an activity online, the crowds usually consist of highly diverse individuals. People with different backgrounds, cultural identities, and technical skillsets take part in these activities, usually in their free time, to get financial benefits and/or emotional satisfaction.
In crowdtesting, diversity is reflected in several ways. Those also can be categorized into human and environmental diversity.
Human diversity simply means what was mentioned before - people with a diverse set of skills and backgrounds come together to test software. This is particularly important as developers usually get stagnant in their testing approach and have what is called ‘developer bias.’ Meaning, they assume certain user flows, which prove to be overly simplistic and perfect. Crowdtesting allows more explorative testing, whereby real users can find cases of which engineers would have never thought of themselves. In fact, this has been the case for most start-ups Buglance has facilitated testing for, and engineers are left surprised by the reported bugs.
Human diversity, also characterized by geographical variety, lets software be tested within many countries. Testers can report mistakes in the content of an application and issues related to compatibility. For example, if an application is a web-based one, certain countries might have restrictions on specific library or browser usages.
Environmental diversity is a crucial feature of crowdtesting. If you read through user feedback on mobile application stores (e.g., Google Play, App Store), you will see that the majority of negative reviews are related to certain device ranges. It is hard to make sure that when an application is released, it is compatible with outdated or new devices and their libraries. Therefore, by testing on thousands of devices, tech teams can ensure that the number of devices on which an application causes an issue is minimized.
Note that not all crowdtesting platforms allow this feature. In Buglance, environmental diversity is easily showcased through the platform. Currently, the platform holds more than 10,000 unique devices. In fact, when setting up an application for testing, you can also choose a range of devices to test on, their models, brands, OS versions and other technical specifications.
4. Crowdsourced testing saves your budget
Crowdsourcing is well-known for its ability to reduce costs. It does so by splitting the task at hand into smaller chunks and distributing it throughout the community.
Testing might not be on the list of things that a start-up would like to decrease its expenses. However, product development and staff salaries certainly are on this list.
It is estimated that testing can take up anywhere from 20% to 50% of development time, depending on the type and complexity of the software being developed. And an average salary of a single software developer in the United States is around $87,000.
A team of developers usually consists of 4 to 9 developers, bringing the annual expenditure on software engineers to anywhere between $350,000 and $800,000.
And this brings us to the total annual amount spent on testing:
- Minimum of $70,000 (4 developers only)
- A whopping maximum of $400,000
- If you have around 9 developers in your team, working on a very complex user-facing application
It becomes clear that decreasing the amount of time a team of engineers spends on testing can significantly impact a company’s bank statement.
Furthermore, because crowdtesting lets engineers focus on the development of a product and reveal more defects than a normal testing procedure would, it also increases user trust. In the long run, this would also have its effect on the company's reputation, and ultimately the revenue.
In Buglance, by crowdsourcing your testing, you can reduce not only the time your team of engineers spend on testing but also get access to 40,00 testers for the price of one.
5. You have real user interaction throughout the testing
The crowd being a community of real users does not necessarily translate to immense benefits within all industries; however, it does so for testing activities.
There is no doubt that traditional testing methods and outsourced testing tools can help extensively test an application. However, what these methods are commonly missing is real user interaction.
It is, therefore, one of the defining characteristics of crowdtesting, that thousands of real users can help test software both functionally and non-functionally in real environments.
It is important to note that crowdtesting and other testing methodologies have their benefits and drawbacks, and are more suitable for specific types of testing. As mentioned above, crowdtesting can speed up testing endeavors and detect a higher number of bugs more quickly. However, it does not replace traditional testing methods completely. A number of functional testing methods with a rigid structure and test cases might work better for some testing purposes.
Read through our Crowdtesting White Paper to learn more about how real users can bring incredible results within crowdtesting. There, we also write in detail about crowdtesting’s advantages and disadvantages, use cases, and examples.
Conclusion
In our attempt to explain what crowdtesting is, in this article, we have first tried to determine its core concept, crowdsourcing. We have considered the commonly misinterpreted cases of crowdsourcing and why they should not be labeled as such.
Following the foundations of crowdsourcing concepts, we then explored their benefits and how they apply in testing. In this case, these benefits are reflected in many ways, besides the obvious.
To see for yourself how the magic of crowdsourcing applies in testing, you can request a Demo from us and get access to the crowd of 40,000+ testers and 10,000+ devices from around the world.