Crowd testing - Yes or No?
To put it simply, crowd sourced testing (aka crowd testing) is a platform where testers from different geographical locations can test a software or a website or an application hosted on the internet. It is an ecosystem for software testing.
Crowd sourced testing platforms facilitate software testing but not necessarily perform the testing of the websites or applications.
To better explain this, a client (a person or a company who needs software testing to be done) would post a project along with the URL for the website to be tested. This published project is visible to all the registered testers. The testers, who have the right skill set and interest in the project, will choose to participate in testing it. Each project has a set deadline. The testers file the defects within the set deadline. This makes it possible to complete the testing in matter of hours or weeks depending on the depth of testing expected.
The advantages of crowd sourced testing:
1. The whole world becomes a stage for testing the website or application since registered testers are from different geographical location.
2. Cost effective since the client pays only for the accepted defects.
3. Testers are unbiased towards the website/application.
4. Different groups of testers can be targeted to test localization.
5. Requirement of having an in-house testing team is nullified especially for non-IT companies.
6. Out-sourced testing involves minimum payment to the testing company. In crowd sourced testing, the testers may or may not get paid depending the the quality of the defects they file.
7. The incentive to find defects is higher for testers since their payment is directly proportional to the number of their defects that get accepted.
So on the whole, it looks like there is no downside, huh?
Wrong, there are couple of disadvantages to the crowd sourced testing approach.
1. Website/applications with sensitive IP cannot be tested. If the whole premise of the company's success depends on guarding its IP before entering the market, then crowd testing is not possible.
2. The number of defects (especially non-severe ones) getting filed by the testers may be overwhelming. To avoid this, a good test case and a strong set of testing guidelines can be uploaded along with the project details.
3. Unless an application/website supports shared access over the internet, it cannot be crowd tested.
4. If the application/website is not stable enough to go through a iteration of testing, testers may loose interest and move on. This can be avoided by making sure the application/website development is complete before publishing the project.
5. Communicating with the crowd (testers) might be tricky but as long as there is an established channel to clarify the doubts at early stages, this should not be a big problem.
6. Testers focusing on wrong aspects of the website/application. This also can be overcome with a good testcase and testing guidelines explaining the important or critical aspects of the website/application.
These are some of the disadvantages that I can think of. Please feel free to add to my list of disadvantages (and advantages!).
Finally crowd sourced testing is useful when the application/website is user centric. The success of the application/website depends on the user experience.
To sum it up, if the website/application is not IP sensitive, needs a diverse set of users, is user centric in nature and the company's main objective is to go to market as quickly as possible without spending a lot on testing but at the same time getting a thoroughly tested application/website to the market in a stipulated duration, then Crowd sourced testing is a good option to consider.