Crowdsourced Software Testing: A Timely Opportunity
DOI:
https://doi.org/10.18034/ei.v8i1.491Keywords:
Crowdsourcing, Software Testing, intermediaries, expertiseAbstract
The concept of crowdsourcing has gained a lot of attention lately. Many companies are making use of this concept for value creation, as well as the performance of varied tasks. Despite its wide application, little is known about crowdsourcing, especially when it comes to crowdsourced software testing. This paper explores the crowdsourced software testing concept from a wider perspective ranging from a cost-benefit analysis, crowdsourcing intermediaries, and the level of expertise in the crowd. Drawing from a varied range of sources, a systematic literature review is done, where the research narrows down to ten most relevant peer-reviewed sources of high impact rating. In a comparative analysis between crowdsourced software testing and in-house testing, it is found that crowd testing has numerous advantages when it comes to efficiency, user heterogeneity, and cost-effectiveness. The study indicates that intermediaries play a key role in managing the connection between the crowd and crowdsourcing companies despite various challenges. A comparison between novice testers and expert testers reveals that both the two have their unique capabilities in their respective domains.
Downloads
References
Billington, C., & Davidson, R. (2013). Leveraging open innovation using intermediary networks. Production and Operations Management, 22(6), 1464-1477. DOI: https://doi.org/10.1111/j.1937-5956.2012.01367.x
Daniel, F., Kucherbaev, P., Cappiello, C., Benatallah, B., & Allahbakhsh, M. (2018). Quality control in crowdsourcing: A survey of quality attributes, assessment techniques, and assurance actions. ACM Computing Surveys (CSUR), 51(1), 1-40. DOI: https://doi.org/10.1145/3148148
Donepudi, P. K. (2017). Machine Learning and Artificial Intelligence in Banking. Engineering International, 5(2), 83-86. https://doi.org/10.18034/ei.v5i2.490 DOI: https://doi.org/10.18034/ei.v5i2.490
Foong, E., Gergle, D., & Gerber, E. M. (2017). Novice and Expert Sensemaking of Crowdsourced Design Feedback. Proceedings of the ACM on Human-Computer Interaction, 1(CSCW), 1-18. DOI: https://doi.org/10.1145/3134680
Leicht, N., Blohm, I., & Leimeister, J. M. (2017). Leveraging the power of the crowd for software testing. IEEE Software, 34(2), 62-69. DOI: https://doi.org/10.1109/MS.2017.37
Leicht, N., Knop, N., Blohm, I., Müller-Bloch, C., & Leimeister, J. M. (2016). When is crowdsourcing advantageous? The case of crowdsourced software testing. DOI: https://doi.org/10.2139/ssrn.3159166
Leicht, N., Rhyn, M., & Hansbauer, G. (2016). Can laymen outperform experts? The effects of user expertise and task design in crowdsourced Software Testing.
Liu, D., Bias, R. G., Lease, M., & Kuipers, R. (2012). Crowdsourcing for usability testing. Proceedings of the American Society for Information Science and Technology, 49(1), 1- 10. DOI: https://doi.org/10.1002/meet.14504901100
Muhdi L, Boutellier R (2011) the crowdsourcing process: an intermediary mediated idea generation approach in the early phase of innovation. Int J Entrepren Innov Manag 14(4):315–332. doi:10.1504/IJEIM.2011.043052 DOI: https://doi.org/10.1504/IJEIM.2011.043052
Stol, K. J., & Fitzgerald, B. (2014, May). Two's a company; three's a crowd: a case study of crowdsourcing software development, in Proceedings of the 36th International Conference on Software Engineering (pp. 187-198). DOI: https://doi.org/10.1145/2568225.2568249
Zogaj, S., Bretschneider, U., & Leimeister, J. M. (2014). Managing crowdsourced software testing: a case study based insight on the challenges of a crowdsourcing intermediary. Journal of Business Economics, 84(3), 375-405. DOI: https://doi.org/10.1007/s11573-014-0721-9
--0--
Downloads
Published
Issue
Section
License
Engineering International is an Open Access journal. Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal the right of first publication with the work simultaneously licensed under a CC BY-NC 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of their work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal. We require authors to inform us of any instances of re-publication.
