| 2018 | |
| [1] | , "Knowledge Management in Software Testing: A Systematic Snowball Literature Review", In e-Informatica Software Engineering Journal, vol. 12, no. 1, pp. 51–78, 2018.
DOI: 10.5277/e-Inf180103. Download article (PDF)Get article BibTeX file |
Authors
Krzysztof Wnuk, Thrinay Garrepalli
Abstract
Description : Software testing benefits from the usage of Knowledge Management (KM) methods and principles. Thus, there is a need to adopt KM to the software testing core processes and attain the benefits that it provides in terms of cost, quality, etc.
Aim : To investigate the usage and implementation of KM for software testing. The major objectives include 1. To identify various software testing aspects that receive more attention while applying KM. 2. To analyse multiple software testing techniques, i.e. test design, test execution and test result analysis and highlight KM involvement in these. 3. To gather challenges faced by industry due to the lack of KM initiatives in software testing.
Method : A Systematic Literature Review (SLR) was conducted utilizing the guidelines for snowballing reviews by Wohlin. The identified studies were analysed in relation to their rigor and relevance to assess the quality of the results.
Results : The initial resulting set provided 4832 studies. From these, 35 peer-reviewed papers were chosen among which 31 are primary, and 4 are secondary studies. The literature review results indicated nine testing aspects being in focus when applying KM within various adaptation contexts and some benefits from KM application. Several challenges were identified, e.g., improper selection and application of better-suited techniques, a low reuse rate of software testing knowledge, barriers in software testing knowledge transfer, no possibility to quickly achieve the most optimum distribution of human resources during testing, etc.
Conclusions : The study brings supporting evidence that the application of KM in software testing is necessary, e.g., to increase test effectiveness, select and apply testing techniques. The study outlines the testing aspects and testing techniques that benefit their users.
Keywords
KM, software testing, knowledge, systematic literature review
References
[1] E.F. de Souza, R. de Almeida Falbo, and N.L. Vijaykumar, “Knowledge management initiatives in software testing: A mapping study,” Information and Software Technology, Vol. 57, 2015, pp. 378 – 391. [Online]. http://www.sciencedirect.com/science/article/pii/S0950584914001335
[2] T.H. Davenport and L. Prusak, Working knowledge: How organizations manage what they know. Harvard Business Press, 1998.
[3] J. Andrade, J. Ares, M.A. Martínez, J. Pazos, S. Rodríguez, J. Romera, and S. Suárez, “An architectural model for software testing lesson learned systems,” Information and Software Technology, Vol. 55, No. 1, 2013, pp. 18–34.
[4] Y. Liu, J. Wu, X. Liu, and G. Gu, “Investigation of knowledge management methods in software testing process,” in International Conference on Information Technology and Computer Science, Vol. 2. IEEE, 2009, pp. 90–94.
[5] R. Abdullah, Z.D. Eri, and A.M. Talib, “A model of knowledge management system in managing knowledge of software testing environment,” in 5th Malaysian Conference in Software Engineering (MySEC). IEEE, 2011, pp. 229–233.
[6] A. Desai and S. Shah, “Knowledge management and software testing,” in Proceedings of the International Conference & Workshop on Emerging Trends in Technology. ACM, 2011, pp. 767–770.
[7] K. Nogeste and D.H. Walker, “Using knowledge management to revise software-testing processes,” Journal of Workplace Learning, Vol. 18, No. 1, 2006, pp. 6–27.
[8] L. Xu-Xiang and W.N. Zhang, “The PDCA-based software testing improvement framework,” in International Conference on Apperceiving Computing and Intelligence Analysis (ICACIA. IEEE, 2010, pp. 490–494.
[9] X. Li and W. Zhang, “Ontology-based testing platform for reusing,” in Sixth International Conference on Internet Computing for Science and Engineering (ICICSE). IEEE, 2012, pp. 86–89.
[10] J. Kajihara, G. Amamiya, and T. Saya, “Learning from bugs (software quality control),” IEEE Software, Vol. 10, No. 5, 1993, pp. 46–54.
[11] C. O’Dell and C. Jackson Grayson Jr, “Knowledge transfer: discover your value proposition,” Strategy & Leadership, Vol. 27, No. 2, 1999, pp. 10–15.
[12] I. Nonaka and H. Takeuchi, The knowledge-creating company: How Japanese companies create the dynamics of innovation. Oxford University Press, 1995.
[13] A.D. Marwick, “Knowledge management technology,” IBM Systems Journal, Vol. 40, No. 4, 2001, pp. 814–830.
[14] O.K. Wei and T.M. Ying, “Knowledge management approach in mobile software system testing,” in IEEE International Conference on Industrial Engineering and Engineering Management. IEEE, 2007, pp. 2120–2123.
[15] T. Abdou and P. Kamthan, “A knowledge management approach for testing open source software systems,” in International Performance Computing and Communications Conference (IPCCC). IEEE, 2014, pp. 1–2.
[16] J. Sirathienchai, P. Sophatsathit, and D. Dechawatanapaisal, “Simulation-based evaluation for the impact of personnel capability on software testing performance,” Journal of Software Engineering and Applications, Vol. 5, No. 08, 2012, p. 545.
[17] V.H. Nasser, W. Du, and D. MacIsaac, “An ontology-based software test generation framework.” in The 22nd International Conference on Software Engineering and Knowledge Engineering, SEKE, 2010, pp. 192–197.
[18] H. Li, F. Chen, H. Yang, H. Guo, W.C.C. Chu, and Y. Yang, “An ontology-based approach for gui testing,” in 33rd Annual IEEE International Computer Software and Applications Conference, Vol. 1. IEEE, 2009, pp. 632–633.
[19] E.F. Barbosa, E.Y. Nakagawa, and J.C. Maldonado, “Towards the establishment of an ontology of software testing.” in International Conference on Software Engineering & Knowledge Engineering, 2006, pp. 522–525.
[20] N. Juristo, A.M. Moreno, and S. Vegas, “Reviewing 25 years of testing technique experiments,” Empirical Software Engineering, Vol. 9, No. 1, 2004, pp. 7–44.
[21] A.C.C. Natali, A.R.C. da Rocha, G.H. Travassos, and P.G. Mian, “Integrating verification and validation techniques knowledge into software engineering environments,” Proceedings of 4as Jornadas Ibeoamericanas de Ingeniería del Software e Ingeniería del Conocimiento, JIISIC, Vol. 4, 2004, pp. 419–430.
[22] N. Juristo, A.M. Moreno, and S. Vegas, “Towards building a solid empirical body of knowledge in testing techniques,” ACM SIGSOFT Software Engineering Notes, Vol. 29, No. 5, 2004, pp. 1–4.
[23] R.L. Glass, R. Collard, A. Bertolino, J. Bach, and C. Kaner, “Software testing and industry needs,” IEEE Software, Vol. 23, No. 4, 2006, pp. 55–57.
[24] R. Jain and S. Richardson, “Knowledge partitioning and knowledge transfer mechanisms in software testing: An empirical investigation,” in Proceedings of the 1st Workshop on Advances and Innovations in Systems Testing, 2007.
[25] S. Vegas, “Identifying the relevant information for software testing technique selection,” in International Symposium on Empirical Software Engineering, ISESE. IEEE, 2004, pp. 39–48.
[26] X. Liu, G. Gu, L. Yongpu, and W. Ji, “Research and application of knowledge management model oriented software testing process,” in 11th Joint International Conference on Information Sciences. Atlantis Press, 2008.
[27] A. Beer and R. Ramler, “The role of experience in software testing practice,” in 34th Euromicro Conference Software Engineering and Advanced Applications. IEEE, 2008, pp. 258–265.
[28] J. Itkonen, M.V. Mäntylä, and C. Lassenius, “The role of the tester’s knowledge in exploratory software testing,” IEEE Transactions on Software Engineering, Vol. 39, No. 5, 2013, pp. 707–724.
[29] O. Taipale, K. Karhu, and K. Smolander, “Observing software testing practice from the viewpoint of organizations and knowledge management,” in First International Symposium on Empirical Software Engineering and Measurement. IEEE, 2007, pp. 21–30.
[30] L. Xue-Mei, G. Guochang, L. Yong-Po, and W. Ji, “Research and implementation of knowledge management methods in software testing process,” in WRI World Congress on Computer Science and Information Engineering, Vol. 7. IEEE, 2009, pp. 739–743.
[31] B.A. Kitchenham, T. Dyba, and M. Jorgensen, “Evidence-based software engineering,” in Proceedings of the 26th International Conference on Software Engineering. IEEE Computer Society, 2004, pp. 273–281.
[32] J. Webster and R.T. Watson, “Analyzing the past to prepare for the future: Writing a literature review,” MIS quarterly, 2002, pp. xiii–xxiii.
[33] W. Hayes, “Research synthesis in software engineering: a case for meta-analysis,” in Sixth International Software Metrics Symposium. IEEE, 1999, pp. 143–151.
[34] C. Wohlin, “Guidelines for snowballing in systematic literature studies and a replication in software engineering,” in Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, ser. EASE ’14. New York, NY, USA: ACM, 2014, pp. 38:1–38:10. [Online]. http://doi.acm.org/10.1145/2601248.2601268
[35] B. Kitchenham, O.P. Brereton, D. Budgen, M. Turner, J. Bailey, and S. Linkman, “Systematic literature reviews in software engineering – a systematic literature review,” Information and software technology, Vol. 51, No. 1, 2009, pp. 7–15.
[36] B. Kitchenham, R. Pretorius, D. Budgen, O.P. Brereton, M. Turner, M. Niazi, and S. Linkman, “Systematic literature reviews in software engineering – a tertiary study,” Information and Software Technology, Vol. 52, No. 8, 2010, pp. 792–805.
[37] B. Kitchenham and S. Charters, “Guidelines for performing systematic literature reviews in software engineering,” 2007, No. EBSE 2007-01.
[38] C.W. Knisely and K.I. Knisely, Engineering communication. Cengage Learning, 2014.
[39] D.S. Cruzes and T. Dybå, “Research synthesis in software engineering: A tertiary study,” Information and Software Technology, Vol. 53, No. 5, 2011, pp. 440–455.
[40] C. Goulding, Grounded theory: A practical guide for management, business and market researchers. Sage, 2002.
[41] R. Hoda, J. Noble, and S. Marshall, “Using grounded theory to study the human aspects of software engineering,” in Human Aspects of Software Engineering. ACM, 2010, p. 5.
[42] B.G. Glaser, A.L. Strauss, and E. Strutzel, “The discovery of grounded theory; strategies for qualitative research.” Nursing research, Vol. 17, No. 4, 1968, p. 364.
[43] M. Dixon-Woods, S. Agarwal, D. Jones, B. Young, and A. Sutton, “Synthesising qualitative and quantitative evidence: a review of possible methods,” Journal of Health Services Research & Policy, Vol. 10, No. 1, 2005, pp. 45–53.
[44] D. Koznov, V. Malinov, E. Sokhransky, and M. Novikova, “A knowledge management approach for industrial model-based testing,” in Proceedings of the International Conference on Knowledge Management and Information Sharing, 2009, pp. 200–205.
[45] M. Rodgers, A. Sowden, M. Petticrew, L. Arai, H. Roberts, N. Britten, and J. Popay, “Testing methodological guidance on the conduct of narrative synthesis in systematic reviews: effectiveness of interventions to promote smoke alarm ownership and function,” Evaluation, Vol. 15, No. 1, 2009, pp. 49–73.
[46] M. Ivarsson and T. Gorschek, “A method for evaluating rigor and industrial relevance of technology evaluations,” Empirical Software Engineering, Vol. 16, No. 3, 2011, pp. 365–395.
[47] A. Jonsson and G. Svingby, “The use of scoring rubrics: Reliability, validity and educational consequences,” Educational Research Review, Vol. 2, No. 2, 2007, pp. 130–144.
[48] B. Moskal, K. Miller, and L. King, “Grading essays in computer ethics: rubrics considered helpful,” ACM SIGCSE Bulletin, Vol. 34, No. 1, 2002, pp. 101–105.
[49] A. Vickers, “Ensuring scientific rigour in literature review,” Acupuncture in Medicine, Vol. 13, No. 2, 1995, pp. 93–96.
[50] P. Runeson, M. Host, A. Rainer, and B. Regnell, Case study research in software engineering: Guidelines and examples. John Wiley & Sons, 2012.
[51] R. Wieringa, N. Maiden, N. Mead, and C. Rolland, “Requirements engineering paper classification and evaluation criteria: A proposal and a discussion,” Requirements Engineering, Vol. 11, No. 1, 2006, pp. 102–107.
[52] P. Goodman Jon and C. Huseman Richard, Leading with Knowledge: The Nature of Competition in the 21st Century. Sage, London, 1999.
[53] L. Rajiv and M. Sarvary, “KM and competition in the consulting industry,” 1999, p. 485.
[54] R. de Almeida Falbo, “Experiences in using a method for building domain ontologies,” in The 16th International Conference on Software Engineering and Knowledge Engineering, SEKE, 2004, pp. 474–477.
[55] S. Vegas and V. Basili, “A characterisation schema for software testing techniques,” Empirical Software Engineering, Vol. 10, No. 4, 2005, pp. 437–466.
[56] A. Abran, P. Bourque, R. Dupuis, and J.W. Moore, Guide to the software engineering body of knowledge – SWEBOK. IEEE Press, 2001.
[57] M. Cataldo, P.A. Wagstrom, J.D. Herbsleb, and K.M. Carley, “Identification of coordination requirements: implications for the design of collaboration and awareness tools,” in Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work. ACM, 2006, pp. 353–362.
[58] D. Graham, E. Van Veenendaal, and I. Evans, Foundations of software testing: ISTQB certification. Cengage Learning EMEA, 2008.
[59] C. Wohlin, P. Runeson, M. Höst, M.C. Ohlsson, B. Regnell, and A. Wesslén, Experimentation in software engineering. Springer Science & Business Media, 2012.
[60] K. Petersen, R. Feldt, S. Mujtaba, and M. Mattsson, “Systematic mapping studies in software engineering.” in EASE, Vol. 8, 2008, pp. 68–77.
[61] C. Kaner and W.P. Bond, “Software engineering metrics: What do they measure and how do we know?” in In METRICS 2004. IEEE CS. Citeseer, 2004.
[62] E.F. de Souza, R. de Almeida Falbo, and N.L. Vijaykumar, “Knowledge management applied to software testing: A systematic mapping,” in The 25th International Conference on Software Engineering and Knowledge Engineering, SEKE, Boston, USA, 2013, pp. 562–567.
[63] A. Tinkham and C. Kaner, “Learning styles and exploratory testing,” in Proceedings of the Pacific Northwest Software Quality Conference, 2003.
[64] S. Vegas, N. Juristo, and V. Basili, “Packaging experiences for improving testing technique selection,” Journal of Systems and Software, Vol. 79, No. 11, 2006, pp. 1606–1618.
[65] J. Itkonen, M.V. Mantyla, and C. Lassenius, “How do testers do it? An exploratory study on manual testing practices,” in Proceedings of the 3rd International Symposium on Empirical Software Engineering and Measurement. IEEE Computer Society, 2009, pp. 494–497.
[66] A. Freitas and R. Vieira, “An ontology for guiding performance testing,” in Proceedings of the 2014 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT), Vol. 1. IEEE Computer Society, 2014, pp. 400–407.
[67] T.E. Lee, “Applying knowledge management approach for software testing,” 2007.
[68] E.F. de Souza, R. de Almeida Falbo, and N.L. Vijaykumar, “Ontologies in software testing: a systematic literature review,” in VI Seminar on Ontology Research in Brazil, 2013, p. 71.
[69] C. Kerkhof, J. van den Ende, and I. Bogenrieder, “Knowledge management in the professional organization: a model with application to CMG software testing,” Knowledge and Process Management, Vol. 10, No. 2, 2003, pp. 77–84.
[70] S. Vegas, N. Juristo, and V.R. Basili, “Maturing software engineering knowledge through classifications: A case study on unit testing techniques,” IEEE Transactions on Software Engineering, Vol. 35, No. 4, 2009, pp. 551–565.
[71] R. Gentry and F. Shirazi, “A knowledge management analysis of an in-house manual software testing,” International Journal of Computer Application, Vol. 1, No. 5, 2015, pp. 13–37.
[72] E.F. de Souza, R. de Almeida Falbo, and N.L. Vijaykumar, “Using ontology patterns for building a reference software testing ontology,” in 17th IEEE International Enterprise Distributed Object Computing Conference Workshops (EDOCW). IEEE, 2013, pp. 21–30.
[73] V.H. Nasser, W. Du, and D. MacIsaac, “Knowledge-based software test generation.” in The 21st International Conference on Software Engineering and Knowledge Engineering, SEKE, 2009, pp. 312–317.
[74] H. Li, H. Guo, F. Chen, H. Yang, and Y. Yang, “Using ontology to generate test cases for GUI testing,” International Journal of Computer Applications in Technology, Vol. 42, No. 2-3, 2011, pp. 213–224.