Empirical Studies on Software Product Maintainability Prediction: A Systematic Mapping and Review

[1]    P. Bourque and R.E. Fairley, Guide to the software engineering body of knowledge (SWEBOK (R)): Version 3.0. IEEE Computer Society Press, 2014.

[2]    P. Oman and J. Hagemeister, “Construction and testing of polynomials predicting software maintainability,” Journal of Systems and Software, Vol. 24, No. 3, 1994, pp. 251–266.

[3]    A. Kaur and K. Kaur, “Statistical comparison of modelling methods for software maintainability prediction,” International Journal of Software Engineering and Knowledge Engineering, Vol. 23, No. 06, 2013, pp. 743–774.

[4]    M. Riaz, E. Mendes, and E. Tempero, “A systematic review of software maintainability prediction and metrics,” in Proceedings of the 3rd International Symposium on Empirical Software Engineering and Measurement. IEEE Computer Society, 2009, pp. 367–377.

[5]    M. Riaz, “Maintainability prediction of relational database-driven applications: a systematic review,” in 16th International Conference on Evaluation & Assessment in Software Engineering. IET, 2012, pp. 263–272.

[6]    B.A. Orenyi, S. Basri, and L.T. Jung, “Object-oriented software maintainability measurement in the past decade,” in International Conference on Advanced Computer Science Applications and Technologies (ACSAT). IEEE, 2012, pp. 257–262.

[7]    S.K. Dubey, A. Sharma, and A. Rana, “Analysis of maintainability models for object oriented system,” International Journal on Computer Science and Engineering, Vol. 3, No. 12, 2011, p. 3837.

[8]    A.M. Fernández-Sáez, M. Genero, and M.R. Chaudron, “Empirical studies concerning the maintenance of UML diagrams and their use in the maintenance of code: A systematic mapping study,” Information and Software Technology, Vol. 55, No. 7, 2013, pp. 1119–1142.

[9]    A.M. Fernández-Sáez, D. Caivano, M. Genero, and M.R. Chaudron, “On the use of UML documentation in software maintenance: Results from a survey in industry,” in 18th International Conference on Model Driven Engineering Languages and Systems (MODELS). IEEE, 2015, pp. 292–301.

[10]    K. Petersen, S. Vakkalanka, and L. Kuzniarz, “Guidelines for conducting systematic mapping studies in software engineering: An update,” Information and Software Technology, Vol. 64, 2015, pp. 1–18.

[11]    B.A. Kitchenham, D. Budgen, and O.P. Brereton, “Using mapping studies as the basis for further research – A participant-observer case study,” Information and Software Technology, Vol. 53, No. 6, 2011, pp. 638–651.

[12]    A. Idri, F.A. Amazal, and A. Abran, “Analogy-based software development effort estimation: A systematic mapping and review,” Information and Software Technology, Vol. 58, 2015, pp. 206–230.

[13]    W. Li and H. Sallie, “Object oriented metrics that predict maintainability,” Journal of Systems and Software, Vol. 23, No. 2, 1993, pp. 111–122.

[14]    A. Kaur, K. Kaur, and K. Pathak, “Software maintainability prediction by data mining of software code metrics,” in International Conference on Data Mining and Intelligent Computing (ICDMIC). IEEE, 2014, pp. 1–6.

[15]    P. Omidyar, eBay web services. [Online]. http://developer.ebay.com [accessed: 2018-10-26].

[16]    B.R. Reddy and O. Aparajita, “Performance of maintainability index prediction models: A feature selection based study,” Evolving Systems, 2017.

[17]    A. Jain, S. Tarwani, and A. Chug, “An empirical investigation of evolutionary algorithm for software maintainability prediction,” in Students’ Conference on Electrical, Electronics and Computer Science (SCEECS), 2016, pp. 1–6.

[18]    M. Genero, J.A. Olivas, M. Piattini, and F.P. Romero, “A controlled experiment for corroborating the usefulness of class diagram metrics at the early phases of OO developments,” in ADIS, 2001.

[19]    X. Jin, Y. Liu, J. Ren, A. Xu, and R. Bie, “Locality preserving projection on source code metrics for improved software maintainability,” in Australasian Joint Conference on Artificial Intelligence. Springer, 2006, pp. 877–886.

[20]    K. Beck and E. Gamma, JUnit. [Online]. https://junit.org [accessed: 2018-10-26].

[21]    R. Malhotra and A. Chug, “Application of group method of data handling model for software maintainability prediction using object oriented systems,” International Journal of System Assurance Engineering and Management, Vol. 5, No. 2, 2014, pp. 165–173.

[22]    S.J. Sayyad and T. Menzies, The PROMISE Repository of Software Engineering Databases, School of Information Technology and Engineering, University of Ottawa, Canada. [Online]. http://promise.site.uottawa.ca/SERepository Accessed: 2017-09-11.

[23]    Software Engineering – Product Quality – Part 2: External Metrics, Part 3: Internal Metrics, Part 4: Quality in Use Metrics, ISO/IEC Std. TR 9126-2-3-4, 2003, 2004.

[24]    Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – System and software quality models, Geneva, ISO Std. 25010, 2010.

[25]    D. Spinellis, Chidamber and Kemerer Java Metrics (CKJM). [Online]. http://www.spinellis.gr/sw/ckjm/ [accessed: 2017-11-21].

[26]    Jetbrains Homepage. [Online]. http://www.jetbrains.com/idea/ [accessed: 2017-11-19].

[27]    Krakatau Professional Homepage. [Online]. http://www.powersoftware.com/kp/ [accessed: 2017-11-19].

[28]    R. Ferenc, A. Beszedes, M. Tarkiainen, and T. Gyimothy, “Columbus – reverse engineering tool and schema for C++,” in Proceedings of the International Conference on Software Maintenance (ICSM’02), ICSM ’02. Washington, DC, USA: IEEE Computer Society, 2002, pp. 172–181.

[29]    G.A. Di Lucca, A.R. Fasolino, F. Pace, P. Tramontana, and U. De Carlini, “WARE: a tool for the reverse engineering of web applications,” in Proceedings of the Sixth European Conference on Software Maintenance and Reengineering, 2002, pp. 241–250.

[30]    B.R. Reddy, S. Khurana, and A. Ojha, “Software maintainability estimation made easy: A comprehensive tool coin,” in Proceedings of the Sixth International Conference on Computer and Communication Technology 2015. New York: ACM, 2015, pp. 68–72.

[31]    Analyst4j standard tool. [Online]. https://codeswat.com/ [accessed: 2018-01-01].

[32]    S. Almugrin, W. Albattah, and A. Melton, “Using indirect coupling metrics to predict package maintainability and testability,” Journal of System and Software, Vol. 121, No. C, 2016, pp. 298–310.

[33]    R. Malhotra, “A systematic review of machine learning techniques for software fault prediction,” Applied Soft Computing, Vol. 27, 2015, pp. 504–518.

[34]    J. Wen, S. Li, Z. Lin, Y. Hu, and C. Huang, “Systematic literature review of machine learning based software development effort estimation models,” Information and Software Technology, Vol. 54, No. 1, 2012, pp. 41–59.

[35]    S.D. Conte, H.E. Dunsmore, and Y. Shen, Software engineering metrics and models. Benjamin-Cummings Publishing Co., Inc., 1986.

[36]    A.R. Gray and S.G. MacDonell, “A comparison of techniques for developing predictive models of software metrics,” Information and software technology, Vol. 39, No. 6, 1997, pp. 425–437.

[37]    P. Oman and J. Hagemeister, “Construction and testing of polynomials predicting software maintainability,” Journal of Systems and Software, Vol. 24, No. 3, 1994, pp. 251–266.

[38]    S. Haykin, Neural networks: a comprehensive foundation. Prentice Hall PTR, 1994.

[39]    V. Vapnik, The nature of statistical learning theory. New York: Springer-Verlag, 1995.

[40]    A.G. Ivakhnenko, “The group method of data of handling; A rival of the method of stochastic approximation,” Soviet Automatic Control, Vol. 13, 1968, pp. 43–55.

[41]    A.G. Ivakhnenko and Y. Koppa, “Regularization of decision functions in the group method of data handling,” Soviet Automatic Control, Vol. 15, No. 2, 1970, pp. 28–37.

[42]    P. Brereton, B.A. Kitchenham, D. Budgen, M. Turner, and M. Khalil, “Lessons from applying the systematic literature review process within the software engineering domain,” Journal of Systems and Software, Vol. 80, No. 4, 2007, pp. 571–583.

[43]    J. Magne and M. Shepperd, “A systematic review of software development cost estimation studies,” IEEE Transactions on Software Engineering, Vol. 33, No. 1, 2007, pp. 33–53.

[44]    S. Muthanna, K. Kontogiannis, K. Ponnambalam, and B. Stacey, “A maintainability model for industrial software systems using design level metrics,” in Proceedings Seventh Working Conference on Reverse Engineering. IEEE, 2000, pp. 248–256.

[45]    M. Thwin and T. Quah, “Application of neural networks for estimating software maintainability using object-oriented metrics,” in International Conference on Software Engineering and Knowledge Engineering, 2003, pp. 69–73.

[46]    M. Genero, M. Piattini, E. Manso, and G. Cantone, “Building UML class diagram maintainability prediction models based on early metrics,” in 5th International Workshop on Enterprise Networking and Computing in Healthcare Industry. IEEE, 2003, pp. 263–275.

[47]    M. Kiewkanya, N. Jindasawat, and P. Muenchaisri, “A methodology for constructing maintainability model of object-oriented design,” in Fourth International Conference on Quality Software. IEEE, 2004, pp. 206–213.

[48]    G.A. Di Lucca, A.R. Fasolino, P. Tramontana, and C.A. Visaggio, “Towards the definition of a maintainability model for web applications,” in Eighth European Conference on Software Maintenance and Reengineering. IEEE, 2004, pp. 279–287.

[49]    C. Van Koten and A. Gray, “An application of Bayesian network for predicting object-oriented software maintainability,” Information and Software Technology, Vol. 48, No. 1, 2006, pp. 59–67.

[50]    J.H. Hayes and L. Zhao, “Maintainability prediction: A regression analysis of measures of evolving systems,” in 21st International Conference on Software Maintenance (ICSM’05). IEEE, 2005, pp. 601–604.

[51]    S.C. Misra, “Modeling design/coding factors that drive maintainability of software systems,” Software Quality Journal, Vol. 13, No. 3, 2005, pp. 297–320.

[52]    Y. Zhou and H. Leung, “Predicting object-oriented software maintainability using multivariate adaptive regression splines,” Journal of Systems and Software, Vol. 80, No. 8, 2007, pp. 1349–1361.

[53]    S.S. Dahiya, J.K. Chhabra, and S. Kumar, “Use of genetic algorithm for software maintainability metrics’ conditioning,” in 15th International Conference on Advanced Computing and Communications. IEEE, 2007, pp. 87–92.

[54]    M. Genero, E. Manso, A. Visaggio, G. Canfora, and M. Piattini, “Building measure-based prediction models for UML class diagram maintainability,” Empirical Software Engineering, Vol. 12, No. 5, 2007, pp. 517–549.

[55]    K. Shibata, K. Rinsaka, T. Dohi, and H. Okamura, “Quantifying software maintainability based on a fault-detection/correction model,” in 13th Pacific Rim International Symposium on Dependable Computing. IEEE, 2007, pp. 35–42.

[56]    K. Aggarwal, Y. Singh, A. Kaur, and R. Malhotra, “Application of artificial neural network for predicting maintainability using object-oriented metrics,” World Academy of Science, Engineering and Technology, International Journal of Computer, Electrical, Automation, Control and Information Engineering, Vol. 2, No. 10, 2008, pp. 3552–3556.

[57]    Y. Tian, C. Chen, and C. Zhang, “AODE for source code metrics for improved software maintainability,” in Fourth International Conference on Semantics, Knowledge and Grid. IEEE, 2008, pp. 330–335.

[58]    Y. Zhou and B. Xu, “Predicting the maintainability of open source software using design metrics,” Wuhan University Journal of Natural Sciences, Vol. 13, No. 1, 2008, pp. 14–20.

[59]    H. Yu, G. Peng, and W. Liu, “An application of case based reasoning to predict structure maintainability,” in International Conference on Computational Intelligence and Software Engineering. IEEE, 2009, pp. 1–5.

[60]    A. Sharma, P. Grover, and R. Kumar, “Predicting maintainability of component-based systems by using fuzzy logic,” in International Conference on Contemporary Computing. Springer, 2009, pp. 581–591.

[61]    L. Wang, X. Hu, Z. Ning, and W. Ke, “Predicting object-oriented software maintainability using projection pursuit regression,” in First International Conference on Information Science and Engineering. IEEE, 2009, pp. 3827–3830.

[62]    H. Mittal and P. Bhatia, “Software maintainability assessment based on fuzzy logic technique,” ACM SIGSOFT Software Engineering Notes, Vol. 34, No. 3, 2009, pp. 1–5.

[63]    M.O. Elish and K.O. Elish, “Application of treenet in predicting object-oriented software maintainability: A comparative study,” in 13th European Conference on Software Maintenance and Reengineering. IEEE, 2009, pp. 69–78.

[64]    S. Rizvi and R.A. Khan, “Maintainability estimation model for object-oriented software in design phase (MEMOOD),” arXiv preprint arXiv:1004.4447, 2010.

[65]    A. Kaur, K. Kaur, and R. Malhotra, “Soft computing approaches for prediction of software maintenance effort,” International Journal of Computer Applications, Vol. 1, No. 16, 2010, pp. 69–75.

[66]    C. Jin and J.A. Liu, “Applications of support vector mathine and unsupervised learning for predicting maintainability using object-oriented metrics,” in Second International Conference on Multimedia and Information Technology, Vol. 1. IEEE, 2010, pp. 24–27.

[67]    L. Cai, Z. Liu, J. Zhang, W. Tong, and G. Yang, “Evaluating software maintainability using fuzzy entropy theory,” in 9th International Conference on Computer and Information Science. IEEE, 2010, pp. 737–742.

[68]    S.O. Olatunji, Z. Rasheed, K. Sattar, A. Al-Mana, M. Alshayeb, and E. El-Sebakhy, “Extreme learning machine as maintainability prediction model for object-oriented software systems,” Journal of Computing, Vol. 2, No. 8, 2010, pp. 49–56.

[69]    P. Dhankhar, H. Mittal, and A. Mittal, “Maintainability prediction for object oriented software,” International Journal of Advances in Engineering Sciences, Vol. 1, No. 1, 2011, pp. 8–11.

[70]    S.K. Dubey and A. Rana, “A fuzzy approach for evaluation of maintainability of object oriented software system,” International Journal of Computer Applications, Vol. 49, No. 21, 2012.

[71]    S.K. Dubey, A. Rana, and Y. Dash, “Maintainability prediction of object-oriented software system by multilayer perceptron model,” ACM SIGSOFT Software Engineering Notes, Vol. 37, No. 5, 2012, pp. 1–4.

[72]    N. Tagoug, “Maintainability assessment in object-oriented system design,” in International Conference on Information Technology and e-Services. IEEE, 2012, pp. 1–5.

[73]    S. Sharawat, “Software maintainability prediction using neural networks,” environment, Vol. 3, No. 5, 2012, pp. 750–755.

[74]    H.A. Al-Jamimi and M. Ahmed, “Prediction of software maintainability using fuzzy logic,” in International Conference on Computer Science and Automation Engineering. IEEE, 2012, pp. 702–705.

[75]    R. Malhotra and A. Chug, “Software maintainability prediction using machine learning algorithms,” Software Engineering: An International Journal, Vol. 2, No. 2, 2012.

[76]    T. Bakota, P. Hegedűs, G. Ladányi, P. Körtvélyesi, R. Ferenc, and T. Gyimóthy, “A cost model based on software maintainability,” in 28th International Conference on Software Maintenance (ICSM). IEEE, 2012, pp. 316–325.

[77]    Y. Dash, S.K. Dubey, and A. Rana, “Maintainability prediction of object oriented software system by using artificial neural network approach,” International Journal of Soft Computing and Engineering (IJSCE), Vol. 2, No. 2, 2012, pp. 420–423.

[78]    P. Hegedűs, G. Ladányi, I. Siket, and R. Ferenc, “Towards building method level maintainability models based on expert evaluations,” in Computer Applications for Software Engineering, Disaster Recovery, and Business Continuity. Springer, 2012, pp. 146–154.

[79]    D. Chandra, “Support vector approach by using radial kernel function for prediction of software maintenance effort on the basis of multivariate approach,” International Journal of Computer Applications, Vol. 51, No. 4, 2012.

[80]    H. Aljamaan, M.O. Elish, and I. Ahmad, “An ensemble of computational intelligence models for software maintenance effort prediction,” in International Work-Conference on Artificial Neural Networks. Springer, 2013, pp. 592–603.

[81]    P. Hegedűs, T. Bakota, G. Ladányi, C. Faragó, and R. Ferenc, “A drill-down approach for measuring maintainability at source code element level,” Electronic Communications of the EASST, Vol. 60, 2013.

[82]    X.L. Hao, X.D. Zhu, and L. Liu, “Research on software maintainability evaluation based on fuzzy integral,” in International Conference on Quality, Reliability, Risk, Maintenance, and Safety Engineering (QR2MSE). IEEE, 2013, pp. 1279–1282.

[83]    F. Ye, X. Zhu, and Y. Wang, “A new software maintainability evaluation model based on multiple classifiers combination,” in International Conference on Quality, Reliability, Risk, Maintenance, and Safety Engineering (QR2MSE). IEEE, 2013, pp. 1588–1591.

[84]    M.A. Ahmed and H.A. Al-Jamimi, “Machine learning approaches for predicting software maintainability: A fuzzy-based transparent model,” IET software, Vol. 7, No. 6, 2013, pp. 317–326.

[85]    S.O. Olatunji and A. Ajasin, “Sensitivity-based linear learning method and extreme learning machines compared for software maintainability prediction of object-oriented software systems,” ICTACT Journal On Soft Computing, Vol. 3, No. 03, 2013.

[86]    A. Mehra and S.K. Dubey, “Maintainability evaluation of object-oriented software system using clustering techniques,” Internationa Journal of Computers and Technology, Vol. 5, No. 02, 2013, pp. 136–143.

[87]    J. Al Dallal, “Object-oriented class maintainability prediction using internal quality attributes,” Information and Software Technology, Vol. 55, No. 11, 2013, pp. 2028–2048.

[88]    A. Kaur, K. Kaur, and K. Pathak, “A proposed new model for maintainability index of open source software,” in Proceedings of 3rd International Conference on Reliability, Infocom Technologies and Optimization. IEEE, 2014, pp. 1–6.

[89]    A. Pratap, R. Chaudhary, and K. Yadav, “Estimation of software maintainability using fuzzy logic technique,” in International Conference on Issues and Challenges in Intelligent Computing Techniques (ICICT). IEEE, 2014, pp. 486–492.

[90]    L. Geeta, A. Kavita, and B. Rizwan, “Maintainability measurement model for object oriented design,” International Journal of Advanced Research in Computer Science and Software Engineering, Vol. 4, No. 11, 2014, pp. 945–956.

[91]    R. Malhotra and A. Chug, “A metric suite for predicting software maintainability in data intensive applications,” in Transactions on Engineering Technologies. Springer Netherlands, 2014, pp. 161–175.

[92]    S. Misra and F. Egoeze, “Framework for maintainability measurement of web application for efficient knowledge-sharing on campus intranet,” in Computational Science and Its Applications – ICCSA 2014. Cham: Springer International Publishing, 2014, pp. 649–662.

[93]    M.O. Elish, H. Aljamaan, and I. Ahmad, “Three empirical studies on predicting software maintainability using ensemble methods,” Soft Computing, Vol. 19, No. 9, 2015, pp. 2511–2524.

[94]    L. Kumar and S.K. Rath, “Neuro-genetic approach for predicting maintainability using Chidamber and Kemerer software metrics suite,” in Recent Advances in Information and Communication Technology 2015. Cham: Springer International Publishing, 2015, pp. 31–40.

[95]    S.O. Olatunji and A. Selamat, “Type-2 fuzzy logic based prediction model of object oriented software maintainability,” in Intelligent Software Methodologies, Tools and Techniques. Cham: Springer International Publishing, 2015, pp. 329–342.

[96]    L. Kumar, D.K. Naik, and S.K. Rath, “Validating the effectiveness of object-oriented metrics for predicting maintainability,” Procedia Computer Science, Vol. 57, 2015, pp. 798–806.

[97]    L. Kumar and S.K. Rath, “Hybrid functional link artificial neural network approach for predicting maintainability of object-oriented software,” Journal of Systems and Software, Vol. 121, No. C, 2016, pp. 170–190.

[98]    A. Chug and R. Malhotra, “Benchmarking framework for maintainability prediction of open source software using object oriented metrics,” International Journal of Innovative Computing, Information and Control, Vol. 12, No. 2, 2016, pp. 615–634.

[99]    L. Kumar, K. Mukesh, and K.R. Santanu, “Maintainability prediction of web service using support vector machine with various kernel methods,” International Journal of System Assurance Engineering and Management, Vol. 8, No. 2, 2017, pp. 205–6222.

[100]    S. Tarwani and A. Chug, “Predicting maintainability of open source software using gene expression programming and bad smells,” in 5th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), 2016, pp. 452–459.

[101]    S. Tarwani and A. Chug, “Sequencing of refactoring techniques by greedy algorithm for maximizing maintainability,” in International Conference on Advances in Computing, Communications and Informatics (ICACCI), 2016, pp. 1397–1403.

[102]    L. Kumar, S.K. Rath, and A. Sureka, “Empirical analysis on effectiveness of source code metrics for predicting change-proneness,” in Proceedings of the 10th Innovations in Software Engineering Conference, ISEC ’17. New York, NY, USA: ACM, 2017, pp. 4–14.

[103]    G. Kanika and C. Anuradha, “Evaluation of instance-based feature subset selection algorithm for maintainability prediction,” in International Conference on Advances in Computing, Communications and Informatics (ICACCI), 2017, pp. 1482–1487.

[104]    K. Shivani and T. Kirti, “Maintainability assessment for software by using a hybrid fuzzy multi-criteria analysis approach,” Management Science Letters, Vol. 7, 2017, pp. 255–274.

[105]    K. Lov and K.R. Santanu, “Software maintainability prediction using hybrid neural network and fuzzy logic approach with parallel computing concept,” International Journal of System Assurance Engineering and Management, Vol. 8, No. S2, 2017, pp. 1487–1502.

[106]    L. Kumar, A. Krishna, and S.K. Rath, “The impact of feature selection on maintainability prediction of service-oriented applications,” Service Oriented Computing and Applications, Vol. 11, No. 2, 2017, pp. 137–161.

[107]    L. Kumar, S.K. Rath, and A. Sureka, “Using source code metrics and multivariate adaptive regression splines to predict maintainability of service oriented software,” in 18th International Symposium on High Assurance Systems Engineering (HASE), 2017, pp. 88–95.

[108]    L. Kumar, S.K. Rath, and A. Sureka, “Using source code metrics to predict change-prone web services: A case-study on ebay services,” in IEEE Workshop on Machine Learning Techniques for Software Quality Evaluation – MaLTeSQuE. IEEE, 2017, pp. 1–7.

[109]    R. Malhotra and R. Jangra, “Prediction and assessment of change prone classes using statistical and machine learning techniques,” Journal of Information Processing Systems, Vol. 13, No. 4, 2017, pp. 778–804.

[110]    G. Szőke, G. Antal, C. Nagy, R. Ferenc, and T. Gyimóthy, “Empirical study on refactoring large-scale industrial systems and its effects on maintainability,” Journal of Systems and Software, Vol. 129, 2017, pp. 107–126.

[111]    Y. Gokul and M. Gopal, “An authoritative method using fuzzy logic to evaluate maintainability index and utilizability of software,” Advances in Modelling and Analysis B, Vol. 60, No. 3, 2017, pp. 566–580.

[112]    P. Hegedűs, I. Kádár, R. Ferenc, and T. Gyimóthy, “Empirical evaluation of software maintainability based on a manually validated refactoring dataset,” Information and Software Technology, Vol. 95, No. 1, 2018, pp. 313–327.

[113]    L. Kumar and S. Ashish, “A comparative study of different source code metrics and machine learning algorithms for predicting change proneness of object oriented systems,” arXiv preprint arXiv:1712.07944, 2018.

[114]    G. Scanniello, C. Gravino, M. Genero, J.A. Cruz-Lemus, and G. Tortora, “On the impact of UML analysis models on source-code comprehensibility and modifiability,” ACM Trans. Softw. Eng. Methodol., Vol. 23, No. 2, 2014, pp. 13:1–13:26.

[115]    A.M. Fernández-Sáez, M.R.V. Chaudron, M. Genero, and I. Ramos, “Are forward designed or reverse-engineered UML diagrams more helpful for code maintenance?: A controlled experiment,” in Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering. New York, NY, USA: ACM, 2013, pp. 60–71.

[116]    G. Scanniello, C. Gravino, G. Tortora, M. Genero, M. Risi, J.A. Cruz-Lemus, and G. Dodero, “Studying the effect of UML-based models on source-code comprehensibility: Results from a long-term investigation,” in Proceedings of the 16th International Conference on Product-Focused Software Process Improvement, Vol. 9459, New York, 2015, pp. 311–327.

[117]    A.M. Fernández-Sáez, M. Genero, D. Caivano, and M.R. Chaudron, “Does the level of detail of UML diagrams affect the maintainability of source code?: A family of experiments,” Empirical Software Engineering, Vol. 21, No. 1, 2016, pp. 212–259.

[118]    G. Scanniello, C. Gravino, M. Genero, J.A. Cruz-Lemus, G. Tortora, M. Risi, and G. Dodero, “Do software models based on the UML aid in source-code comprehensibility? Aggregating evidence from 12 controlled experiments,” Empirical Software Engineering, Vol. 23, No. 5, 2018, pp. 2695–2733.