| 2021 | |
| [1] | , "A Framework for the Regression Testing of Model-to-Model Transformations", In e-Informatica Software Engineering Journal, vol. 15, no. 1, pp. 65–84, 2021.
DOI: 10.37190/e-Inf210104. Download article (PDF)Get article BibTeX file |
Authors
Issam Al-Azzoni, Saqib Iqbal
Abstract
Background: Model transformations play a key role in Model-Driven Engineering (MDE). Testing model transformation is an important activity to ensure the quality and correctness of the generated models. However, during the evolution and maintenance of these model transformation programs, frequently testing them by running a large number of test cases can be costly. Regression test selection is a form of testing, which selects tests from an existing test suite to test a modified program.
Aim: The aim of the paper is to present a test selection approach for the regression testing of model transformations. The selected test case suite should be smaller in size than the full test suite, thereby reducing the testing overhead, while at the same time the fault detection capability of the full test suite should not be compromised.
Method: approach is based on the use of a traceability mapping of test cases with their corresponding rules to select the affected test items. The approach is complemented with a tool that automates the proposed process.
Results: Our experiments show that the proposed approach succeeds in reducing the size of the selected test case suite, and hence its execution time, while not compromising the fault detection capability of the full test suite.
Conclusion: The experimental results confirm that our regression test selection approach is cost-effective compared to a retest strategy.
Keywords
Model Transformation, Regression Testing, MDE
References
1. A.R. da Silva, “Model-driven engineering: A survey supported by the unified conceptual mode,” Computer Languages, Systems & Structures , Vol. 43, 2015, pp. 139–155.
2. P. Mohagheghi and V. Dehlen, “Where is the proof? – A review of experiences from applying mde in industry,” in Proceedings of the European Conference on Model Driven Architecture – Foundations and Applications , 2008, pp. 432–443.
3. B. Alkhazi, C. Abid, M. Kessentini, D. Leroy, and M. Wimmer, “Multi-criteria test cases selection for model transformations,” Automated Software Engineering , Vol. 27, No. 1, 2020, pp. 91–118.
4. J. Troya, S. Segura, J.A. Parejo, and A. Ruiz-Cortés, “Spectrum-based fault localization in model transformations,” ACM Transactions on Software Engineering and Methodology , Vol. 27, No. 3, 2018.
5. D. Calegari and N. Szasz, “Verification of model transformations: A survey of the state-of-the-art,” Electronic Notes in Theoretical Computer Science , Vol. 292, 2013, pp. 5–25.
6. G.M.K. Selim, J.R. Cordy, and J. Dingel, “Model transformation testing: The state of the art,” in Proceedings of the First Workshop on the Analysis of Model Transformations , 2012, pp. 21–26.
7. G. Rothermel and M.J. Harrold, “Analyzing regression test selection techniques,” IEEE Transactions on Software Engineering , Vol. 22, No. 8, 1996, pp. 529–551.
8. S. Yoo and M. Harman, “Regression testing minimization, selection and prioritization: A survey,” Software Testing, Verification and Reliability , Vol. 22, No. 2, 2012, pp. 67–120.
9. J. Shelburg, M. Kessentini, and D.R. Tauritz, “Regression testing for model transformations: A multi-objective approach,” in Proceedings of the International Symposium on Search Based Software Engineering , 2013, pp. 209–223.
10. E. Seidewitz, “What models mean,” IEEE Software , Vol. 20, No. 5, 2003, pp. 26–32.
11. Eclipse Modeling Framework (EMF) , Eclipse Foundation. [Online]. https://www.eclipse.org/modeling/emf/ [Accessed December 2020].
12. K. Czarnecki and S. Helsen, “Feature-based survey of model transformation approaches,” IBM Systems Journal , Vol. 45, No. 3, 2006, pp. 621–645.
13. Eclipse Epsilon , Eclipse Foundation. [Online]. https://www.eclipse.org/epsilon/ [Accessed December 2020].
14. D. Kolovos, L. Rose, A. García-Domínguez, and R. Paige, The epsilon book . Eclipse Foundation., 2018. [Online]. https://www.eclipse.org/epsilon/doc/book/
15. ATL – A model transformation technology , Eclipse Foundation. [Online]. https://www.eclipse.org/atl/ [Accessed December 2020].
16. The Epsilon Unit Testing Framework , Eclipse Foundation. [Online]. https://www.eclipse.org/epsilon/doc/eunit/ [Accessed December 2020].
17. A. García-Domínguez, D.S. Kolovos, L.M. Rose, R.F. Paige, and I. Medina-Bulo, “EUnit: A unit testing framework for model management tasks,” in Proceedings of the International Conference on Model Driven Engineering Languages and Systems , 2011, pp. 395–409.
18. EMF Compare , Eclipse Foundation. [Online]. https://www.eclipse.org/emf/compare/ [Accessed December 2020].
19. Epsilon – Examples . [Online]. https://www.eclipse.org/epsilon/examples/ [Accessed December 2020].
20. V. Aranega, J. Mottu, A. Etien, T. Degueule, B. Baudry, and J. Dekeyser, “Towards an automation of the mutation analysis dedicated to model transformation,” Software Testing, Verification and Reliability , Vol. 25, No. 5-7, 2015, pp. 653–683.
21. F. Fleurey, J. Steel, and B. Baudry, “Validation in model-driven engineering: Testing model transformations,” in Proceedings of the First International Workshop on Model, Design and Validation , 2004, pp. 29–40.
22. S. Sen, B. Baudry, and J. Mottu, “On combining multi-formalism knowledge to select models for model transformation testing,” in Proceedings of the First International Conference on Software Testing, Verification, and Validation , 2008, pp. 328–337.
23. Object Constraint Language , Object Management Group, 2014. [Online]. http://www.omg.org/spec/OCL/2.4 [Accessed December 2020].
24. I. Al-Azzoni, “ATL transformation of queueing networks to queueing Petri nets,” in Proceedings of the International Conference on Model-Driven Engineering and Software Development (MODELSWARD) , 2017, pp. 261–268.
25. ATL Transformation Example: BibTeXML to DocBook , 2005. [Online]. https://www.eclipse.org/atl/atlTransformations/BibTeXML2DocBook/ExampleBibTeXML2DocBook[v00.01].pdf [Accessed December 2020].
26. F. Jouault, J. Bézivin, C. Consel, I. Kurtev, and F. Latry, “Building DSLs with AMMA/ATL, a case study on SPL and CPL telephony languages,” in Proceedings of the ECOOP Workshop on Domain-Specific Program Development , 2006.
27. ATL Transformations Zoo , Eclipse Foundation. [Online]. https://www.eclipse.org/atl/atlTransformations/ [Accessed December 2020].
28. F. Fleurey, B. Baudry, P. Muller, and Y.L. Traon, “Qualifying input test data for model transformations,” Software and System Modeling , Vol. 8, No. 2, 2009, pp. 185–203.
29. J. Troya, A. Bergmayr, L. Burgueño, and M. Wimmer, “Towards systematic mutations for and with ATL model transformations,” in Proceedings of International Conference on Software Testing, Verification and Validation Workshops , 2015, pp. 1–10.
30. J. Troya, Implementation of the Spectrum-Based Fault Localization in Model Transformations , 2018. [Online]. https://github.com/javitroya/SBFL_MT [Accessed December 2020].
31. D.C. Montgomery and G.C. Runger, Applied Statistics and Probability for Engineers, 6th Edition . John Wiley and Sons, 2013.
32. C. Wohlin, P. Runeson, M. Hst, M.C. Ohlsson, B. Regnell, and A. Wessln, Experimentation in Software Engineering . Springer Publishing Company, 2012.
33. E. Guerra, J. Sánchez Cuadrado, and J. de Lara, “Towards effective mutation testing for ATL,” in Proceedings of the International Conference on Model Driven Engineering Languages and Systems , 2019, pp. 78–88.
34. D. Honfi, G. Molnár, Z. Micskei, and I. Majzik, “Model-based regression testing of autonomous robots,” in Proceedings of the International System Design Language Forum , 2017, pp. 119–135.
35. Eclipse , Eclipse Foundation. [Online]. https://www.eclipse.org/ [Accessed December 2020].
36. L.A. Rahim and J. Whittle, “A survey of approaches for verifying model transformations,” Software and Systems Modeling , Vol. 14, No. 2, 2015, pp. 1003–1028.
37. C.A. González and J. Cabot, “ATLTest: A white-box test generation approach for ATL transformations,” in Proceedings of the International Conference on Model Driven Engineering Languages and Systems , 2012, pp. 449–464.
38. J. Cabot, R. Clarisó, E. Guerra, and J. de Lara, “Verification and validation of declarative model-to-model transformations through invariants,” Journal of Systems and Software , Vol. 83, No. 2, 2010, pp. 283–302.
39. K. Ehrig, J.M. Küster, and G. Taentzer, “Generating instance models from meta models,” Software and Systems Modeling , Vol. 8, No. 4, 2009, pp. 479–500.
40. J. Troya and A. Vallecillo, “Towards a rewriting logic semantics for ATL,” in Proceedings of the International Conference on Theory and Practice of Model Transformations , 2010, pp. 230–244.
41. P. Zech, P. Kalb, M. Felderer, C. Atkinson, and R. Breu, “Model-based regression testing by OCL,” International Journal on Software Tools for Technology Transfer , Vol. 19, No. 1, 2017, pp. 115–131.
42. L. Naslavsky, H. Ziv, and D.J. Richardson, “A model-based regression test selection technique,” in Proceedings of the IEEE International Conference on Software Maintenance , 2009, pp. 515–518.
43. M. Al-Refai, S. Ghosh, and W. Cazzola, “Supporting inheritance hierarchy changes in model-based regression test selection,” Software and Systems Modeling , Vol. 18, No. 2, 2019, pp. 937–958.
44. J. Troya, S. Segura, and A. Ruiz-Cortés, “Automated inference of likely metamorphic relations for model transformations,” The Journal of Systems and Software , Vol. 136, 2018, pp. 188–208.
45. F. Jouault, “Loosely coupled traceability for ATL,” in Proceedings of the European Conference on Model Driven Architecture Workshop on Traceability , 2005, pp. 29–37.
46. L. Naslavsky, H. Ziv, and D. Richardson, “MbSRT2: Model-based selective regression testing with traceability,” in Proceedings of the International Conference on Software Testing, Verification and Validation , 2010, pp. 89–98.
47. S. Winkler and J. von Pilgrim, “A survey of traceability in requirements engineering and model-driven development,” Software and Systems Modeling , Vol. 9, No. 4, 2010, pp. 529–565.
48. B. Legeard, “Model-based testing: Next generation functional software testing,” in Practical Software Testing: Tool Automation and Human Factors , M. Harman, H. Muccini, W. Schulte, and T. Xie, Eds. Schloss Dagstuhl – Leibniz-Zentrum für Informatik, Germany, 2010.
49. A.C. Dias Neto, R. Subramanyan, M. Vieira, and G.H. Travassos, “A survey on model-based testing approaches: A systematic review,” in Proceedings of the International Workshop on Empirical Assessment of Software Engineering Languages and Technologies , 2007, pp. 31–36.
50. M. Utting and B. Legeard, Practical Model-Based Testing – A Tools Approach . Morgan Kaufmann, 2007.
51. L.C. Briand, Y. Labiche, and S. He, “Automating regression test selection based on UML designs,” Information and Software Technology , Vol. 51, No. 1, 2009, pp. 16–30.
52. Q. Farooq, M.Z.Z. Iqbal, Z.I. Malik, and A. Nadeem, “An approach for selective state machine based regression testing,” in Proceedings of the International Workshop on Advances in Model-based Testing , 2007, pp. 44–52.
53. Y. Chen, R.L. Probert, and D.P. Sims, “Specification-based regression test selection with risk analysis,” in Proceedings of the Conference of the Centre for Advanced Studies on Collaborative Research , 2002.
54. L.C. Briand, Y. Labiche, and G. Soccar, “Automating impact analysis and regression test selection based on UML designs,” in Proceedings of the International Conference on Software Maintenance , 2002, pp. 252–261.
55. Y. Chen, R.L. Probert, and H. Ural, “Regression test suite reduction using extended dependence analysis,” in Proceedings of the International Workshop on Software Quality Assurance , 2007, pp. 62–69.
56. B. Korel, L.H. Tahat, and B. Vaysburg, “Model based regression test reduction using dependence analysis,” in Proceedings of the International Conference on Software Maintenance , 2002.
57. B. Vaysburg, L.H. Tahat, and B. Korel, “Dependence analysis in reduction of requirement based test suites,” in Proceedings of the International Symposium on Software Testing and Analysis , 2002, pp. 107–111.
58. N. Almasri, L. Tahat, and B. Korel, “Toward automatically quantifying the impact of a change in systems,” Software Quality Journal , Vol. 25, No. 3, 2017, pp. 601–640.
59. M. Felderer, B. Agreiter, and R. Breu, “Evolution of security requirements tests for service-centric systems,” in Engineering Secure Software and Systems , Ú. Erlingsson, R. Wieringa, and N. Zannone, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, pp. 181–194.
60. M. Felderer, B. Agreiter, and R. Breu, “Managing evolution of service centric systems by test models,” in Proceedings of the Tenth IASTED International Conference on Software Engineering , 2011, pp. 72–80.