Comparison of a textual versus a graphical notation for the maintainability of MDE domain models: an empirical pilot study

Models are a useful tool to increase the developer’s productivity and satisfaction when performing maintenance tasks. However, in order to maximise these advantages, the right selection of notations must be made. Unfortunately, the software engineering field lacks a body of empirical evidence that supports such selection. A suboptimal decision in this regard may have negative consequences over the maintenance process. The aim of the study was to compare a textual and a graphical notation with respect to the efficiency, effectiveness and satisfaction of junior software developers while performing analysability and modifiability tasks on two different applications. We have carried out a quasi-experiment with 86 third-year students of the Computer Engineering degree at the University of Alicante. Subjects were randomly classified into two groups, and each group performed 20 maintenance tasks with a textual and a graphical notation. We measured and compared the efficiency, effectiveness and satisfaction of subjects assigned to each treatment. The analysed data show that only the actual analysability coverage (AACov) and the actual modifiability efficiency (AMEffc) are affected by the type of notation used, regardless of the application. In both cases, subjects using the textual notation performed significantly better, although the effect size was low to moderate (AACov η2=0.106\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta ^2=0.106$$\end{document}, AMEffc η2=0.187\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta ^2=0.187$$\end{document}). This is a pilot study, and it has been conducted to enhance the design of future studies in this area. Thus, it should be replicated to extend our conclusions to other subject profiles, different textual and graphical notations, and application types.