The results achieved through evaluation of the proposed method justified aim and objectives of this research. However, the limitations of this research need to be made explicit thereby opening avenues for future work. The limitations are summarised as follows:
• Limitations to use structured scenario sentences. The current version of iSketch mainly supports the use of structured scenario sentences for specifying in detail the requirements of the system under design. It requires the user to using a sub-ject-object-predicate structure when writing scenario sentences to get good mod-el transformation results.
• Limitation of the extraction and translation rules for model transformation. The rules that the model transformation process of iSketch depends on, may not con-tain all necessary rules for the generation of AOMs, which are required for the construction of sequence diagram and class diagram initial software models. De-spite this limitation, it was noticed that the rules performed better when struc-tured scenario sentences were used.
• Limitations in collaborative sketching. iSketch currently does not support col-laborative modelling of requirements as some of the existing related sketching tools (FlexiSketch, SUMLOW and Knight) do. This is mainly because; the aim of this research is to demonstrate the feasibility of the proposed methods in a single-user device. However, collaborative modelling of requirements will allow participants in either a co-located or geographically distributed elicitation meet-ings to have concurrent editing access to a synchronized Sketch Canvas.
• Limitations of auto-generated software models. The initial software models pro-duced by iSketch are mostly first cut sequence and class diagram models, which may need to be refined by requirements analysts.
• Limitation in the notation for modelling requirements. The current version of iSketch limits users to mainly the use of use case diagram for expressing re-quirements. It also limits user to hand-draw the symbols of the use case diagram and then subsequently drag and drop the symbols on to the canvas where the use case diagram is drawn. Hence, end users need to use drag and drop and know the syntax of use case diagram to benefit from the features of iSketch.
Possible future endeavours from this research can be summarised into the follow-ing:
• A support for sketching and recognition of hand-drawn requirements models without the need to drag and drop the symbols needed for the diagram.
• A support for more notations that can be used for expressing requirements mod-els as well as generating initial software modmod-els. This will not restrict users to understanding a particular type of notation before they can benefit from iSketch.
• A refactoring of the rules used in the model transformation process to support any type of NL structure for specifying scenario sentences.
References
[1] J. A. Landay, "SILK: sketching interfaces like krazy." pp. 398-399.
[2] B. Plimmer, and M. Apperley, "Computer-aided sketching to capture preliminary design." pp. 9-12.
[3] R. Jolak, B. Vesin, M. Isaksson, and M. R. V. Chaudron, “Towards a New Generation of Software Design Environments: Supporting the Use of Informal and Formal Notations with OctoUML,” 2016.
[4] C. H. Damm, K. M. Hansen, and M. Thomsen, "Tool support for cooperative object-oriented design: gesture based modelling on an electronic whiteboard."
pp. 518-525.
[5] N. Mangano, A. Baker, M. Dempsey, E. Navarro, and A. van der Hoek,
"Software design sketching with calico." pp. 23-32.
[6] D. Wüest, "Bridging the gap between requirements sketches and semi-formal models."
[7] N. Mangano, A. Baker, and A. Van Der Hoek, "Calico: a prototype sketching tool for modeling in early design." pp. 63-68.
[8] T. Hammond, and R. Davis, "Tahuti: A geometrical sketch recognition system for uml class diagrams." p. 25.
[9] Q. Chen, J. Grundy, and J. Hosking, “SUMLOW: early design-stage sketching of UML diagrams on an E-whiteboard,” Software Practice and Experience, vol.
38, no. 9, pp. 961, 2008.
[10] D. Wuest, N. Seyff, and M. Glinz, “FlexiSketch: a lightweight sketching and metamodeling approach for end-users,” Software & Systems Modeling, pp. 1-29, 2017.
[11] P. Wais, A. Wolin, and C. Alvarado, "Designing a sketch recognition front-end:
user perception of interface elements." pp. 99-106.
[12] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks." pp. 1097-1105.
[13] L. Zhao, "A pattern language for evolving scenarios into multi-perspective models for requirements specification." pp. 21-27.
[14] T. Dyba, B. A. Kitchenham, and M. Jorgensen, “Evidence-based software engineering for practitioners,” IEEE software, vol. 22, no. 1, pp. 58-65, 2005.
[15] G. R. S. Murthy, and R. S. Jadon, “A review of vision based hand gestures recognition,” International Journal of Information Technology and Knowledge Management, vol. 2, no. 2, pp. 405-410, 2009.
[16] E. Sanchez-Nielsen, L. Anton-Canalis, and M. Hernández-Tejera, “Hand gesture recognition for human-machine interaction,” 2004.
[17] W. Rucklidge, Efficient visual recognition using the Hausdorff distance:
Springer-Verlag, 1996.
[18] A. Attwenger, Advantages and Drawbacks of Gesture-based Interaction: Grin Publishing, 2017.
[19] M. Schmidt, and G. Weber, "Recognition of multi-touch drawn sketches." pp.
479-490.
[20] H. Ogawa, and K. Taniguchi, “Thinning and stroke segmentation for handwritten Chinese character recognition,” Pattern recognition, vol. 15, no. 4, pp. 299-308, 1982.
[21] C. Leung, Y. S. Cheung, and Y. L. Wong, “A knowledge-based stroke-matching method for Chinese character recognition,” IEEE transactions on systems, man, and cybernetics, vol. 17, no. 6, pp. 993-1003, 1987.
[22] H. Al‐Tabtabai, and A. P. Alex, “Using genetic algorithms to solve optimization problems in construction,” Engineering Construction and Architectural Management, vol. 6, no. 2, pp. 121-132, 1999.
[23] C.-K. Lin, K.-S. Chou, B.-S. Jeng, C.-H. Shih, T.-K. Su, and T. I. Fan, "A knowledge model based on-line recognition system." pp. 157-160.
[24] M. Bresler, D. Prusa, and V. Hlavac, “Online recognition of sketched arrow-connected diagrams,” International Journal on Document Analysis and Recognition (IJDAR), vol. 19, no. 3, pp. 253-267, 2016.
[25] G. Costagliola, M. De Rosa, and V. Fuccella, “Local context-based recognition of sketched diagrams,” Journal of Visual Languages & Computing, vol. 25, no.
6, pp. 955-962, 2014.
[26] V. Deufemia, M. Risi, and G. Tortora, "Hand-drawn diagram recognition with hierarchical parsing: an experimental evaluation," Information systems:
crossroads for organization, management, accounting and engineering, pp. 217-225: Springer, 2012.
[27] T. Hammond, and B. Paulson, “Recognizing sketched multistroke primitives,”
ACM Transactions on Interactive Intelligent Systems (TiiS), vol. 1, no. 1, pp. 4, 2011.
[28] G. Casella, V. Deufemia, V. Mascardi, G. Costagliola, and M. Martelli, “An agent-based framework for sketched symbol interpretation,” Journal of Visual Languages & Computing, vol. 19, no. 2, pp. 225-257, 2008.
[29] T. Hammond, and R. Davis, “LADDER, a sketching language for user interface developers,” Computers & Graphics, vol. 29, no. 4, pp. 518-532, 2005.
[30] F.-H. J. Ernest, “Jess, The Java Expert System Shell,” Distributed Computing Systems, Sandia National Laboratories, 2000.
[31] G. Costagliola, V. Deufemia, and M. Risi, "Sketch grammars: A formalism for describing and recognizing diagrammatic sketch languages." pp. 1226-1230.
[32] G. Costagliola, V. Deufemia, and G. Polese, “A framework for modeling and implementing visual notations with applications to software engineering,” ACM Transactions on Software Engineering and Methodology (TOSEM), vol. 13, no.
4, pp. 431-487, 2004.
[33] M. J. Fonseca, C. Pimentel, and J. A. Jorge, "CALI: An online scribble recognizer for calligraphic interfaces." pp. 51-58.
[34] F. Brieler, and M. Minas, "Recognition and processing of hand-drawn diagrams using syntactic and semantic analysis." pp. 181-188.
[35] M. Minas, “Concepts and realization of a diagram editor generator based on hypergraph transformation,” Science of Computer Programming, vol. 44, no. 2, pp. 157-180, 2002.
[36] Z. Yuan, and G. Jin, "Sketch Recognition Based Intelligent Whiteboard Teaching System." pp. 867-870.
[37] H. H. Hse, and A. R. Newton, “Recognition and beautification of multi-stroke symbols in digital ink,” Computers & Graphics, vol. 29, no. 4, pp. 533-546, 2005.
[38] V. Vapnik, The nature of statistical learning theory: Springer science &
business media, 2013.
[39] A. Khotanzad, and Y. H. Hong, “Invariant image recognition by Zernike moments,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, no. 5, pp. 489-497, 1990.
[40] T. M. Sezgin, and R. Davis, "HMM-based efficient sketch recognition." pp. 281-283.
[41] Z.-X. Sun, B. Zhang, Q.-H. Qiu, and L.-S. Zhang, "A freehand sketchy graphic input system: SketchGIS." pp. 3232-3237.
[42] E. Lank, J. Thorley, S. Chen, and D. Blostein, "On-line recognition of UML diagrams." pp. 356-360.
[43] T. A. Hammond, and R. Davis, "Recognizing interspersed sketches quickly." pp.
157-166.
[44] F. Brieler, and M. Minas, “A model-based recognition engine for sketched diagrams,” Journal of Visual Languages & Computing, vol. 21, no. 2, pp. 81-97, 2010.
[45] A. Fahmy, W. Abdelhamid, and A. Atiya, "Interactive Sketch Recognition Framework for Geometric Shapes." pp. 323-334.
[46] B. Paulson, and T. Hammond, "Paleosketch: accurate primitive sketch
[50] O. D. Trier, A. K. Jain, and T. Taxt, “Feature extraction methods for character recognition-a survey,” Pattern recognition, vol. 29, no. 4, pp. 641-662, 1996.
[51] K. Aruleba, S. Ewert, I. Sanders, and M. Raborife, "Pre-Processing and Feature Extraction Technique for Hand-drawn Finite Automata Recognition." pp. Page-1.
[52] O. Déniz, G. Bueno, J. s. Salido, and F. De la Torre, “Face recognition using histograms of oriented gradients,” Pattern recognition letters, vol. 32, no. 12, pp. 1598-1603, 2011.
[53] L. Heer, “Sketch-based Metamodel Construction A Literature Review,” 2018.
[54] T. Mens, and P. Van Gorp, “A taxonomy of model transformation,” Electronic notes in theoretical computer science, vol. 152, pp. 125-142, 2006.
[55] D. Wüest, and M. Glinz, "Flexible sketch-based requirements modeling,"
Requirements Engineering: Foundation for Software Quality, pp. 100-105:
Springer, 2011.
[56] D. Wüest, N. Seyff, and M. Glinz, “FlexiSketch: a lightweight sketching and metamodeling approach for end-users,” Software & Systems Modeling, vol. 18, no. 2, pp. 1513-1541, 2019.
[57] D. Wüest, N. Seyff, and M. Glinz, "Semi-automatic generation of metamodels from model sketches." pp. 664-669.
[58] J. J. López-Fernández, A. Garmendia, E. Guerra, and J. de Lara, "Example-based generation of graphical modelling environments." pp. 101-117.
[59] M. Vogel, T. Warnecke, C. Bartelt, and A. Rausch, "Scribbler---drawing models in a creative and collaborative environment: from hand-drawn sketches to domain specific models and vice versa." pp. 93-94.
[60] H. Cho, J. Gray, and E. Syriani, "Creating visual domain-specific modeling languages from end-user demonstration." pp. 22-28.
[61] B. Roth, M. Jahn, and S. Jablonski, “A Method for Directly Deriving a Concise Meta Model from Example Models,” 2013.
[62] D. Wüest, N. Seyff, and M. Glinz, "Flexisketch: A mobile sketching tool for software modeling," Mobile Computing, Applications, and Services, pp. 225-244: Springer, 2012.
[63] D. Wüest, N. Seyff, and M. Glinz, "Sketching and notation creation with FlexiSketch Team: Evaluating a new means for collaborative requirements elicitation." pp. 186-195.
[64] M. R. V. Chaudron, and R. Jolak, "A Vision on a New Generation of Software Design Environments." pp. 11-16.
[65] A. Donaldson, and A. Williamson, "Pen-based input of uml activity diagrams for business process modelling." p. 26.
[66] E. Lank, J. S. Thorley, and S. J.-S. Chen, "An interactive system for recognizing hand drawn UML diagrams." p. 7.
[67] B. Plimmer, and M. Apperley, "Software to sketch interface designs." pp. 73-80.
[68] A. Coyette, and J. Vanderdonckt, "A sketching tool for designing anyuser, anyplatform, anywhere user interfaces." pp. 550-564.
[69] J. A. Landay, and B. A. Myers, "Interactive sketching for the early stages of user interface design." pp. 43-50.
[70] J. Grundy, and J. Hosking, "Supporting generic sketching-based input of diagrams in a domain-specific visual language meta-tool." pp. 282-291.
[71] J. Grundy, J. Hosking, N. Zhu, and N. Liu, "Generating domain-specific visual language editors from high-level tool specifications." pp. 25-36.
[72] C. Alvarado, and R. Davis, "SketchREAD: a multi-domain sketch recognition engine." pp. 23-32.
[73] A. Apte, V. Vo, and T. D. Kimura, "Recognizing multistroke geometric shapes:
an experimental evaluation." pp. 121-128.
[74] A. Fantechi, S. Gnesi, G. Lami, and A. Maccari, “Applications of linguistic techniques for use case analysis,” Requirements engineering, vol. 8, no. 3, pp.
161-170, 2003.
[75] J. Cleland-Huang, R. S. Hanmer, S. Supakkul, and M. Mirakhorli, “The twin peaks of requirements and architecture,” IEEE software, vol. 30, no. 2, pp. 24-29, 2013.
[76] Q. Yu, Y. Yang, F. Liu, Y.-Z. Song, T. Xiang, and T. M. Hospedales, “Sketch-a-net: A deep neural network that beats humans,” International journal of computer vision, vol. 122, no. 3, pp. 411-425, 2017.
[77] M. Eitz, J. Hays, and M. Alexa, “How do humans sketch objects?,” ACM Transactions on graphics (TOG), vol. 31, no. 4, pp. 1-10, 2012.
[78] R. G. Schneider, and T. Tuytelaars, “Sketch classification and classification-driven analysis using fisher vectors,” ACM Transactions on graphics (TOG), vol. 33, no. 6, pp. 1-9, 2014.
[79] X. Zhang, Y. Huang, Q. Zou, Y. Pei, R. Zhang, and S. Wang, “A hybrid convolutional neural network for sketch recognition,” Pattern recognition letters, 2019.
[80] M. Eitz, K. Hildebrand, T. Boubekeur, and M. Alexa, “Sketch-based image retrieval: Benchmark and bag-of-features descriptors,” IEEE transactions on visualization and computer graphics, vol. 17, no. 11, pp. 1624-1636, 2011.
[81] R. Hu, and J. Collomosse, “A performance evaluation of gradient field hog descriptor for sketch based image retrieval,” Computer Vision and Image Understanding, vol. 117, no. 7, pp. 790-806, 2013.
[82] F. Wang, L. Kang, and Y. Li, "Sketch-based 3d shape retrieval using convolutional neural networks." pp. 1875-1883.
[83] D. C. Lee, M. Hebert, and T. Kanade, "Geometric reasoning for single image structure recovery." pp. 2136-2143.
[84] B. Klare, Z. Li, and A. K. Jain, “Matching forensic sketches to mug shot photos,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.
33, no. 3, pp. 639-646, 2010.
[85] S. Ouyang, T. Hospedales, Y.-Z. Song, and X. Li, "Cross-modal face matching:
beyond viewed sketches." pp. 210-225.
[86] D. Rubine, “The automatic recognition of gestures,” 1991.
[87] K. Brunnström, T. Lindeberg, and J.-O. Eklundh, "Active detection and classification of junctions by foveation with a head-eye system guided by the scale-space primal sketch." pp. 701-709.
[88] M. Shpitalni, and H. Lipson, “Classification of sketch strokes and corner detection using conic sections and adaptive clustering,” 1997.
[89] D. Rubine, Specifying gestures by example: ACM, 1991.
[90] H. Zhang, S. Liu, C. Zhang, W. Ren, R. Wang, and X. Cao, "Sketchnet: Sketch classification with web images." pp. 1105-1113.
[91] P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun,
“Overfeat: Integrated recognition, localization and detection using convolutional networks,” arXiv preprint arXiv:1312.6229, 2013.
[92] M. D. Zeiler, and R. Fergus, "Visualizing and understanding convolutional networks." pp. 818-833.
[93] D. H. Hubel, and T. N. Wiesel, “Receptive fields and functional architecture of monkey striate cortex,” The Journal of physiology, vol. 195, no. 1, pp. 215-243, 1968.
[97] A. G. Kleppe, J. Warmer, W. Bast, and M. D. A. Explained, "The model driven architecture: practice and promise," Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 2003.
[98] J. Bézivin, G. Dupé, F. Jouault, G. Pitette, and J. E. Rougui, "First experiments with the ATL model transformation language: Transforming XSLT into XQuery." p. 46.
[99] J. J. López-Fernández, J. S. Cuadrado, E. Guerra, and J. De Lara, “Example-driven meta-model development,” Software & Systems Modeling, vol. 14, no. 4, pp. 1323-1347, 2015.
[100] T. Kühne, “Matters of (meta-) modeling,” Software & Systems Modeling, vol. 5, no. 4, pp. 369-385, 2006.
[101] P. Mohagheghi, M. A. Fernandez, J. A. Martell, M. Fritzsche, and W. Gilani,
"MDE adoption in industry: challenges and success criteria." pp. 54-59.
[102] M. Strembeck, and U. Zdun, “An approach for the systematic development of domainâ€specific languages,” Software: Practice and Experience, vol. 39, no.
15, pp. 1253-1292, 2009.
[103] J. L. C. Izquierdo, J. Cabot, J. López-Fernández, J. S. Cuadrado, E. Guerra, and J. De Lara, "Engaging end-users in the collaborative development of domain-specific modelling languages." pp. 101-110.
[104] M. F. Bertoa, and A. Vallecillo, “Quality attributes for software metamodels,”
Málaga, Spain, 2010.
[105] Y. Wang, L. Zhao, X. Wang, X. Yang, and S. Supakkul, “PLANT: A pattern language for transforming scenarios into requirements models,” International Journal of Human-Computer Studies, vol. 71, no. 11, pp. 1026-1043, 2013.
[106] P. Haumer, K. Pohl, and K. Weidenhaupt, “Requirements elicitation and validation with real world scenes,” IEEE Transactions on software engineering, vol. 24, no. 12, pp. 1036-1054, 1998.
[107] H. Kaindl, “A design process based on a model combining scenarios with goals and functions,” IEEE Transactions on Systems, Man, and Cybernetics-Part A:
Systems and Humans, vol. 30, no. 5, pp. 537-551, 2000.
[108] A. G. Sutcliffe, N. A. M. Maiden, S. Minocha, and D. Manuel, “Supporting scenario-based requirements engineering,” IEEE Transactions on software engineering, vol. 24, no. 12, pp. 1072-1088, 1998.
[109] C. Alexander, A pattern language: towns, buildings, construction: Oxford university press, 1977.
[110] C. Alexander, The timeless way of building: New York: Oxford University Press, 1979.
[111] M. Johnson, The body in the mind: The bodily basis of meaning, imagination, and reason: University of Chicago Press, 1987.
[112] G. Lakoff, Women, fire, and dangerous things: What categories reveal about the mind: University of Chicago press, 2008.
[113] C. Rolland, C. Souveyet, and C. B. Achour, “Guiding goal modeling using scenarios,” IEEE transactions on software engineering, vol. 24, no. 12, pp.
1055-1071, 1998.
[114] N. M. Maiden, "CREWS-SAVRE: Scenarios for acquiring and validating requirements," Domain Modelling for Interactive Systems Design, pp. 39-66:
Springer, 1998.
[115] K. Zachos, N. Maiden, and A. Tosar, “Rich-media scenarios for discovering requirements,” IEEE software, vol. 22, no. 5, pp. 89-97, 2005.
[116] K. Simonyan, and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
[117] T. Bray, “The javascript object notation (json) data interchange format,” 2014.
[118] J. Daniel, “Alloy: a lightweight object modelling notation,” ACM Trans. Softw.
Eng. Methodol., vol. 11, no. 2, pp. 256-290, 2002.
[119] D. Jackson, Software Abstractions: logic, language, and analysis: MIT press, 2012.
[120] P. P.-S. Chen, “English sentence structure and entity-relationship diagrams,”
Information Sciences, vol. 29, no. 2-3, pp. 127-149, 1983.
[121] M.-C. De Marneffe, and C. D. Manning, Stanford typed dependencies manual, Technical report, Stanford University, 2008.
[122] T. Yue, L. C. Briand, and Y. Labiche, “aToucan: an automated framework to derive UML analysis models from use case models,” ACM Transactions on Software Engineering and Methodology (TOSEM), vol. 24, no. 3, pp. 13, 2015.
[123] Z. Carter. "Jison," https://zaa.ch/jison/.
[124] M. Elbendak, P. Vickers, and N. Rossiter, “Parsed use case descriptions as a basis for object-oriented class model generation,” Journal of Systems and Software, vol. 84, no. 7, pp. 1209-1223, 2011.
[125] C. Larman, Applying UML and Patterns: An Introduction to Object Oriented Analysis and Design and Interative Development: Pearson Education India, 2012.
[126] H. Gomaa, "Designing concurrent, distributed, and real-time applications with UML." pp. 737-738.
[127] H. Gomaa, Software modeling and design: UML, use cases, patterns, and software architectures: Cambridge University Press, 2011.