Evolution of visual programming

Visual programming grew in popularity in the early 1990s, with ambitious visions of what could be accomplished by abstracting away from the complexities of writing code. It was perceived that a visual representation would map more closely to an end-user’s mental model of a programming problem. Nevertheless, empirical evidence comparing visual programming languages against textual equivalents was sparse and contradictory [128]. Moreover, attempting to scale visual programming up to incorporate the dense syntax of a general-purpose programming language was an intractable problem, particularly given the constraints on screen space.

In recent years, however, visual programming environments have become more prevalent as

64 CHAPTER 4. END-USER DEVELOPMENT

Table 4.1: Visual programming paradigms, their formal classifications, and illustrative examples

Paradigm Diagrams Icons Blocks

Class Connection-based Geometric-based Connection-based

Relation Linking Iconic sentence Overlapping

Example

educational tools for teaching programming syntax and semantics. For children and young adults learning computer science, visual programming environments such asScratch[129],Alice[130], andApp Inventor[131] are appealing both visually, but also in allowing interesting applications to be built with minimal initial effort. Such environments are currently being used in school curricula to teach programming concepts to children and young adults.

Visual programming environments are more than just teaching tools, however, and have proven useful for domain-specific applications, where problem concepts from a limited domain vocabulary can be mapped to visual components, without becoming intractably large. Domain experts can use visual programming in their work practices without the need to learn textual syntax and semantics. The aforementionedLabVIEW is a prime example [126], as well as Max, a visual language for audio processing applications2_.

Visual programming is abstractly defined as programming in two dimensions, covering a wide variety of concrete implementations. Previous work from the 1990s described taxonomies of visual programming languages [132, 133]. However, in the two decades since these classification systems were published, a wealth of new visual languages have been developed. A more recent classification system was proposed by Costagliola et al. [134], which can be used to classify the visual metaphors used in modern environments. These metaphors tend to fall within one of three categories: diagrammatic programming, iconic programming, and blocks-based programming. Examples of these metaphors with classification details are illustrated in Table 4.1. Each has its own benefits and drawbacks, which will be discussed as follows.

4.1. EUD AND VISUAL PROGRAMMING 65

4.1.3.1 Diagrammatic programming

Diagrammatic programming is one form of a “connection” visual language as defined by Costagliola et al [134]. Diagrammatic languages resemble flowcharts, wherein programming constructs are represented as boxes, and connected together in various ways to visually represent control flow. A prominent example of this in the educational domain isRaptor[135]. Flowgorithm3 andVisual Logic4 are other commercial applications that can be used to create executable flowcharts. The aforementioned professional environmentsMaxandLabVIEW use a data flow diagrammatic approach to allow the developer to ‘wire’ components together. As a paradigm employed in task automation tools, Desolda et al. implemented a diagrammatic programming paradigm in their comparative usability study. However, it emerged that even study participants with technical expertise found this paradigm to be the worst in terms of performance and satisfaction over PBS-based alternatives. They concluded that“even technical users, who should be acquainted with wired notations, perform better and are more satisfied if using other interaction paradigms”[125].

4.1.3.2 Iconic programming

Iconic programming languages fall within the “geometric” visual language class. In iconic programming environments, code is represented by graphical symbols (or icons) that can be spatially arranged to represent how an application should function. Geometric languages are distinct from connection-based visual languages in that there are no explicit connections between the icons. Instead, the spatial location of icons relative to one another determine their relationships. One example is the Kodu environment, which uses graphical icons to allow developers to compose program behaviour in the form of event-driven clauses [136]. GALLAG Stripis another similar environment that represent actions and responses as graphical icons, for composing rules in context-aware applications [137].

4.1.3.3 Blocks-based programming

A blocks-based programming language is formed of constructs resembling jigsaw puzzle pieces. Like diagrammatic languages, they are also “connection” visual languages, wherein the visual elements have insets and protrusions that show how they should fit together in the correct way, consistent with the jigsaw puzzle metaphor. This visual syntax reduces the possibility of errors and allows the user to focus on the semantics of their application [138].

3_{http://www.flowgorithm.org/ Accessed 17/03/18} 4_{http://www.visuallogic.org/ Accessed 17/03/18}

66 CHAPTER 4. END-USER DEVELOPMENT

Figure 4.3:Palettes and canvas of Scratch (left) and App Inventor (right)

Scratchis a prominent example of blocks-based programming [129]. It is targeted at children and young adults, allowing them to drag and drop blocks to create visually appealing games and animations.App Inventoris a similar project to the work in this thesis in terms of programming paradigm and development target [131], which allows users with no programming experience to develop a wide range of fully functional apps for Android devices. Both environments are notable for their ability to motivate and engage users, through the rapid development of meaningful applications, but also comprise a variety of programmatic constructs such as loops, conditional statements, and variables that allow complex behaviour to be implemented if desired.

App Inventor utilises Blockly5, an open-source library for creating blocks-based languages. Blockly has supported the creation of a variety of such languages in a growing number of domains, but particularly for educational purposes. A recent article inCommunications of the ACMprovides a thorough discussion of the success of blocks-based programming and the reasons for its utility as an educational tool [138]. While its benefit for supporting students’ transition to textual programming is not of relevance to the work in this thesis, the learnability of the paradigm itself is particularly important if it is to be adopted in time-constrained practices. Blocks-based programming is the development paradigm of choice for Jeeves, the visual programming environment implemented in the work of this thesis. (The specific design details of Jeeves will be discussed in Chapter 5.) From a general perspective, salient features common to all blocks-based environments have been applied, which can be observed in the screenshots of ScratchandApp Inventorin Figure 4.3. These features include:

4.1. EUD AND VISUAL PROGRAMMING 67

• Drag-and-drop canvas:In blocks-based environments, the canvas is a central feature onto which developers can drag, drop and rearrange their application blocks. These canvases can be panned and zoomed, allowing users to layout their applications in a flexible way.

• Categorical palettes: To supportrecognition over recall[138], all blocks can be accessed through named palettes adjacent to the canvas. Clicking on a palette name displays all the blocks in that category, allowing more experienced users to efficiently find the blocks they need, while minimising effort required for novices to do so.

• Effective visual metaphor: The jigsaw puzzle metaphor has been identified as an effective visual metaphor for novice programmers. Program constructs that can not be combined in a syntactically correct way do not fit together as puzzle pieces, thus eliminating syntax errors, which are a major source of frustration in learning to program textually. Bau et al. refer to this feature as“syntax-directed editing with constrained direct manipulation”[138].

• Secondary notation: Blocks-based languages make substantial use of secondary notation to convey additional meaning to both developers and readers of specifications. The shapes, colours and sizes of blocks are all suggestive of their relationship to one another. Further, the position of blocks in 2D space can be used as an extra form of secondary notation, in contrast with iconic programming languages where 2D position is enforced. Further, blocks do not have to be connected in a network as in diagrammatic languages, which minimises the potential for confusing masses of connections.