1_PEOPLE, TOOLS AND PROBLEMS

Behovet for brukervennlighet (HCI): I utformingen av applikasjoner og verktøy er vi opptatt prinsipper som gir maksimal brukervennlighet.
Behovet for modifiserebarhet: I utformingen av verktøy må vi ofte innse at behovene endrer seg hele tiden. Applikasjoner er designet for et bestemt formål og har ikke innebygd mekanismer som gjør den modifiserbar – tilpasset skiftende behov.
Behovet for en problem orientering: Et kraftig programmeringsspråk kan brukes til svært mange formål. Det har sin styrke når det gjelder modifiserbarhet. Ofte er de lite brukervennlige og det finnes få byggesteiner innenfor bestemte fagområder. Derfor blir utviklingen en langsommelig prosess.

Brukervennlighet

Når vi snakker om brukervennlighet vil mange tenke på gode innføringer i grunnleggende spørsmål knyttet til design og presentasjon av informasjon - hvordan lage gode websider. Se for eksempel hva Startsiden skriver om web-design:

Vi i Startsiden vurderer hundrevis av nettsteder hver eneste uke. Det er en tøff jobb og vi gråter ofte, for hjerteskjærende smakløshet preger mer enn én webløsning. Og det er faktisk ikke alltid amatører som står bak; noen tar seg også betalt for å produsere blinkende piler, flagg som vaier i vinden, jordkloder som snurrer, MIDI-filer som piper som kastrerte kirkemus, framesett som kan skremme Fanden på høylys dag og grafikkfiler støpt i bly.
Derfor: Til alle som vever nettsteder, enten på gøy eller i ramme alvor, plukk med dere gode råd fra noen av nettets glupeste guruer! Web Style Guide fra webteamet ved Yale-New Haven Medical Center finnes både som bok og som komplett nettutgave. Dette er en fin innføring i kunsten å kombinere velprøvde redaksjonelle prinsipper med designløsninger for det nye (?) mediet Internett. Sun Microsystems har også utviklet en Guide to Web Style som er verdt å titte nærmere på..
       Jakob Nielsen er blant nettets fremste talsmenn for funksjonalitet og brukernytte, og nettstedet hans Useit.com kommer du ikke utenom. Hans Top Ten Mistakes in Web Design fra 1996 er fremdeles pensum for alle nettvevere. Mer om brukernytte finner du bl.a. hos WebWord.
       David Siegel er en høyt profilert ringrev blant webdesignere, og forfatter av en av de mest populære bøkene om nettdesign; Creating Killer Websites. Ikke alt som kommer fra David Siegel er like bra, og deler av stoffet på sidene hans ser ut til å ha gått ut på dato. Mannen er likevel verdt å lytte til.
       Konkrete designløsninger i HTML og andre språk finner du for eksempel hos Webmonkey. Startsidens egen kategori Webdesign er et annet naturlig utgangspunkt.
       Ofte er det mye å lære fra andres feil. Interface Hall of Shame gir en god oversikt over eksempler på dårlige løsninger. Web Pages That Suck er også en gullgruve i så måte. (Startsiden mars 2000)

Guidene nevnt ovenfor sammen med de som er vist til før i kurset tar ikke for seg bare de siste nyvinningene på teknologifronten, men er gode innføringer i grunnleggende spørsmål knyttet til design og presentasjon av informasjon. Det finnes velprøvde metoder som kan være med å hjelpe oss i utformingen og testing program og verktøy. I prinsippet er det to viktige innfallsvinkler enten brukertesting eller at eksperter går igjennom

designforslagene. Den siste former blir ofte kalt for heuristisk evaluering. Se en kort oppsummering på norsk av noen sentrale punkt i Jacob Nielsens bok Usability Engineering:

Brukertesting

Heuristisk evaluering

Brukervennlighet- modifiserbarhet – problemtilpassing

En god del av disse prinsippene kan også brukes i utforming av verktøy, men her står vi samtidig over noen andre utfordringer. Vi må både se behovet for brukervennlighet, behovet for modifiserbarhet og behovet for å komme raskt fram til en løsning som like viktige betingelser for det vi kan kalle et godt verktøy. Problemet har vært at det er vanskelig å balansere mellom forskjellige behov. Økt brukervennlighet har ofte godt på bekostning av modifiserbarhet. Økt faglig tilknytting (bransjespesifikke applikasjoner) har økt brukervennligheten, men nedsatt modifiserbarheten. Økt modifiseringsmuligheter (kraften i verktøyet) kan gi nedsatt brukervennligheten.

Det er ikke sikkert at design av websider for å holde oss til dette eksemplet blir slik vi har tenkt det i framtiden. Gode verktøy kan kanskje automatisere denne prosessen. Et annet problem som melder seg er mengden av informasjon og alle nettportalenes forskjellige uttrykksform. Vil det ikke være bedre å la sluttbrukeren bestemme hvordan og hva en vil se på sien egne private hjemmeside. I dette tilfelle snakker vi om en ny generasjon av fleksible verktøy ?

Utvikling av sluttbruker verktøy er kommet langt på vei. Alexander Repenning tar for seg i teksten nedenfor noen viktige prinsipper:

Figure 1-1: People, Tools and Problems

The three fundamental relationships among people, tools, and problems are:

• People and Tools: This relationship characterizes the usability of a tool, that is how easy it is to understand and use a tool. However, it does not make an assessment about how useful the tool is with respect to a problem to be solved. The function of a hammer, for instance, can be grasped easily, but the hammer may not be a useful tool to bake bread.

• Tools and Problems: This relationship describes the degree of orientation of a tool toward a problem domain. Additionally, many problems are oriented toward tools. Assembling a car would quite naturally lead toward the use of traditional tools such as screwdrivers. The fact that screwdrivers are, among many others, handy tools to assemble a car is not accidental. The tools have been known for a very long time and were taken into account when the parts of the car were designed.

• People and Problems: This relationship describes people’s understanding of the problem. There can be a complex interaction with the tools used. In many cases the problem is not understood fully first and then “implemented” using the appropriate tools. Instead, the understanding of the problem evolves with the progress of solving the problem. That is, the process of solving the problem will uncover the true nature of the problem. Nardi points out that tools such as spreadsheets help people to solve problems by supporting the incremental development of external, physical models of their problems [84] .

This section discusses and contrasts two important properties of these relationships:

• Control and Effort: Different tools for identical problem domains may differ from each other with respect to the degree of control a person will have over the problem-solving process. A high degree of control may be desirable but it is typically associated with a large effort.

• Domain-orientation [27, 28] : Domain-orientation is the degree to which a tool reflects properties of the problem domain. A general-purpose tool is domain independent and therefore can be used for a large variety of problem domains. However, it may provide little support for an individual problem domain and inflict a large transformation distance [29] .

Introduction to People, Tools and Problems

There is often a danger in getting trapped in a certain perspective on some issue after an extended exposure. A reader of this thesis has likely to have been exposed to some notion of programming in one way or another. In an attempt to gain common ground and to take a step back to widen the perspective, I will make use of an analogy. The analogy - how to get bread - is, hopefully, a task that most can relate to. It does not make the claim to be a perfect match to programming. In fact, the analogy may look absurd at first, but it will shed some light on why people choose a particular degree of control and effort with respect to a task (including programming and baking bread). The analogy will be used throughout this dissertation.

Lets assume that a person is about to solve the problem of getting bread. Feasible alternatives for this person to deal with this demand include the following approaches:

• Bake the Bread (Do it from Scratch): We can offer that person a baking tutorial. On the one hand, the person has a high cost in terms of time but, on the other hand, the person benefits by acquiring deep knowledge about baking ingredients and also acquires procedural skills in baking (how to knead the dough prepare the oven). This hands-on experience will not only teach the person how to bake the N different kinds of breads that were featured in the baking tutorial, but it will also give that person some model of bread making. Consequently, the person can use this model to create variations of breads he has never done before by modifying ingredients as well as baking procedures. Based on the experience, intuition, and luck of the person, the newly created breads will be more or less edible.

• Use a Bread Machine (Construction Kit): The bread machine will dramatically reduce the need for procedural skills. The task of baking bread is substituted by the task of looking up the right kind of ingredients for the desired bread type and pushing the appropriate set of buttons on the machine. At this level of operation the talent of the person making the bread is largely irrelevant. Due to the dedicated nature of the bread machine only a small set of tasks can be tackled with the machine but with very low effort from the person using the machine. This may lead to de-skillification, i.e., the reliance of man on machines may cause the complete inability to do things (like baking bread) without the support of a machine. This would be tolerable for the person interested only in the product of a process (e.g., the bread) and not interested in how to make bread. Furthermore, the black box approach of the bread machine prevents the person from modifying the baking procedure beyond the degree of control featured in the baking machine controls. Some people have also pointed out that the use of a bread machine will also have social impacts because the process of bread making create a bond between the people involved in the process. Maybe not too surprisingly these comments have come, without exception, from people NOT actually involved in bread making.

• Defrost Frozen Bread (Partial Delegation): Minimal procedural knowledge and minimal tool requirements. The frozen bread probably comes with instructions on how to preheat the oven (how long, what temperature). The control we have is minimal but can be crucial. We can bake the bread at a time that pleases us. Also we have control over how dark the bread should get.

• Buy the Bread at the Store (Complete Delegation): No procedural knowledge whatsoever is required with this approach. The goal of making bread has been replaced by the less ambitious goal of buying bread. The person has only limited control over the kind of bread he can get by driving to the most promising place and picking the bread most closely satisfying his needs.

These different approaches are all specific to the same problem domain of getting bread. Baking bread, using a bread machine, defrosting bread, and buying bread are different means leading to the same ends. However, the approaches vary considerably in their degree of control and effort. Is the same true for software?

Control and Effort are Orthogonal to Domain-Orientation

We have seen four different approaches to solve the problem of getting bread. The “getting bread” problem helps to illustrate certain points related to programming. Most importantly, it illustrates that there is no single, “right” degree of control; different people have different needs. Furthermore, it indicates that the notion of control and effort is orthogonal to domain-orientation. Now, let’s relate the four levels of bread making back to the task of programming:

• Do it from Scratch (Bake the Bread): This is the lowest level of computer usage; before we use it we have to learn how to write a program and then we create and use the program. With this approach we have a very high degree of control because we can address every facet of our needs with the appropriate program. Since the person writing the program and the person using the program are identical we have at least the potential for the solution to match the problem optimally. However, as discussed before, the process of programming is very complex and requires a high effort.

• Construction Kit (Use a Bread Machine): Here we have the help of a tool to reduce our effort but at the same time it reduces our degree of control. Unlike the general-purpose programming approach, we are using tools that are very specifically oriented toward an application domain. Similar to the bread machine, a construction kit can be viewed a compromise in terms of effort and control. We are no longer in full control of all the design parameters but at the same time the effort of using a construction kit is significantly reduced by the anticipation of a solution approach built in to the tool.

• Partial Delegation (Defrost Frozen Bread): In the case of partial delegation we delegate the task to other entities such as programmers or agents. Ideally, we can delegate all tasks that are of no interest to us such that we are left with the tasks that we like to have control over. In the case of defrosting bread the only control we like to have over the bread baking process may be to determine the point in time when the bread should be oven fresh. A non-baking example is the specialization of a form letter. Some person may have gone through the trouble of creating a letter head. A different person can make use of this effort by copying the prefabricated form letter and filling in the specifics.

• Complete Delegation (Buy the Bread at the Store): This is the highest level of computer use. We, or maybe even somebody else such as a system administrator, have bought a piece of software to solve our problem. That software, as it comes out of the box, may not be a perfect match to our problem. We may have to customize it, or if this is not sufficient we may have to adjust ourselves to it. In other words, we have only very little control in a situation like this. If we have found a good solution to the problem then we have little effort in using it. Of course, we still need to learn how to use the software effectively, but compared to the task of writing the software ourselves, this effort is relatively small.

This analogy between the problem of getting bread and programming indicates that control and effort are additional dimensions to domain-orientation. If we have very specific needs that are not covered with some off-the-shelf solution we have to go through the effort of learning how to control the process we are interested in - be it bread baking or programming. It is important to note that there is no “right” way to do these things. Depending of personal preference, background, time constraints, and skills one approach might be preferred over another. Worse yet, the situation may change over time. For example, it may make sense to bake bread from scratch Saturday afternoon but the same approach might be out of the question on Monday morning.

What kind of tools can we build if the relationships among people, tools, and problems are so intricate? Can a tool address only a single problem solution approach or could there be special kinds of architecture accommodating a variety of approaches? The following section lists a number of approaches, differing in their degrees of domain-orientation and control, to make the computer more useful and usable [27] .

Past "Solutions" to Simplify Programming

In the early ages of computers the majority of programming complexity arose from the need to have a deep understanding of the computer architecture and not from the complexity of the problem to be solved. The computer as a programmable tool reflected in no way the needs of different people to solve different kinds of problems. Instead, computer programming was technology driven. The performance of computers was so limited that problems and people had to adjust to the constraints dictated by early CPU designs.

Eventually the continuous improvement of computer technology enabled designers of programming mechanisms to trade in performance for a reduction of programming complexity through the design of higher-level programming languages. The goal was and still is to minimize the total complexity of programming by minimizing the accidental complexity introduced by the task of programming in addition to the intrinsic complexity of the problem to be solved. The following sections elaborate the notions of intrinsic and accidental complexity and describe several approaches to simplify programming.

Intrinsic and Accidental Complexity

The complexity of programming can be divided into the complexity intrinsic to a problem to be solved by a program and the accidental complexity of the programming language and its environment to implement the program 11In this model the intrinsic complexity cannot be reduced by any sort of mechanism because it merely reflects the nature of the problem to be solved. Of course, as in the case of the bread machine, we can build mechanisms that encapsulate and abstract certain intricacies of a problem-solving process. However, by doing so we are not reducing the intrinsic complexity (e.g., baking the bread). We are simply delegating part of the intrinsic complexity to a mechanism that had to be built by another human being.

Accidental complexity, on the other hand, is complexity introduced by the artifact and not found in the original problem to be solved. Accidental complexity is related to the quality of match between the problem and a solution. Designers of artifacts representing solutions do not typically build in accidental complexity. However, they may be forced to operate with trade-offs due to factors such as the price of the artifact, its maintainability, etc. These trade-offs, in turn, quite often lead to accidental complexity. In complex artifacts it is hard to distinguish between intrinsic and accidental complexity because the border between means and ends can easily be blurred. For instance, the tools resulting from solving a problem often change the way we look at solving a problem. Zipf 118describes the interaction of means and ends as re-fitting where the activity of “job seeks tools” is typically followed by “tools seek job.” The following example serves as a simple instance of accidental complexity probably due to financial reasons. Figure 1-2 shows the control panel of an industrial-size coffee machine as it is still used in a large Swiss company.

Figure 1-2: Control Panel of Coffee Machine

The user interface of this coffee machine consists of two parts: a lookup table to find the code to a specific choice (e.g., coffee with cream but without sugar corresponds to code 13), and a key panel (0-9) to enter that code. The need to map the specification of the desired product to a code before the selection is done cannot be explained in terms of the intrinsic complexity of getting coffee. Instead, this need is part of accidental complexity featured in the interface design of the machine. The relationship of codes and products has no helpful meaning in everyday life beyond that particular brand of coffee machine. The numbers in the product/ingredient lookup table could have been easily replaced with a matrix of actual keys. Pressing these keys would reduce the memory load of the user by no longer requiring him or her to remember a two digit number (even if it is just for a very short time).

I can only speculate why the company producing the machine made the design decision of using a lookup table. In terms of the user interface, extending the palette of products and ingredients only requires printing an extended version of the lookup table featuring new codes for new combinations. The codes can still be entered using the same 0-9 keypad mechanism. Extending the palette of products would have much greater implications with the matrix of direct keys approach. It would require adding new keys with all the consequential work involved (drilling holes, wiring the new keys, etc.). Hence, we can speculate that the company has chosen an increased complexity now in order to make future capability extensions of the machine cheaper.

Accidental complexity can change the way we think about problems. New problem representations can emerge from accidental complexity in unpredictable ways. In the case of the accidentally complex coffee machine, for example, people would start to remember the product codes and ignore the lookup table. Furthermore, the codes were also used to communicate with colleagues. I was frequently asked to get 13s and 14s for my colleagues after lunch. The use of product codes as representation and communication media is more efficient than the use of the full product descriptions. The description “a 15 and a 27” is more concise than “an espresso with cream but without sugar and a caffeine-free coffee with cream and sugar.”

In terms of the people, problems, and tools relationship, intrinsic complexity emerges from the problem to be solved. Accidental complexity, on the other hand, can have multiple sources, including:

• Tool/Problem Mismatch: tools may not be suited for a problem. For instance, to fix a simple problem with a radio using plumbing tools can be very problematic.

• People/Tool Mismatch: even tools that seem to be suited for solving a problem may be of little use in the hands of a person not familiar with the tools. On the other hand, tools generally perceived to be mismatched for a job can work out satisfactory in the hands of a person highly skilled with these tools.

Unlike the task of getting coffee, most real-world programming problems are typically very complex. How does intrinsic and accidental complexity manifest itself in programming? While a programming language cannot reduce the complexity intrinsic to a problem, it can and should minimize the accidental complexity. In the following sections, a number of approaches that attempt to reduce the accidental complexity of programming are described.

High-Level Programming Languages: Delegating Accidental Complexity to Compilers

Accidental complexity is often due to the immature state of a technology. Early cars, for example, required their users to be engineers. Cars back then were used for the same purpose of transportation as today’s cars. However, engines were very unreliable and hence required a lot of maintenance. Additionally, tasks such as starting the engine required technical knowledge and muscle power. The complexity arising from using and maintaining early cars was not part of the problem of moving from point A to point B. In other words, that complexity was accidental. Over time this accidental complexity was reduced to the point where users no longer needed to be engineers.

People who programmed early computers were faced with similar problems. Early computers had only very limited resources (memory and speed). These resources had to be used very carefully. Consequently, people were forced to a very high degree of control at the cost of high effort of programming. Programming in assembler enables the programmer to control every bit in memory and make use of all the instruction featured by a CPU. With the advent of faster CPUs and the availability of cheaper memory the control of programs at the CPU instruction level became less necessary and slowly was replaced by so-called high-level languages.

A surprisingly large resistance by programmers to high-level languages was partly due to the bad implementations of early high-level language compilers, which produced large and inefficient code. Furthermore, programmers lost control over certain implementation aspects. To have mastered the accidental complexity of assembler programming became part of a programmer’s identity, and hence moving to a higher level of programming meant giving up a part of this identity.

Despite the above-mentioned problems, high-level programming languages have largely replaced low-level languages. On the one hand, programmers have lost control over accidental complexity; for instance, they no longer control the use of processor registers. On the other hand, however, the delegation of accidentally complex tasks, such as the allocation of registers, to a high-level programming language compiler has enabled programmers to create programs that solve intrinsically more complex problems.

It's Only a Syntactic Problem - Really?

High-level programming languages have reduced accidental complexity by factoring out processor-specific intricacies and supporting the management of more advanced memory paradigms (including arrays, stacks, and records). But orthogonal to semantics there is the dimension of syntax. Syntax can be viewed as a mechanism to package semantics, i.e., a formal description of structure to make semantics tangible. Syntax gives shape (textually, verbally, spatially) to conceptual entities. That is, if we have a certain semantics captured in a syntax then we can express ourselves and communicate with other humans or with machines. In the early 1950s it was believed that syntax was the main key to unlocking the problems of programming. In other words, it was believed that the improved syntax of programming languages would strengthen the relationship between people and tools:

One line of argumentation promoting the English-likeness of programming languages led to the design of COBOL. In 1950 COBOL was advertised as a new communication media:

"With the new COBOL coding system, programming as a profession is obsolete. Everyone will program. The boss will dictate a program to his secretary, who will keypunch it onto cards for him."

Back then it seemed that the suitability hinges on the form of a programming language and that the meaning of the language is less crucial as long as it includes basic primitives such as iteration, and conditionals. As we know by now, COBOL did not quite live up to the above promise. In fact, quite the opposite is true. Nonetheless, up to this day the biggest fleet of programmers is programming using COBOL.

. Task-Specific Languages

The previous mechanisms of programming have treated languages as isolated entities. Task-specific programming languages, in contrast, are task-oriented, embedded languages. Although these languages are oriented toward a specific task, they still support general-purpose programming.

Example task-specific languages include:

• Post Script is an imaging language for printers. Post Script is typically embedded physically into a piece of hardware, i.e., a printer. The essence of the language is the drawing commands, which create and control graphic output on a device such as a printer or a display. However, Post Script’s funcionality includes a fully featured set of programming primitives such as variables, control structures, and procedures.

• Mathematica contains its own embedded language to visually render collections of data and functions.

• S is a statistical package that features a language to manipulate and render data.

• HyperTalk is the language used in the HyperCard system to write scripts that get attached to HyperCard user interface components, such as buttons.

• FORTH was originally designed and used to control large telescopes. Quite quickly, a FORTH user community emerged that uses FORTH for different purposes.

Interestingly, task-specific languages do not quite fit into any of the approaches described in the “getting bread” analogy. They can hardly be viewed as delegation mechanisms (partially or complete). Task specificity makes task-specific languages similar to construction kits. Like bread machines, task-specific languages include the main ingredients for a specific task. However, most task-specific languages feature, additionally, a set of general-purpose primitives. Post Script, for instance, despite its primary focus on image-oriented issues, could be used to write applications of a very different nature. Theoretically, it would be possible to write a C compiler in Post Script.

. Let's Make it Visual: Visual Programming

Similar to COBOL, visual programming languages make strong claims about simplifying programming by improving the form of programs. Unlike COBOL, visual programs are no longer limited to genuine textual representation, but instead make use of diagrams. Some [6, 52, 53] argue that visual programming systems are supposed to help users to program computers by capitalizing on human nonverbal capabilities such as spatial reasoning skills (Figure 1-3). Others, such as Smith [106] , prefer to view visual programming tools as approaches to augment the creative nature of visual thinking 2] .

Figure 1-3: Visual Programming Tools

The goal of this paper is to define a theory about the evolution of useful visual programming languages. I believe that the main problems of visual programming environments come from:

(i) the lack of problem domain-orientation, i.e., there is no support for the tools/problems relation, and

(ii) the static interpretation of programs and programming.

To that end, this section describes the shortcomings of current visual programming systems and sets the stage for a programming substrate used to create dynamic visual environments.

A state of mind that I call Visual Puritanism describes the tenacious desire of many members of the visual programming community to completely replace instead of to gently augment text-based with non-textual approaches to programming. The goal of visual programming should not be to make things visual per se, but to help people overcome at least some of the problems associated with the very complex task of programming. Instinctive justifications for visualization, such as "one picture is worth a thousand words," will fail to simplify programming. Figure 1-4 shows a data flow diagram [53] representing the expression x²- 2x + 3. In general, data flow diagrams have many good applications, but in this particular instance dealing with algebraic expressions, it is unclear why one would prefer the visual representation of the data flow over the textual representation.

Figure 1-4: Data Flow Diagram Representing x²- 2x + 3

The visual programming community is relatively young and immature. It therefore, tends to over-focus on the importance of "pure" visual representations. Visual programming should be about capitalizing on human spatial reasoning skills, but it should NOT make the mistake to ignore other human skills including the ability to deal with abstract entities such as text. Hence, a good (visual) programming environment should carefully combine both skills.

Empirical tests suggest that despite their versatility, general-purpose visual programming systems have limited usefulness. Green has compared the performance of programmers using textual and non-textual representations [48] . Surprisingly, he found that in many cases the programmers using textual representations outperformed the those using non-textual representations. Brooks [11] , being even more skeptical, concluded that visual programming is temporary hype leading into a blind alley:

“A favorite subject for Ph.D. dissertations in software engineering is graphical, or visual, programming - the application of computer graphics to software design. Sometimes the promise held out by such an approach is postulated by analogy with VLSI chip design, in which computer graphics plays so fruitful a role. Sometimes the theorist justifies the approach by considering flowcharts as the ideal program-design medium and by providing powerful facilities for constructing them.

Nothing even convincing, much less exciting, has yet emerged from such efforts. I am persuaded that nothing will.”

Before we can find a way out of this alley we need to understand some of the main claims and problems of visual programming.

Visual Languages Lack Domain-Orientation

With very few exceptions visual programming languages are of a very general purpose nature. In other words, they lack problem domain-orientation. These general-purpose visual programming systems that visually represent concepts found in conventional programming languages such as boxes and arrows actually represent procedures and procedure calls. These systems have two advantages. First, they are applicable to a wide range of domains. Second, visual representations can facilitate syntactic constructions. For instance, shapes in BLOX Pascal (Figure 1-5) guide the programmer toward making syntactically correct constructions [39] . However, general-purpose visual programming systems are difficult to use for novice programmers (e.g., BLOX Pascal still requires basic Pascal knowledge) and provides little - if any - gratification to expert programmers.

Figure 1-5: BLOX Pascal

There is nothing intrinsic to the concept of visual programming languages that prevents them from being domain-oriented. Construction kits, for instance, could be viewed as domain-oriented visual programming systems. Domain-oriented visual-programming systems represent artifacts pertinent to the end-user’s task at hand. A system such as the pinball construction kit (Figure 1-6) is easy to use for novice programmers [31] . The process of programming in a construction kit-like environment consists of the selection and composition of domain-oriented components (e.g., pinball components). However, it is difficult to reuse systems tailored to a specific domain for other domains. For instance, the pinball construction kit could not be reused for numerical applications such as tax forms. Perhaps the biggest problem with domain-oriented systems is that no high-level substrate exists facilitating their creation.

Figure 1-6: Pinball Construction Kit

Empirical tests suggest that despite their versatility, general purpose visual programming languages, in many situations, provide little or no advantage over conventional textual programming languages [48] . Domain-oriented visual programming systems, on the other hand, seem to be very useful for non-programmers but are difficult to build from scratch [40] . The problem is, therefore, to find a new mechanism that will overcome the disadvantages of each approach without sacrificing their advantages.

The Dimensionality of Programs

An often-used justifications for VL research is that visual languages, in contrast to traditional, text-based languages, make use of more than just one dimension. Furthermore, it is argued that this characteristic makes better use of human cognitive abilities. Higher dimensionality is used as one of the main criteria to distinguish visual from non-visual languages. The following two claims make assertions about the dimensionality of textual and non-textual programs.

Claim 1: Traditional, Text-Based Languages Are One Dimensional

Nobody would argue that the manifestation of a "good" Pascal program is not two dimensional. Indentation, a visual manifestation of program characteristics in the X dimension, is a crucial component of the Pascal philosophy. The visual programming community (for instance Shu [103] ), however, argues that this manifestation is for the human eye only, and because it has no relevance to how the program gets parsed it is irrelevant. That is, a Pascal compiler would treat the following two fragments of code identically:

Fragment 1:

WHILE NOT EOF (File) DO BEGIN

READ (File, Character);

DO-SOME-OPERATION (Character);

END;

Fragment 2:

WHILE NOT EOF (File) DO BEGIN READ (File, Character); DO-SOME-OPERATION (Character); END;

We seem to distinguish between interpretation by the human (presentation of a program) and interpretation by the machine (semantics of the language). Generally, the characteristics of a physical program manifestation pertinent to semantics seem to be a subset of the characteristics of the presentation. That is, there are typically more things that matter to a human than to a machine in terms of visual representation. For instance, the presentation information of a nicely indented Pascal program gets lost through compilation. The VL community leans toward the classification of programming languages by their semantics in the case of text-based languages such as Pascal and, hence, concludes that Pascal (and related languages) are one dimensional. Taking this definition literally and knowing that FORTRAN has a very limited interpretation of column information to distinguish statements from comments, we could conclude that FORTRAN, given its two-dimensional semantics, is a visual language.

Claim 2: Visual Languages Are ³ Two Dimensional

A large set of contemporary visual programming languages have been classified as "icons on strings" [71] . That is, we have some sort of objects with a visual manifestation (e.g., icon) and some sort of spatial relationship among these objects (e.g., links stringing icons together). The links can have many different semantics. Most frequently, the links will represent concepts such as control flow or data flow. Exceptions to the “icons on strings” theme include systems such as BLOX Pascal, in which a program consists of a spatial composition of jigsaw puzzle pieces representing Pascal primitives (see Figure 1-5). Although icons themselves have a two-dimensional representation nothing prevents a user from arranging a set of icons (THE program) in a one-dimensional manner. A flow chart, for instance, is still a flow chart even if we align every element of the flow chart in a single line (Figure 1-7).

Figure 1-7: Isomorphic Flow Charts: Neat and Linear

So, again, the second dimension is interpreted only by the human and not the machine. In other words, the dimensionality of this type of visual language is about the same as the dimensionality of text-based languages. Therefore, if we use dimensionality as a criterion for “visuality” we would be forced to conclude that a flow chart is about as “visual” as a Pascal program.

The point is not to assert that Pascal is a visual programming language but the question arises whether we can come up with a more meaningful criterion for “visuality” than dimensionality. Additionally, I like to point out that the discrepancy between human and machine interpretation of current diagrammatic representation could be one of the main causes of the low visual language acceptance. One of the reasons why so many people dislike visual languages is because they think VLs make bad use of screen estate [11] , i.e., visual programs "are too big." Maybe we need to increase the number of spatial characteristics pertinent to machine interpretation by including so far unused characteristics crucial to human interpretation. For instance, in Figure 1-8 chart 1 seems to be much tidier than chart 2, probably because it makes better use of implicit spatial relations such as writing order (in the US: left to right, top to bottom).

Figure 1-8: Neat and Messy Charts

Petre et al. [89] call the use of layout and perceptual clues, which are not formally part of a notation describing the semantics of a visual language, “secondary notation.” Spatial relationships, including adjacency, clustering, white space, and labeling, are employed to clarify information such as structures, functions, and relationships in diagrammatic representations. Furthermore, they argue that the secondary notation is crucial to comprehensibility:

“Poor use of secondary notation is one of the things that typically distinguishes a novice’s representation from an expert’s.”

As long as visual languages give the user so much freedom that the icons, representing primitives in the visual program, can be completely rearranged to a total, nonsensical mess without changing the semantics of the program, I do not think we have found “truly” two-dimensional programming languages.

Visual Programming Is Natural Because People like to Doodle

One claim of visual programming is that it is “natural” because people, including traditional programmers, often create diagrammatic representations to solve problems. I myself doodle a lot to program: class hierarchies, data structures, state diagrams, etc. All these things appear to be intrinsically spatial to me. These diagrams help me to understand certain issues of programming problems or they help me to communicate problems to peers. However, what is the nature of these diagrams in terms of their life span, completeness, consistency , and complexity?

• Life span: Although at the time of solving a problem (often just a very small fraction of a big programming project) doodling a diagram may be crucial, the diagram is often disposed of after the problem is solved. In other words, the life span of these diagrams is often very short, from a couple of minutes to a couple of days.

• Completeness: The diagram is typically only related to a very small fragment of the entire project (e.g., a diagram of a data structure). Even these fragments are complete only to the point where they fulfill their purpose of explaining an abstract relationship to another person.

• Consistency: In the process of sketching a diagram, understanding often progresses or even changes radically over time and so does the notation used. That is, not only does the diagram evolve over time but also the notation capturing the semantics of the diagram. This can lead to inconsistencies in the diagram. Due to the short life span of a diagram this usually does not have bad implications.

• Complexity: Diagrams are less useful if they become to complex. The advantage of using diagrams arises from human perceptual abilities such as recognizing interesting patterns and structures in images. But no matter what spatial notation we use there always will be a critical mass of complexity at which a visual representation becomes overwhelming and therefore loses its "intuitive" appeal. This, of course, does not mean that there cannot be a mechanisms to break up complexity, but so far I have not seen a convincing one.

No doubt, the use of diagrams to solve problems can be crucial. However, we have to be careful with simplistic conclusions such as: “to doodle is natural” therefore “visual programming is natural.”

The problems solved by doodling on the back of some envelope are different in nature from the problems arising in big programming projects. This does not mean that there cannot be any useful visual programming environment. However, we have to be aware that the nature of a small problem may be very different from the nature of a more complex problem.

Explicit Versus Implicit Spatial Notations

One problem of visual programming is that it makes extensive use of screen estate [11] . Partial solutions include the use of larger screens or mechanisms to zoom in and out of spatial representations. However, I believe that the main problem arises from the use of explicit spatial notations that are due to the general-purpose nature of most visual programming languages. Implicit spatial notations, although less flexible than explicit spatial notations, can be used in domain-oriented applications to substantially increase the spatial density of visual representations.

Explicit as well as implicit spatial notations are concerned with the visual representation of relations between objects in diagrams. I distinguish two types of objects: The core objects, such as programming primitives in visual programming languages, are the objects pertinent to a user. Unlike implicit notations, explicit notations include relation objects, such as labeled arrows, in addition to the core objects.

The use of relation objects in explicit spatial notations makes a representation very flexible. The relation between the two core objects in Figure 1-9 indicated by the arrow relation object could have very different semantics such as father, super class, boss, etc.

Figure 1-9: Explicit Spatial Notation

Flexibility arises from the existence of a relation object that can be augmented in unlimited ways. An arrow, for instance, can have many different characteristics including labels, color, hatch pattern, arrow type, and thickness. This flexibility can be employed by users to create new relation types. On the other hand, the explicit notations are wasteful with respect to spatial attributes of core objects. The position, for instance, in Figure 1-9 of the two core objects is completely irrelevant. What matters is the orientation of the relation object with respect to the core objects. Consequently, the semantics of a diagram using explicit spatial notations is not related to the Gestalt of the core objects. In other words, we could rearrange the core objects to any visually pleasant or unpleasant diagram without changing the semantics of the diagrams (see Figure 1-9).

The combination of wasting spatial attributes of core objects and the use of relation objects can lead to low spatial densities of explicit spatial notations. In visualization tools, such as object class graphers, the space used by relation objects as well as their number can easily exceed space and number of core objects.

At the cost of flexibility, implicit spatial notations can increase the spatial density of representations by (i) avoiding the use of relation objects, and (ii) making more economical use of spatial attributes of core objects. In implicit spatial notations semantics is derived from spatial relations between core objects. These spatial relations, in turn, directly emerge from the positions of core objects. Spatial relations can be topological, (proximity, order, enclosure, continuity) and Euclidean (angularity, parallelism, and distance) [51] .

Implicit spatial notations are less flexible because they rely on the more subtle perception of implicit spatial relations. Since these spatial relations are implicit, that is they have no physical manifestation in a diagram, they cannot have characteristics such as labels, colors, hatch patterns, arrow types, or thicknesses. It can be difficult to discern a large number of different spatial relations. Most visual programming paradigms based on implicit spatial notations employ only a very small number of spatial relations. ChemTrains [4-6] , for instance, uses only one (enclosure).

The use of implicit spatial notation can lead to very efficient and natural representations if there is an intuitive mapping between the semantics to be represented and the spatial relation selected. In Figure 1-10 the relation between two core objects, a roof and a frame of a house, is captured implicitly; the roof is above the frame.

Figure 1-10: Implicit Spatial Notation: Roof is Above Frame of House

In this case, the spatial orientation of how we perceive real roofs and real houses through our bodies in the three-dimensional world quite naturally maps to the two-dimensional figure above of a house with a roof. In this case it seems absurd to introduce any relation objects such as the arrows in Figure 1‑11.

Figure 1-11: Explicit Spatial Notation: Roofs and Frames of Houses

Especially the diagram in Figure 1-11 on the right-hand side not only seems to be overly explicit, it even seems to be misleading. On the one hand, the relation object (the arrow) indicates that the roof is above the frame. On the other hand, secondary notation [89] , in this case the relative position of the roof and the frame, suggests that the roof is below the frame. The roof, the frame, and the relation between them are not abstract. We are misled by the mismatch between our understanding of physical reality and spatial notation.

This is not to say that implicit spatial notations are intrinsically superior to explicit spatial notations. However, it suggests that:

the implicit spatial notation should be preferred over the explicit spatial notation if we are familiar with the objects and relations of a representation either from our physical experience or from our understanding of the world in terms of spatial metaphors such as orientation [64] .

The majority of visual programming languages are of a general-purpose nature. Consequently, they patronize the use of explicit spatial notations. More specifically, most of the visual programming languages belong to the class of “icons on strings” environments [71] , where the “icons” are the core objects and the “strings” the relation objects. The migration from general-purpose languages toward more task or domain-oriented languages would not only reduce the transformational distance between the problems to be solved by computer users, but it would increase the spatial density of visual programs by enabling the visual language designer to move from explicit spatial notations to implicit spatial notations.

A good example of a highly dense representation entailing an implicit spatial notation is the game of chess. The relationships between the chess pieces are very complex due to the large number of pieces that can participate in many relationships simultaneously. It would be simply overwhelming to have an explicit spatial notation such as arrows between pieces denoting all possible relationships.

The use of implicit spatial notation is so natural that we often are not even aware of the presence of a notation at all. The way we create drawing on a white board or indeed the way we write relies on implicit spatial notation. This has only very little to do with our direct physical experience. For instance, the characters of our alphabet are artifacts of our culture and not natural entities. The way we write words by stringing together individual characters horizontally from left to right (Western civilization) adheres to a very powerful implicit spatial notation. Although we have no problem reading the following sentence:

we gain very little from the flexibility of explicit spatial notations. Because we do not usually think about spatial notation in this kind of domain, the transformation from implicit to explicit spatial notation will sound somewhat complex. The semantics of words - a word is a sequence of succeeding characters - can be mapped from the currently implicit spatial notation - the character Y immediately right to the character X is the successor of X - to an explicit spatial notation - an arrow connects a character with its successor. As a result of this notational shift, we gained the ability to move the individual characters arbitraring without changing the sentence. However, it becomes immediately apparent that by giving up the Gestalt principles (a word is a horizontal string of characters) we have also lost the ability to read the sentence efficiently:

What are the implications of explicit and implicit spatial notations to the relationship of people, problems, and tools? Explicit spatial notations are best for tools that are not specific for some problem nor specific to some kind of person using it. Implicit spatial notations, on the other hand, are best suited for more problem domain-oriented applications. The implicit spatial notations are often based on either physical experience or metaphorical understanding of space. The latter may depend on the culture in which the notation was shaped. In the Arabic, Japanese, and Chinese cultures, the implicit spatial notations are different.

Object-Oriented Programming

The paradigm of object-oriented programming (OOP) has unified the formerly isolated entities representing state and behavior into a single concept called the object. Another major contribution is the support of incremental definition of state and behavior by means of inheritance at the level of programming languages. The mapping of physical entities such as buttons and windows onto the computational concept of objects seems to be more “natural” than the creation of data structures and procedures. With respect to the people, tools, and problems relationship, OOP appears to be ideal because it reduces the transformational distance between the problem and the tool.

Also, OOP as a tool claims to be more natural, but this claim regarding the “naturalness” is still debated. Despite this claim the creation of good, object-oriented code is less than trivial. The question is one of whether natural concepts are not appealing to the C programming community or if the object-oriented approach is not so natural after all.

How can useful dynamic visual environments be created with OOP? Many OOP environments come with libraries of simple graphical objects such as rectangles and buttons. However, these environment do not have an intrinsic spatial notation (implicit or explicit).

Spreadsheets

Spreadsheets support the definition of implicit spatial notations. The spatial structure of a grid can be used to represent task-specific relations. Nardi argues that spreadsheets support the incremental development of external, physical models of the problem to be solved [24]. According to her, typical spreadsheet users do not map a mental model of a solution for a problem onto the spreadsheet. Instead, the spreadsheet is an evolving artifact. As the artifact evolves so does the understanding of the problem and its solution.

Spreadsheets are task-specific but not domain-oriented. The formula language featured by spreadsheets furnishes task-specific primitives [84] such as taking averages, rounding numbers, and operating on dates. However, the notion of cells and formulas is neither domain-oriented nor domain-orientable. Cells may contain domain-oriented data or formulas; however, this domain-orientation is implicit. That is, cells may contain fragments of domain-oriented information but the models are only in the heads of the users. For instance, a cell B1 containing the formula =A1*3.14*2 could be perceived as a cell computing the circumference of a circle based on its radius stored in cell A1. We can make this induction based on the interpretation that the value 3.14 is an approximation of the constant p and, furthermore, recognizing the “radius of circle to circumference” formula. Of course, this induction may be completely wrong. For instance, the value 3.14 may not be intended to approximate p and, hence, the formula may have been utterly unrelated to the domain of circles and geometric relationships. Consequentially, the direct relation between a spreadsheet as a tool and the problem to be solved is very weak. Nonetheless, the intuitive nature of the relation between the tools and people and between people and problems has led to the big success of spreadsheets in enabling end users to create solutions to their problems.

Spatial Metaphors

Metaphors have always played an important role in the understanding of the rather abstract concept of a computer. They have helped us to comprehend and to construct new concepts related to computers by creating analogies to familiar concepts.

Files, folders, procedures, and memory, to name just a few, visible or invisible, are all metaphors of concepts with which we are already familiar. Spatial metaphors such as the desktop metaphor are enabling an increasing number of people to use computers. However, metaphors also have their problems:

• Metaphors break down: The analogy between real-world situations and metaphors are limited in scope. Resulting inconsistencies may be unavoidable, and they can confuse users.

• Metaphors are rigid: The spatial metaphors employed in current user interfaces are very rigid; that is, they are not intended to be extended by application designers or users.

• Metaphors lack domain-orientation: Spatial metaphors focus on a generic user interface part of systems and neglect the domain-specific functionality of applications.

• Metaphors are conservative: It is the very nature of a metaphor to relate the present to the past. Considering this, we must realize that metaphors lack the element of progress, and hence might engage designers too much in emulating past technology instead of creating completely new media.

Metaphors Break Down

Spatial metaphors support the transfer of skills by implying likeness or analogies to real-world situations. The transfer of skills can be positive by suggesting helpful operations to users; however, it can also be negative by implying limitations, for example, due to the physical nature of the real-world situation. For instance, the paper and pencil metaphor employed in pen-based user interfaces, on the one hand, is very powerful in re-using skills acquired by the use of physical papers and pencils. On the other hand, though, physical papers and pencils are very limited in their functionality. Moving fragments of a drawing on real paper from one location to another is a non-trivial task. At the expense of consistency a spatial metaphor may choose, and in many cases will choose, to extend or modify the meaning of the vocabulary of operations suggested by the real-world situation with a certain degree of physics-free magic. Interface designers have to be aware of resulting inconsistencies. Designers need to make trade-offs between the potential for confusion due to inconsistency and the value added by extending the metaphor.

Rigidity: How Can Users Extend Metaphors?

Spatial metaphors appear to be very rigid with respect to their extensibility by end users or application designers. Macintosh users, for instance, may change certain parameters of their desktop such as the color and pattern, or they can change the position of objects such as files and folders by dragging them; however, there is no easy way to include new behaviors. A study by Apple showed that people attempt to deal with the flow of information in their real-world workspaces by creating piles of documents. A team of Apple researchers tried to reflect this activity by extending the Macintosh desktop metaphor with a “pile” metaphor [72] . Their work indicated that adding this metaphor required a significant amount of knowledge and effort that would be way beyond the scope of an end-user or even an application designer. The question remains whether spatial metaphors are intrinsically difficult to extend or if only the implementations of spatial metaphors are difficult to extend.

Metaphors Lack Domain-Orientation

From the viewpoint of the HCI community, it seems to be desirable to completely separate the user interface and the application [88] . In terms of user interfaces, the universe of computer applications seems to be reducible to a set of interface objects such as buttons, windows, and menus. In this scheme the application independent spatial metaphor serves as an access mechanism to application specific functionality. We (the human-interface experts) believe that we have successfully reduced the complexity of creating computer systems by separating the interface from the application. Yet, if we look at real-world artifacts, such as cars, we note that most of the critical interfaces are very application-specific. For instance, a steering wheel makes perfect sense to control the direction of the car; however, it would be of little use attached to a coffee machine. The strong separation of spatial interface metaphor and application might be good from an interface designer’s point of view, but this does not necessarily imply that it also would serve users well. After all, who would like to steer a car by double-clicking some buttons?

Metaphors Are Conservative

Metaphors can be the source of conservative forces complicating the introduction of radically new concepts that cannot be explained as simple analogies of familiar concepts.

Summary: The Diversity of People and Problems Requires Substrates to Build Domain-Oriented Tools

The list of presented solution approaches (Table 1-1) is by no means as exclusive as has been presented. On the contrary, the concepts entailed in the different approaches are highly overlapping. Despite that large overlap, however, the individual user communities are largely disjoint. The challenge is to cross community borders and to reasonably combine the advantages of these approaches into a new paradigm, avoiding their disadvantages and introducing new concepts if necessary.

Table 1-1: Past Approaches

Approach

Pros

Cons

High-Level Programming Languages