1
Three-Dimensional Sketching within
an Iterative Design Workflow
A Thesis
Presented to the Faculty of the Graduate School of Cornell University
in Partial Fulfillment of the Requirements for the Degree of Master of Science
By
Nicholas Cassab-Gheta May 2019
2
© 2019 Andrés E Gutiérrez A © 2019 Nicholas Cassab-Gheta
i
ABSTRACT
Starting in the 15th century, intellectuals and theorists began developing the techniques and visual language necessary to study, understand, review, and communicate spatial concepts through drawings and models. Drawing and drafting on paper has since remained the generally preferred method of conceptual design exploration, and designers and architects have honed the necessary techniques to explore, understand, and represent three-dimensional space with this two-dimensional medium. Today, design and especially conceptual design, has evolved into a highly iterative process that usually starts with rough doodles, sketches, and other methods for expressive, quick, and inexpensive exploration before moving the work into a more precise and less forgiving environment: the computer. Digital models already have the ability to offer designers more insights through the rapid exploration of variation, even automated iteration, and powerful simulation and validation, as well as the ability to routinely and to easily generate drawings. Even still, rough physical models and drawings are widely encouraged by educators and professionals prior to entering the digital realm, and drawing on paper endures as the de facto method of choice for designers to record and develop their ideas. Our experience and research suggests that the main issues preventing designers from fully embracing digital methods as viable alternatives or even improvements on their traditional mediums, particularly in the early-phases of the design process, are the limitations of the customary input technologies and methodologies as well as the subsequent interaction and representation capabilities available with and of the inputted geometry.
Pen and multi-touch technologies have made big leaps in recent years though hardware specifications and costs still remain among the biggest hindrances; most importantly, currently affordable display surface areas are not sufficiently large or of adequate resolution for design type explorations. Our prototype 3-dimensional drawing platform was developed using a large format Microsoft Surface Hub, which we believe is the closest hardware solution currently available to an ideal high-resolution,
multi-ii
touch, drafting board. Moreover, Windows 10’s intrinsic cross-platform characteristics provide a framework for our prototype to also work on more inexpensive personal computers, such as Microsoft’s Surface Book, while the larger format displays remain cost-prohibitive.
This thesis recognizes that a viable digital alternative to drawing on paper must look and feel good and must support pen and multi-touch input. Of course, the full range of possibilities that pens, pencils, markers, and paper mediums offer designers are very difficult to accurately simulate and render digitally, especially when drawing is considered a personal experience that can vary greatly from designer to designer. While we acknowledge that our platform will not be able to provide the full range of possibilities available in expression through traditional drawing, we believe that the added benefit of a more integrated workflow will overcome some of the shortcomings in expression and demonstrate a clear path for future research and development.
Drawing plays different roles throughout the different stages of the design process and is also used to move ideas and intentions back and forth between those stages; from understanding others’ existing designs, to recording and developing one’s own ideas, to voluntarily misreading and reimagining those ideas. Trace paper or “red-lining” are popular common tools that allow designers and architects to draw and redraw over previous sketches or detailed drawings as they iterate, reference, and rethink past design decisions. This thesis demonstrates a more integrated and therefore more powerful way of drawing that recognizes the iterative nature of the design process and the apprehensions preventing designers from embracing digital drawing. By providing a flexible framework for iterative exploration within a 3-dimensional context, we demonstrate a valuable design solution and workflow that is not currently accessible by designers; one that blurs traditional distinctions between drawing and modeling.
This thesis proposes that drawing can be more powerful in 3-dimensions, while acknowledging that 3d modeling is not drawing. We propose a 3d drawing tool, built over an existing 3d modeling ecosystem, which allows designers to do drawing, but within a 3d environment. More specifically, we have developed a system that allows 2-dimensional and 3-dimensional sketching to happen within a 3d scene, one that
iii
intrinsically provides a basis to transition back and forth more fluidly and intuitively between a hand drawing and a virtual model. We feel strongly that the best solution is one that works with existing tools that designers already find comfortable and that are an essential part of their workflows and processes. We therefore built our prototype 3d drawing platform (Cuttlefish 3D) over a popular 3d modeling software ecosystem (Rhinoceros 3D) that has a powerful feature set, a low cost of entry, and a passionate and dedicated user and developer base. In this way, a working drawing can directly become a working digital model and therefore be further and fully explored and developed within powerful 3d modeling software. We further acknowledge that most continued design explorations are part of an iterative process and that any viable solution should therefore provide the designer the ability to then sketch over and within a working digital 3d model and scene.
iii
BIOGRAPHICAL SKETCH
Nicholas Cassab-Gheta was born in New York City on May 22nd 1991. He graduated from the prestigious Stuyvesant High School in 2009 and was admitted to the Bachelor’s of Architecture program at Cornell University. While an undergraduate student, Nicholas engaged in research in 3D Printing, eventually publishing a paper in the journal 3D Printing and Additive Manufacturing. He was a president of Thumbnail, an editor of The Cornell Journal of Architecture Issue 9, and an editor-in-chief of Association Volume 6. Nicholas graduated from Cornell with honors, receiving the Charles Goodwin Sands Memorial Medal for best undergraduate thesis. Nicholas has worked for numerous Architecture and Design offices in New York City, including the office of Peter Eisenman, Volley Studio, and HDR.
iv
v
ACKNOWLEDGEMENTS
We would like to acknowledge everyone who helped make this Thesis a reality. Jenny Sabin, thank you for being a wonderful advisor, teacher, and friend. I hope to live by your example, to be incredibly curious, and incredibly caring. Andrea Simitch, your leadership within the Cornell community has been one of the most inspiring things to look up to. Don Greenberg, thank you for being a maverick, and for believing in mavericks. Hurf Sheldon, thank you for providing support for the lab, both emotional and technical. Joe Kider, thank you for showing me that being a jack of all trades is possible. Martin Miller, thank you for always keeping things light, and always being able to make me laugh. Caroline O’Donnell, thank you for being my first friend at Cornell. Cindy Bowman, thank you for always being incredibly helpful. Jan Allen, thank you for putting up with our constant e-mails. Mom, thank you for listening, and for being supportive no matter what. Dad and Raluca, thank you for being there for me.
I would also like to acknowledge those who were along for the journey but are no longer with us. Taylan Cihan, you were an inspiration to us who are still trying to draw the link between sound and image. Arthur Ovaska, your kindness and your wisdom were two of the greatest gifts I’ve ever received. Kevin Pratt, you have no idea how much your example influenced and inspired my interests and my career.
Lastly, I’d like to thank the Graduate School at Cornell, and all the people behind the scenes who support a wonderful and inspiring community to be a part of.
vi
TABLE OF CONTENTS
ABSTRACT ... i
BIOGRAPHICAL SKETCH ... iii
ACKNOWLEDGEMENTS ... v TABLE OF CONTENTS... vi LIST OF FIGURES ... ix 1 OVERVIEW... 18 1.1 Overview of Implementation ... 19 1.2 Thesis Structure ... 20 2 BACKGROUND ... 22 3 RELATED WORK ... 25 3.1 2-Dimensional Software ... 26 3.1.1 Adobe Illustrator ... 26 3.1.2 Adobe Photoshop ... 29 3.1.3 Sketch ... 31
3.1.4 Made with Mischief ... 32
3.1.5 Morpholio Trace ... 35
3.2 3-Dimensional Software ... 37
3.2.1 SketchPad+ by Moreno Piccolotto and Michael Malone... 37
vii
3.2.3 T.E.D.D.Y. (Draw in 2D – Output in 3D) ... 39
3.2.4 3D Model-Assisted Sketching Interface ... 40
3.2.5 Napkin Sketch Surfacing in 3D ... 41
3.2.6 Rhonda Forever ... 42
3.2.7 Field 3D Freehand Drawing ... 43
3.2.8 I Love Sketch / Everybody Loves Sketch ... 44
3.2.9 uMake ... 45
3.2.10 Mental Canvas ... 48
3.2.11 DDDoolz ... 50
3.2.12 CATIA Natural Sketch ... 52
3.3 Other Notable Mediums ... 56
3.3.1 Sketch Furniture by FRONT ... 56
3.3.2 3-D Printing ... 57
3.3.3 Gravity Sketch ... 57
3.3.4 Tilt Brush ... 59
3.3.5 Oculus Quill and Oculus Medium ... 60
4 HUMAN COMPUTER INTERACTION ... 62
4.1 Hardware for Architectural Design ... 63
4.2 Interaction Design for Pen and Touch Interfaces ... 67
4.3 Touch Screen Devices ... 70
viii
6 CHARACTERISTICS OF A SKETCH ... 79
6.1 Understanding Drawing ... 79
6.2 Visual Reasoning: Drawing is a Language ... 81
6.3 Characteristics of a Sketch ... 87
7 ADDING A 3RD-DIMENSION TO THE SKETCH ... 89
7.1 Important Features ... 89
7.1.1 Creating geometry in an empty scene ... 90
7.1.2 Creating geometry from existing geometry ... 93
7.2 Implementation ... 97
7.2.1 Creating Surfaces. ... 99
7.2.2 Selection and Snapping. ... 99
8 RESULTS ... 101 9 FUTURE WORK ... 112 9.1 Introduction to Improvements ... 112 9.2 Improvements in Outputs ... 113 9.3 Improvements in Inputs ... 115 9.4 Visual History ... 116 9.5 Connecting to Distribution ... 118 9.6 Future of CAD ... 120 10 CONCLUSION ... 121
ix
LIST OF FIGURES
Figure 2.1 A sketch over a classic picture of the original Cornell architecture studios where architects are working with traditional methods in an empowered way through the use of modern technologies and proposed techniques. ... 22
Figure 3.1 Two screenshots of Illustrator showing the production of a graphic image through drawing at two different points in time. The screenshot on the left shows the underlying sketch work used by the artist to outline rough contours and the screenshot on the right illustrates how the artist creates final image with variable line qualities. ... 26
Figure 3.2 Screenshot of the Photoshop interface showing an .obj mesh file being painted on. On the left hand side is the unrolled mesh that allows for 2-dimensional painting on a 3-dimensional form shown on the right-hand side. Can input in either 2D or 3D space. ... 29
Figure 3.3 Example of Mischief in use. Notice the designer first drew a ghosted and rough preliminary sketch (a) before adding any detail or shading to the drawing (b). ... 32
Figure 3.4 Example of Mischief artwork by Carly Sanker demonstrating the power of ADF Technology. Here you can see the detail at different levels of zoom showing the scalability, infinite canvas, and sharp line strokes. ... 33
Figure 3.5 Marketing image of the Morpholio Trace application shows the user picking between different line thicknesses to apply over an initial sketch of a 2 dimensional representation of a 3d diagram. .. 35
Figure 3.6 Sketch planes are able to be placed to allow for input into a 3-dimensional scene. ... 37 Figure 3.7 A translucent layer resembling trace paper is used in the interface as a way for the user to toggle between drawing and camera manipulations. ... 38
Figure 3.8 Screenshots of uMake show the ability to work symmetry, the ability to edit splines with control points, and some basic surface modeling functionality... 45
x
Figure 3.9 Wireframe of uMake interface on an iPad ... 46 Figure 3.10 The screenshots above taken from a video on the Mental Canvas website show the same mental canvas sketch from two different orientations. This image describes the way strokes in the Mental Canvas system are placed on canvases arbitrarily placed in 3D. ... 48
Figure 3.11 Typical progression through CATIA using Natural Sketch. First create mood boards and organize inspiration, then initial 2-dimensional sketches, followed by 3-dimensional sketches, massing models, and finally a refined 3d model. ... 52
Figure 3.12 A view of the inspiration and ideation boards collaboratively created and used by designers in CATIA at the beginning of the design process. Notice that these boards include 3D drawings as well as images. ... 54
Figure 3.13 A view of CATIA Natural Sketch’s user interface. ... 55 Figure 3.14 On the left is an image demonstrating the process of drawing the sketches with motion capture technology and on the right are the 3d printed outcomes... 56
Figure 3.15 Above is an image depicting the original prototype of the Gravity Sketch system which features an arbitrsry physical plane and a stylus, the combination of the two allow the user to position a plane in 3d space and input points on that plane. The user here has created an outline of a pair of sunglasses using this technique. ... 57
Figure 3.16 A view of Google’s Tilt Brush, a 3d painting application for the HTC Vive ... 59 Figure 3.17 A view of Oculus Medium, a 3d sculpting application for Oculus Rift ... 60 Figure 4.1 Image of the future of Architectural Design superimposed over a historic image of Architecture students drafting in one of the first Architecture Studios at Cornell University. The image depicts touch screen interfaces of all sizes, as well as a mixed reality headset experience, with a touch screen display. ... 63
xi
Figure 4.2 Photo of a man using the 2016 version of the iPad Pro with the corresponding Apple Pencil. The device is larger than its predecessors, portable, higher resolution, and has a faster processor. The Apple Pencil has multiple sensors built in to measure tilt and pressure ... 65
Figure 4.3 Photo of a man drawing on the 2016 version of the Surface Book by Microsoft. The device has a special hinge and detachable mechanism design that allows the screen to be folded backwards over the keyboard to allow for higher processing and touch/pen interface... 66
Figure 4.4 Thumbnails of common gestures from Windows 8 app development guidelines provided by Microsoft. They show how the difference in interactions changes the look and feel and positioning of menus on screens. ... 67
Figure 4.5 Microsoft’s Universal Windows Platform is designed to easily allow developers to design and develop applications that work on devices of all sizes and interaction logic. The devices above increasing in size from the smartphone on the left to the Surface Hub on the right are all Microsoft devices. ... 68
Figure 4.6 Photo of a man using the Surface Studio by Microsoft which was released for the first time in 2016. The device is designed to be a non-portable desktop. It has more processing power, and a higher resolution and screen size. The stand for the monitor can pivot from a drawing position (shown on the left), to an upright position shown on the right. ... 70
Figure 5.1 Our application has different modes depending on which device you’re using it with. The image above depicts a diagram that shows the “breakpoints” the points in which the look and feel of the software is different depending on the screen size and hardware. ... 71
Figure 5.2 Here is an image of our application as a standalone app for a tablet device. The application is lightweight and allows for some of the functionality while being much more portable. ... 72
xii
Figure 5.3 Here is an image of our application as a Rhino Plug-in. The Rhino Plug-in version of our application is best suited for large format touch and pen displays. This is because access to a keyboard and mouse on one of these displays is relatively difficult, and so all the controls are modified to take advantage of pen and touch. ... 74
Figure 5.4 Here is an image of our application as Grasshopper plug-in. The Grasshopper plug-in version of our software is most suited when the user has a fast computer, an upright monitor and a pen input device. This allows the user to interact with the pen based interface while at the same time being able to interface with Grasshopper. ... 78
Figure 6.1 This figure shows the traditionally labor intensive drafting process that designers embraced using large format papers and tools like trace paper, rulers, and splines to create smoothly varying curves. ... 80
Figure 6.2 The drawing on the left shows iterations on trace while the image on the right shows a number of design iterations spread out along the wall in order to discuss the many different options explored. ... 83
Figure 6.3 This figure shows a quick red-lined sketch over a conceptual Google SketchUp model. The sketch provides a wealth of information impractical to communicate through normal modeling techniques. ... 84
Figure 6.4 shows a freehand sketch of the south façade of the Denver Central Library by Michael Graves ... 85
Figure 6.5 shows a drafted drawing of the south façade of the Denver Central Library by Michael Graves ... 85
Figure 7.1 Sketching on Orthogonal Planes in Virtual 3D space (a) shows a line drawing sketch on a horizontally oriented plane (grey). (b) The abstraction is augmented by sketching on the vertical plane
xiii
(grey) which has been interactively translated from the origin. (c) In a similar way, a sketched line of a second vertical plane is drawn. (This can be more easily seen in the video accompanying this submission). ... 90
Figure 7.2 shows how hand sketched drawings exist in a 3-dimensional space. An “orthogonal drawing tool” is shown here that helps the designer work with 2D sketches orthogonally-oriented within the 3D scene. Quadrant a shows this orthogonal drawing tool; quadrant b shows how the designer can select and draw on a specific plane; quadrant c shows how the designer can move the tool to a desired location within the scene and choose the desired plane to draw on; and quadrant d shows how the designer can draw on this different plane within the 3D environment. ... 91
Figure 7.3 shows how the designer can construct arbitrary 2D planes from existing geometry in the 3D scene. There are three options for doing this: Figure a shows how the designer can construct a new drawing plane by selecting a new origin for the existing plane; Figure b shows how the designer can construct an arbitrary drawing plane by specifying a new origin and a second point along the y-axis; Figure c shows how the designer can construct an arbitrary drawing plane by specifying a new origin, a second point along the y-axis, and a third point that specifies the plane’s orientation about the x axis. ... 92
Figure 7.4 Drawing on a Plane perpendicular to the Line of Sight. When the user orbits the camera, the drawing plane follows, allowing the user to draw in 3D without having to constantly shift and manipulate the drawing plane separately. All the new strokes in this mode are perpendicular to the view plane of the camera and a preset specified distance from the camera. (a) The interactive sketch is shown on the physical drawing surface. (b) This figure shows the relationship between the drawing surface and the observer location at the apex of the pyramid of vision. ... 93
Figure 7.5 This figure shows a drawing method that uses the camera to define the orientation of a temporary drawing plane and object snapping to define the temporary plane’s origin at a given z-value: quadrant
xiv
a shows the original spline drawn on the XY plane; quadrant b shows a new spline starting from one of the end points of the spline drawn in quadrant a; quadrant c shows how the spline drawn in quadrant b was created on a plane perpendicular to the camera direction with an origin at the snap point; quadrant d shows how additional splines can be made in this manner with the new spline snapping one or both endpoints to existing geometry in the scene. ... 94
Figure 7.6 shows 3 methods for surface creation: figure a shows the ability to extrude a spline to create a surface; figure b shows the ability to loft between existing splines to create a surface; and figure c shows the ability the interpolate a surface from boundary splines. ... 95
Figure 7.7 shows the ability to use an arbitrary surface as a drawing surface and to create new geometry by trimming the surface with the drawn geometry: section a shows the ability to select surfaces as drawing "planes"; section b shows a spline drawn on the chosen surface; and section c shows the resulting surface after the middle section has been trimmed with the newly-drawn spline. ... 96
Figure 7.8 shows how you can add perspective dependent drawings over existing geometry within the scene. These drawings can be layered and draw a parallel with how designers work with trace paper using traditional analogue tools. ... 97
Figure 8.1 User experiments with strokes of different colors in Cuttlefish on the large Microsoft display. ... 101
Figure 8.2 User experiments with Cuttlefish and fluidly goes back and forth between modeling with a keyboard and mouse, and drawing on the display. ... 102
Figure 8.3 In this set of figures the user is drawing using Cuttlefish and has changed the orientation of the view in order to draw a different section of the object. ... 103
Figure 8.4 This figure shows a user "red-lining" or editing on top of an existing 3D Model on a mobile Surface Book tablet. ... 103
xv
Figure 8.5 This image depicts two users collaborating on an upright Surface Hub. The user in the background has the pen and is able to make edits on the fly while the user in the foreground has enough space to point and demonstrate areas in need of improvement. ... 104
Figure 8.6 The figure above depicts a section drawing of the Elbphilharmonie in Hamburg, by Herzog & de Meuron drawn in Cuttlefish. It is shown here to demonstrate how the user can quickly sketch a drawing like this with many variations in line weights. ... 105
Figure 8.7 The figure above depicts a plan drawing of the MAXXI museum designed by Zaha Hadid Architects in Rome. The drawing was made in Cuttlefish and shows how figurative sketches and splines are quick and easy to create. ... 106
Figure 8.8 This figure shows a perspective section of the Ford Foundation Headquarters in New York City designed by Kevin Roche. This drawing was created in Cuttlefish and demonstrates how a user can quickly create a sketch from a single perspective. ... 107
Figure 8.9 This figure is an image of a Cuttlefish 3D drawing of the Eames Case Study House in LA. The drawing is a 3D drawing and can be rotated and reoriented to discover different views. ... 108
Figure 8.10 This drawing is a creative interpretation of the Seattle Public Library by OMA. The drawing starts with a detailed section of the building oriented in 3D space. The user then took this as a starting point and created simple floorplates extending out. ... 108
Figure 9.1 Many inputs many outputs. One of the reasons why our prototype is successful is because it accesses a larger array of inputs giving paths to a larger array of outputs. We believe that the most straightforward way to continue improving on this is to keep adding inputs and outputs. ... 112
Figure 9.2 The images above depict the same geometry rendered with 3 different shaders. The geometry was created using Tilt Brush and is very similar to the geometry we get when creating in our system.
xvi
This image is used to explain how shaders have an effect on the overall look and feel of a 3D scene. ... 113
Figure 9.3 The images above depict three different analytical sketches. These are 2 Dimensional sketches that our mind interprets as being in 3D. There have been many attempts to create algorithms that do the same kind of interpretation to create 3D drawings and models from 2-Dimensional sketching. 115
Figure 9.4 The images above depict an explicit history 3D Modeling system in which each iteration or phase of design of the model is stored in a visual history of the model making process. This allows for versioning, this allows for a quick assessment of multiple iterations, and it allows designers and engineers to intelligently manipulate their creations. ... 116
Figure 9.5 An artist’s depiction of a Web VR experience. Notice how the geometry is a stylized lightweight geometry with a low number of polygons. This allows users experiencing the scene with low bandwidth to enjoy the immersive presence of VR without the frame rate dropping too significantly. ... 118
Figure 9.6 An infographic depicting what Autodesk believes to be the future of CAD. There’s a large focus and emphasis on connection. The infographic explains that Building Information Modeling and CAD software will serve to connect people to the information of the project as well as the project itself. ... 120
Figure 10.1 Above is a screenshot of another software for early stage design called 123D Design by Autodesk. It’s shown here to indicate how tools for early stage design are becoming more and more user friendly because there’s a demand to make software of these capabilities accessible to people of all ages and professions... 121
Figure 10.2 This is an image of Frank Gehry’s original sketch for the Guggenheim Museum in Bilbao. It’s depicted here in order to show the importance of the flexibility and speed of sketching in the process of Early Stage Design. ... 122
xvii
Figure 10.3 An image showing 6 different view styles of the same 3d model of the Villa Savoye by Le Corbusier, depicted here to demonstrate the many uses and advantages that come from obtaining a 3D Model. This model can now be analyzed in a number of different ways and displayed in many different systems. ... 123
Figure 10.4 The image above shows an architect’s set of iterations presumably to explain how they arrived at the “best iteration described in the lower right hand corner. These types of images of an architect’s process are common in the architecture industry when critiquing early stage design. ... 124
Figure 10.5 Two side-by-side images of schematic design documents for the Morgan Library Expansion by Renzo Piano in New York. The image on the left is a plan drawing, akin to the type of drawing that would be made in our software and the image on the right is a physical model of the same space. They are shown here to demonstrate the importance of context in the architectural design process. ... 125
Figure 10.6 Using the advantages and aspects of trace paper and combining them with the 3 Dimensional needs of early stage design, we’ve created a software that is specific to architectural use. The image above is of Morpholio trace, another software created specifically for architects, it is shown here to describe the validation of these needs in the architectural profession. ... 126
18
1 OVERVIEW
In this thesis, we present a position that explains why a majority of designers in 2016, particularly architects, still choose pen and paper over computers to explore, develop, and review conceptual designs. We examine the needs of architects and designers through the lens of their processes and reveal that early-stage-design remains an area with a lot of opportunity for improvement. We recognize the importance of freehand drawing within various stages of the design process and the need for further integration of the tools used throughout and between these stages. We further acknowledge that in order to best contribute to the related fields, this thesis must embrace existing workflows and avoid “reinventing the wheel”. We do this by leveraging an existing software solution with a powerful feature set, a low cost of entry, and a passionate and dedicated user and developer base. “In this chapter, we provide an illustrated overview of the system through a sequence of images recorded during a typical design session. We follow this with a cursory introduction to the various components that make up the system and then describe the layout of the remainder of this thesis.”
19
1.1 Overview of Implementation
Our system was developed for the growing Microsoft Surface product lineup, which includes Microsoft’s Surface Hub, Surface Book, and Surface Studio models. This hardware allowed us to integrate pen and touch functionality into the architecture pipeline, which exists almost entirely within the Windows environment. Rather than attempting to independently fulfill every aspect of our vision, we focus our implementation on the areas of research that can provide the biggest impact to designers, particularly architects. Our implementation focused on improved user interaction and overall experience within existing design workflows. We chose a popular early-stage 3D design software, McNeel’s Rhinoceros 3D, as the hook into the design pipeline.
The “sketches” our users create exist in three-dimensional virtual space and are created by converting elements on arbitrarily-oriented planes and surfaces into NURBS definitions which are stored in a Rhino document. Consistent with the early stage design process, we have created our system in a way that mimics the traditional acts of and methods for drawing and sketching. Our system encourages ease of use and iterative design procedures with free-hand and pen-based sketching routines, alongside real-world metaphors, including functionality that enables multiple layers of virtual tracing paper. Although drawing is most efficiently enabled on a large, flat, horizontal or near-horizontal tilted surface, the actual “virtual tracing paper” can be rotated to enable sketching in three dimensions. In fact, using canonical solids, or even spline surfaces, the sketching need not be restricted to planar surfaces. Combining virtual layers enables the merging of components from various sketches.
Our system can be subdivided into four main parts: the core module, the input module, the rhino module, and the rendering module. The most important part of our implementation is the core module as it stores and manipulates all the geometry that is rendered through DirectX. This module also manages the current state of the application, including all input parameters (i.e. stroke type, pressure, width, color, etc.), drawing guides and planes, trace paper, and object visibility. The input module reads the strokes of the pen input as well as interactions with our GUI and calls the appropriate functions in the core module. The rhino
20
module allows the core module to communicate with and use Rhino functions such as spline and surface creation and manipulation, nearest point algorithms, and object state linking (i.e. selection). Finally, the rendering module takes data from the core module and renders it in our window with DirectX.
1.2 Thesis Structure
This thesis, the related research, and much of this writing is the result of a close collaboration between Andrés E Gutiérrez A and Nicholas Cassab-Gheta. While each author has unique parts to their research and writing, both share the majority of the work and chapters included herein. Andrés Gutiérrez focused his independent research on the important history of drawing in design, architecture, and communication and the foundation-shattering impact of being able create 3-Dimensional drawings that are experienced virtually and tangibly as spatial objects. Nicholas Cassab-Gheta focused his independent research on the potential for 3-Dimensional drawing to be extended throughout the digital landscape of existing tools available for design development; for example, alternative design tools can allow for additional input parameters from drawing input to be evaluated and modified algorithmically.
Chapter 2 is a review of the background behind the thesis. This is a description of the context, reasons, backstory behind some of the decisions made during the thesis. Chapter 3 is a comprehensive review of the existing state of software products related to Computer Aided Design, 3D Modeling, and the use of Pen and Touch Displays. Chapter 4 provides an overview of the history and current state of the art in human computer interaction, and how it relates to the difficulties behind designing a 3D Modeling interface around Pen and Touch. Chapter 5 is a description of the Software Landscape, and how that landscape changes and in effect changes the way this software is used. This software has a number of different configurations for users depending on what kind of hardware set up they have. Chapter 6 describes the characteristics of a sketch and its utility in conversation both from the standpoint of the speaker and the listener. Chapter 7 provides a detailed overview of our system. It begins with an overview of our implementation’s aspirations and the reasoning behind why certain approaches were taken. It proceeds to review our system’s core features in details and provides figures that show our system’s interface and
21
examples. The chapter ends by describing the technical details in how the features were implemented. Chapter 8 provides an overview of how we user-tested our system, the results that came out of those experiments, and a critical analysis of our system in context of the experiment and its results. Chapter 9 describes areas of future work, and potential directions for this software to explore in the future. While Chapter 10 concludes the thesis with some final takeaways about what is gained from the prototyping of a software like this.
This proposal documents the state of the art in hardware and software capabilities and puts forward a new way of interacting with digital 3D content, whether you are creating something new, exploring an idea, modifying or annotating existing content, and more. The project describes why drawing and sketching, particularly in 3D, has not yet been fully embraced by designers across different age groups, disciplines, etc. The thesis focuses on the art, the science, and the language of drawing (and sketching) as the preferred medium of exploration and communication for many designers and then describes the opportunities that are available to digital sketching both through new hardware solutions and through new software approaches.
22
2 BACKGROUND
Figure 2.1 A sketch over a classic picture of the original Cornell architecture studios where architects are working
with traditional methods in an empowered way through the use of modern technologies and proposed techniques.
From preliminary conceptual explorations to detailed specifications, the design process is currently segmented by the use of incompatible tools: traditional design methods including hand drawing and model making, software which constrains creative reasoning, and simulation technology that is expensive, difficult to use, and requires excessive specificity. With the improvements and proliferation of modern pen-input technologies and multi-touch displays (electronic drafting boards), we have the ability to maintain some of the best characteristics of hand drawing with the added benefits of computation, simulation, and 3-dimensional representation. Working hand drawings and working 3-3-dimensional digital models can now be blended and more closely integrated into a singular format that can be used to develop and communicate spatial expressions and renderings for aesthetic or functional judgements on one device. Beyond the theoretical revolutionary implications that 3-dimensional hand drawing capabilities have on our understanding of design, and more specifically the language of design (typically drawing), fully integrating
23
early explorations into systems that can automatically and continuously provide useful feedback and even suggestions through computational approaches promises to upend the way we think about and go about designing almost everything.
“To grasp the magnitude of the changes we face, it is important to realize that knowledge created with computer assistance goes well beyond classical knowledge formation—arising from computer processing of digital information resources on a scale that could not be achieved by all peoples of the
earth acting in concert using all their cognitive powers.” – Constable i
For many disciplines, mouse and keyboard technologies have been an adequate form of human computer interaction; the same is not true for design. Written languages have long provided people with a way of recording, formalizing, and communicating concepts, ideas, proposals, and more. In general, professionals like mathematicians, journalists, and businessmen have been able to integrate computers into their workflows through mouse and keyboard technologies with much less friction than designers. Where a mouse and keyboard works relatively well with languages based on text and numbers, it becomes very unintuitive and disconnected when working with graphic languages, for example drawing. Of course, the powers and benefits of computation have succeeding in persuading many designers to partake in the joys and woes of computer aided design, though we must acknowledge the histories of the disciplines and the mindsets of creative exploration, and realize that for a designer to connect with and reason through his or her work, there has to be a shift in the way he or she interacts with their designs through a computer.
“Indeed, as the next generation of photographic cameras will produce 3-D objects rather than 2-D images, and print out three-dimensional physical replications of the originals they portray, at any scale of our choosing – the primacy of two-dimensional projectival images, as we have known them
since Alberti and Gaspard Monge, will soon be challenged by new technologies.” – Neil Spiller ii
New possibilities for 3-dimensional interaction and understanding are on the horizon as a result of the ever-increasing rate of investment, development, and consequent convergence of a variety of technologies. Moore’s law has promised and delivered exponential growth in processing power for decades and graphics processors have been advancing at an even faster rate. Low cost, easily accessible, and
24
powerful sensors, cameras, flash storage, and touch screen displays have saturated markets as a result of the mobile revolution. Further increases in connectivity and communication, both interpersonal and mechanical, through continued growth in internet penetration, speed, and bandwidth continue to disrupt all aspects of modern civilization and the collection and organization of enormous amounts of data has yielded new insights, increased automation, and excitement and concern surrounding the prospects of artificial intelligence. Geometry capture technologies, propelled by Hollywood, virtual reality consumer products, and initiatives like Google’s driverless car, have already begun dominating media channels and permeating markets and new 3d printing services, consumer products, and industrial prototypes continue to emerge at alarming rates. Unsurprisingly this list goes on; the point is that while these technologies have been advancing and developing in their own ways, their convergence begins to promise an imminent 3d revolution that will encompass a number of industries and that threatens to impact fundamental paradigms established as early as the renaissance.
As design becomes further democratized and every person gains the ability to capture, adapt, and reproduce 3-dimensional forms regardless of any technical or formal training, the purposes, processes, and even business models behind design, drawing, craftsmanship, and other associated fields are brought into question. This chapter and the sections within serve to examine and explain the current disconnect between the capabilities of design technology and the needs and desires of 3-dimensional designers, particularly architects. Moreover, this chapter provides the foundational knowledge necessary to properly consider the role of drawing in the world to come.
25
3 RELATED WORK
While not perfect, many of the hardware requirements needed for 3-dimensional digital drawing with pen and multi-touch input have become available and easily accessible over the past few years. These include flat large format displays, with fast accurate multi-touch technology, and the ubiquity of processing power in the form of high powered GPU’s. As hardware options and specifications continue to improve and costs continue to decrease, there are a number of software developers that have taken advantage of the arising opportunities. Some of these developers are large corporations that have been developing graphics hardware for decades, others include startups and small teams pursuing academic research. This chapter reviews the current state of the art in related work and research and development trajectories.
There are three sections below. The first two sections are dedicated to software solutions for traditional computer hardware and tablets that take advantage of multi-touch and pen input. The related work reviewed in these first two sections is subdivided into software that is mostly intended for 2-dimensional graphics work in the first, and software mostly dedicated to 3-2-dimensional in the second. The third section in this chapter is not directly relevant to the pen and multi-touch 3d drawing implementation described in this thesis, but is nonetheless included because it offers perspective into the rapidly changing creation, interaction, and understanding of drawing.
26
3.1 2-Dimensional Software
A lot of very impressive 2-dimensional graphics software, for drawing and sketching has been created, packaged, and sold for many years. This thesis does not seek to be a replacement for these 2d platforms, but given that designers have always been trained in expressing ideas and forms through drawing in 2-dimensions it is important to understand the existing software solutions for 2-dimensional graphics. This section describes why and where designers and creative people enjoy the different existing platforms by evaluating their strengths, limitations, points of frustration, and aspirations.
3.1.1 Adobe Illustrator
Figure 3.1 Two screenshots of Illustrator showing the production of a graphic image through drawing at two different
points in time. The screenshot on the left shows the underlying sketch work used by the artist to outline rough contours and the screenshot on the right illustrates how the artist creates final image with variable line qualities.
Adobe Illustrator is a vector graphics editor currently in its seventeenth product generation. Originally developed in the late 1980s for Apple Macintosh as a ‘commercialization of Adobe’s in-house font development software’, Illustrator has grown to be the de facto vector graphics platform of choice for designers and artists working in a number of different fields.
Illustrator contains many different powerful features. This review does not attempt to provide a comprehensive evaluation of Illustrator within all the contexts that it can be used. Instead this review
27
attempts to draw focus to some of the application’s key features that we believe are worth exploring within a 3-dimensional sketching context and communicates where some of the issues may lie in moving these features into a 3-dimensional space.
While it is true that Illustrator has some 3-dimensional capabilities, and may even appear to have some modeling functionality, it has not catered to exploring 3-dimensional form and space, but rather has focused on generating the illusion of 3-dimensional objects for a 2-dimensional scene. This distinction is quite important and is evidenced in Illustrator’s inability to export 3d content in a way that other 3d applications can understand. The application does however offer three main features – rotate, revolve, and extrude/bevel – for the simulation of simple 3-dimensional models. Each one offers some perspective transformations as well as lighting and shading controls. Texture mapping is also enabled. One intended purpose of this semi-3d functionality is to allow 2-dimensional artists the ability to easily represent 3-dimensional objects within their scenes without having to draw them from scratch; consider an example where a designer creates a label for a wine bottle and wants to visualize his/her design wrapped around the bottle. The bottle and perspective distortion can be quickly and easily simulated using Illustrator’s 3-dimensional techniques.
Beyond its limited 3d functionality Illustrator offers powerful 2-dimensional vector and spline functionality. The ability to add, remove, or change control points, to choose between different spline representations, and to map and render variable widths over the length of the spline, are but a few of the many features offered. Importantly Illustrator defines most of its splines using Bezier splines and is therefore able to read and edit 2-dimensional lines, vectors, hatches, splines, and other geometry from popular CAD file types and software. As a result of this ability to open, edit, and export 2-dimensional CAD files, Illustrator is widely used by architects and other 3d artists in the production and manipulation of drawings.
In exchange for a loss of control over the creation and/or modification of 3-dimensional forms, designers working within Illustrator’s 2-dimensional ecosystem have much more control over the visual
28
representation and expression of their designs through drawing. One particular feature that outlines and greatly strengthens Illustrator’s parametric spline representation is the user’s ability to create their own “pen” or stroke styles. Through Adobe’s Create Cloud suite and mobile apps ecosystem, a user can draw a particular stroke on paper, take a photograph of it with a mobile device and an Adobe app will transform it into a usable stroke for splines and vector graphics within Illustrator.
Finally, Illustrator CC (Creative Cloud) ships with a “Touch” workspace that behaves as its own miniature-program within Illustrator and is simultaneously quite intuitive and straightforward; most people familiar with Illustrator should not have a hard time grasping the basics. It modifies the traditional user interface by removing many features and simultaneously putting forward traditional and new Illustrator features for multi-touch and pen experiences. Despite there being far fewer buttons, Illustrator’s “Touch” workspace works quite well by limiting its functionality to the key commands that are needed for drawing in the more traditional sense. From pen pressure, to larger control points and larger buttons, to undo, redo, and delete buttons, the “Touch” interface incorporates all the commands needed to work without a keyboard or mouse.
There are two features within Illustrator’s “Touch” workspace that stand out: “the Stencil tool” and a new variation on the pen tool that is called “the Curvature tool”. The Stencil tool introduces guide geometry in the form of rulers and French curves to the drawing environment irrespective of stroke styles or line representation. This guide geometry behaves exactly as one would expect in real life: there are two hit zones for multi touch that allow the user to move and scale the ruler or French curve with two fingers. As the user moves the ruler around, it can snap to existing geometry in the artboard. The second tool is the Curvature tool that allows spline drawing through the placement of individual control points. Using a pen or mouse for input, the spline previews before the user places a control point. This allows the user to dynamically preview the shape of a curve before settling on a fixed control point, rather than placing a control point to only later decide that it should be modified.
29
Also worth mentioning within the “Touch” workspace are Illustrator’s “Join tool,” that can join and fillet multiple strokes into a single spline, and Illustrator’s “Shaper tool,” that can understand and transform loose drawings into (editable) perfect geometric shapes. The “Shaper tool” further allows the user to trim strokes by scribbling on one side of an intersection of two or more strokes. This trimming behaves generatively by masking the undesired portions of the geometry definition and allowing the user to access and edit the original geometry as well as elements of the final form like fill colors and stroke types.
3.1.2 Adobe Photoshop
Figure 3.2 Screenshot of the Photoshop interface showing an .obj mesh file being painted on. On the left hand side is
the unrolled mesh that allows for 2-dimensional painting on a 3-dimensional form shown on the right-hand side. Can input in either 2D or 3D space.
Adobe Photoshop has been the industry’s de facto raster graphics editor since it was created by Thomas and John Knoll in 1988. It has a powerful layer manager that allows for the composition of raster images and has features including masking, alpha compositing, filtering, image analysis, scripting, drawing,
30
vector text and shapes, video editing, recording and saving strings of commands, and much, much more. This review, will focus on Photoshop’s 3-dimensional functionality.
Unlike Illustrator, Photoshop can create, import, and export 3d content compatible with other powerful modeling and rendering suites through common file types like .obj, .3ds, and .kmz. Included are basic 3d tools for object rotation, translation, and scale as well as some basic camera controls. Editing capabilities for 3-dimensional form in Photoshop are largely limited to placement and transformations of scene geometry and a merge objects command that begins to suggest the potential for standard Boolean operations. Photoshop also offers the ability to generate and place basic canonical forms like cones, cubes, cylinders, tori, etc. within the scene. The real value behind Photoshop’s 3d capabilities however does not lie in the creation or editing of the geometry but rather in its ability to create textures and other 2-dimensional graphic images like renderings. In this way, Photoshop leverages its core strengths and maintains its focus on the creation, editing, and compositing of raster images.
To allow the mix of different types of content within the same interface Photoshop stores any 3d content created or imported as a 3d layer within its powerful layer manager. For users familiar with Photoshop’s more traditional feature set, 3d layers are different and yet comfortably similar to normal layers. They have a thumbnail, a description, and effects in their traditional locations and styles. Normal operations like opacity or blending changes are also applied to 3d layers in the same way. The magic behind Photoshop’s 3d functionality becomes apparent when you realize that you can paint and apply rendering and lighting effects directly on the 3d models. In Figure 3.2 we can see two views of the same object, a hat. On the left is a flattened mesh of the hat that updates in real time if you draw on the model; of course if you draw on the texture map the hat will update. In this way users can paint detailed textures on their models for rendering or use in other software. Alternatively, users that prefer compositing and creating realistic images in Photoshop could do it all without ever leaving the software.
31 3.1.3
Sketch
Sketch is a lightweight and quite powerful 2-dimensional vector-based application for Mac. It is a very common tool amongst designers who are building digital products and who need to block out, outline and design user interfaces. It is strongly targeted at graphic, interface, and experience designers who design computer or web applications and thereby restricts what might traditionally be considered ‘sketching’ in other areas of design. Rather than focusing traditional tools such as a pen or a pencil, Sketch provides a number of simple vector-based features that work very well for a specific purpose. It focuses on manipulating basic shapes and fonts through transformations, colors, opacities, shadows, and so on. Sketch also has a fairly powerful layer manager, more like Photoshop than Illustrator, where each ‘object’ is in its own layer and layers can be grouped into folders. Like Illustrator, Sketch has artboards which can be scaled or sized as the user decides, however unlike Illustrator, Sketch is much more lightweight. It can supports larger workspaces, to the point that one might consider the project space an ‘infinite canvas’ within which there can exist a substantial amount of artboards.
Of course, given the lack of pen input, and given the general scope and interest of this thesis, Sketch is not an existing solution to the problem we are exploring. That being said, we include Sketch as part of this research because we believe there is great value in parts of the solution they offer that could inform and potentially justify our decisions behind the product. This is particularly true in the interaction between different artboards that are within a project’s infinite canvas, a characteristic which is quite useful in practice. It should also be noted that the lightweight nature of the stored geometry allows for a very responsive experience, for example zooming in and out of different designs and explorations. This responsiveness of scale and variety allows the exploration of alternate designs simultaneously and has proven crucial to Sketch’s success.
It will be interesting to see how Sketch, a leading user-interface design application, evolves. As it stands, as a two dimensional drawing application with limited functionality, it works really well in a specific context. As that context itself evolves, and as user interaction design shifts into 3-dimensional space with
32
technologies like virtual and augmented reality, it will be interesting to follow how very targeted applications like Sketch adapt and incorporate the third dimension.
3.1.4 Made with Mischief
Figure 3.3 Example of Mischief in use. Notice the designer first drew a ghosted and rough preliminary sketch (a)
before adding any detail or shading to the drawing (b).
Mischief is a 2-dimensinoal drawing application for Windows and Mac OS X that was spun out of research by two Scientists at Mitsubishi Electric Research Labs (MERL), Dr. Sarah Frisken and Dr. Ronald Perry. The technology behind Mischief is protected by over 50 patents, and in late 2014, the company 61 Solutions along with the product and the intellectual property portfolio MERL was acquired by the Foundry, who specializes in creating high-end software for the visual effects industry.
What Mischief set out to do, and has managed to do quite well, is to provide a powerful 2-dimensional drawing application that has the natural feel and control of raster graphics without losing the scalability of vector graphics. The user is able to draw very intuitively, with pen and touch input, and interact with their artwork in a similar fashion as you might expect from a pixel based editor. The differentiating factors are the application’s infinite canvas and revolutionary stroke representation that allow designers to scale or zoom without running out of space, without aliasing problems, without loss of edge definition, and all while maintaining small file sizes.
33
“ADFs are a new digital representation of shape which provide numerous advantages including high-quality anti-aliasing, very fast rendering, very small file sizes, multi-scale rendering, support for massive parallelism, and the ability to succinctly represent variable-width, scalable, textured strokes.”
– Made with Mischief iii
Dr. Frisken and Dr. Perry each hold dozens of patents in Adaptively Sampled Distance Fields (ADF) Technology. ADFs are the magic behind Mischief and they make the program feel more like a lightweight app than the powerful program it is. In 2016, the software retailed at a very affordable $25 for the full version and there is a free version with limited features also available to all users. Moreover, the installation is quick and straightforward and the software only takes up 7MB on your hard drive; it really feels more like a lightweight app than a full application. From a usability standpoint, working with ADFs enables very small file sizes and fast response times while working with a zoom factor of 50 trillion-to-1. Neither pixels nor vectors, Frisken describes working with this revolutionary shape representation is “like sitting on the moon and looking at a single flower on Earth and then drawing on a petal of that flower”.iv
Figure 3.4 Example of Mischief artwork by Carly Sanker demonstrating the power of ADF Technology. Here you
34
One of the major benefits of using ADF Technology, beyond the technology itself, is some of the tangential functionality that it allows; the most important of which is a truly responsive infinite canvas, one of Mischief’s core strengths. While 3d modeling environments have traditionally employed this infinite workspace concept, 2-dimensional programs generally tend to more closely follow the paper or screen analogy where borders are part of the constraints. Removing the borders and losing a sense of scale provides a certain freedom of exploration that is more difficult to achieve in traditional software applications. Mischief puts forward an unconventional software approach where the scale of the output, to either paper or to a 2-dimensional image, comes after the artwork is created. This flexibility enables different connections between the artist and the digital artwork - usually more experiential, more emotional, and less technical or formal.
This sketchy, in some ways non-technical, and scale-less approach is what sets Mischief apart, and the flexibility and power of ADF technology is what allows it. More than simply being able to represent strokes sharply throughout a large range of scales, ADFs allow a certain expression of emotion and intent that traditional vector representations struggle with. In their press release The Foundry proclaims that ADF Technology “can accurately represent much richer and more complex shapes than traditional vector-based stroke representation.”vi Pen pressure, pen angle, line and fill opacities, gradients, erasing, etc. are interacted with in a more natural way than with other vector-based applications, in some ways blurring the lines between vector and raster graphics. This results in an experience that an artist can more easily relate to and begin to easily use, as they might in the real world on pen and paper, or even in other raster based graphics drawing/painting applications. The last major hugely positive differencing feature Mischief offers falls in line with this approach. They leverage existing practices within drawing-based professions, such as the use of translucent trace paper, and enable the functionality within the interface. In other words, the entire application can be made translucent; this has very large implications and applications for designers and artist in practice, something easily seen in public feedback about Mischief online.
35
“From the very start that was very much one of our passions – Mischief should be easy to use. You don’t need a hundred menus; you don’t need a manual. You just open it and you start to draw.” –
Friskenvii
Despite all of these innovations and all Mischief’s promise, the application is currently still restricted to drawing in two dimensions. Oddly enough, the hypothetical leap in to 3d should not be a technical hurdle for the Mischief team. The technology that drives Mischief (Adaptively Sampled Distance Fields, or ADF) was created for 3-dimensional medical imaging, and given that most of The Foundry’s services are for 3d content, it is not a stretch to see some of this technology finding its way into the 3d sketching market. The question of implementation and how to use ADFs to draw in 3 dimensions remains.
3.1.5 Morpholio Trace
Figure 3.5 Marketing image of the Morpholio Trace application shows the user picking between different line
36
Morpholio Trace is part of a suite of applications that have been developed for the iPad that cater specifically to designers and architects. Trace is specifically designed to mimic the user experience that is familiar and comfortable to designers who use trace to iterate and revise. The app is a fully functional drawing application that allows the user the added control of manipulating a layer of trace on top of an existing image or drawing. The user has the added freedom of being able to manipulate multiple layers of trace in one drawing session. The user can rotate, and translate using multi-touch gestures that are common to the iPad interface, and they can use a pinch gesture to zoom in or out of their drawings.
The latest version of Morpholio Trace introduced yet another physical metaphor or hallmark of Architectural drawing, the technical pen. This tool harkens back to tools that were originally used to draw starting as early as the 1960’s. The appeal being that you can now control different line-weights and line thicknesses in your drawings in order to convey more information. The application also automatically adapts line-weights based on the magnification level of the drawing.
37
3.2 3-Dimensional Software
This section reviews software that has been developed specifically for 3d content. There are a variety of different projects with different scales and budgets, some are academic research proposals and others are extremely powerful and exclusive software solutions. We include here the software that has had the greatest influence on our understanding and aspirations for 3d drawing using multi-touch and pen input technologies.
3.2.1 SketchPad+ by Moreno Piccolotto and Michael Malone
Figure 3.6 Sketch planes are able to be placed to allow for input into a 3-dimensional scene.
We include this early proposal by two Master of Science graduates of the Program of Computer Graphics at Cornell University. The thinking and objectives behind this thesis have greatly influenced the continued work and research in 3d drawing by students and alumni of Cornell’s Program of Computer Graphics. Unfortunately, at the time this thesis was presented in 1998, adequate hardware was simply not available to realize the full extent of their vision. That being said, Malone’s and Piccolotto’s research offers insights into some of the hesitations that some architects have with computer aided design and begins to suggest strategies that can offer designers the best of both worlds. We have expanded on many of these insights in Chapter 2 of this thesis.
38
3.2.2 Collaborative Design Platform – Violin Yanev Thesis
Figure 3.7 A translucent layer resembling trace paper is used in the interface as a way for the user to toggle between
drawing and camera manipulations. viii
Inspired by a 1999 paper (Urp: A Luminous-Tangible Workbench for Urban Planning and Design) written by John Underkoffler and Hiroshi Ishii out of MIT Media Lab’s Tangible Media Group, researchers in the Technical University of Munich formed a group in 2011 called the Collaborative Design Platform Research Group. The group is led by research director Dr.-Ing. Gerhard Schubert and is supported by the department chair for “architectural computer science” (Architektural Informatics) Dr.-Ing. Frank Petzold and the department chair for augmented reality Prof. Gudrun Klinker, Ph.D. The group’s mission is to “resolve the current discrepancy between familiar, analogue ways of working in the early architectural design stages and the ever increasing use of digital tools in office practice.” ix
Violin Yanev is currently a software engineer at BMW in Munich. In 2012, Yanev graduated with a Master of Science in computer science from the CDP group in the Technical University of Munich; his master’s thesis, a 3d sketching application aimed at architectural design, is titled “3D virtuality sketching: a freehand sketch tool for conceptual urban design in architecture.” The application allows for traditional camera controls within a 3d modeling environment and uses a trace paper metaphor to allow for functional differentiation between basic 3d sketching and traditional 3d modeling interactions. Figure 3.7 shows a layer of trace paper can be overlaid on the viewport in order to enable sketch input and edit sketch geometry.
39
3.2.3 T.E.D.D.Y. (Draw in 2D – Output in 3D)
Figure 3.8 In the above image, the user of T.E.D.D.Y. demonstrates how a 2 Dimensional sketch in one orientation
becomes a 3 Dimensional object in a different orientation, through algorithms that interpret the 2 non-planar strokes
as descriptors of a 3D object. x
T.E.D.D.Y. was a drawing application that appeared out of the research group at the University of Tokyo, and the Tokyo Institute of technology. The application allowed for the creation of freeform meshes that would be rendered using line strokes and automatically shaded depending on orientation. The software proved to be intuitive for the creation of arbitrary 3D Objects. Our interest in T.E.D.D.Y. is in the choice of representation of the 3D object, in which the mesh is rendered as a set of outlines and shaded in order to look like a pencil drawing. This software is also notable for its interface which assumes the user is performing 3D modeling operations based on the position and relationship of the stroke and the existing geometry.
In T.E.D.D.Y. the user is working on the silhouette of an object. By starting with this assumption, T.E.D.D.Y.’s creators are able to interpret strokes in relation to the silhouette as strokes that are performing “cutting” operations, or “creation” operations. The user starts by outlining a silhouette, and the software then interprets the silhouette in to a smooth shape that creates that silhouette. If a stroke is drawn across the silhouette of an object, the computer interprets that as a cutting operation and trims the original object.
40
3.2.4 3D Model-Assisted Sketching Interface
Figure 3.9 The image above demonstrates how a crude sketch generated by the user, selects the most similar 3D
object from a known database. In this case, the rouch outline of a horse, is replaced by a more accurate 3d Model of a
horse, at the same location and scale. xi
This work is from 2012 and is included here because it demonstrates a very interesting implementation in which hand drawn input is being recognized by a computer and is then used to present relevant information from a database of 3d models. In this scenario, a rough shape is drawn and the interface retrieves a set of similar 3d models that can be imported. The application was developed for the goal of helping the user draw by providing the ability to reference and even trace over an external 3d model within a 3d scene. Beyond the ability to import custom guide and reference geometry from an external library, the significance of providing relevant information and even suggestions as a computer analyzes sketched geometry are particularly interesting.
Limitations of the above system occur when there are no pre-existing samples for what the user is attempting to create. This means the software is limited to 3D object “selection”, over 3D Object creation. Even in the sample above, if a user wanted to create the model of a horse rearing its head, the system would have to rely on previously containing that specific 3D model, and even then the horse may still not be in the position the user intended.
41
3.2.5 Napkin Sketch Surfacing in 3D
Figure 3.10 The image above shows how a “trim” operation is performed on a surface model in the aforementioned
software. The user draws a stroke near the edge of the surface and the computer system removes the portion of the
surface on one side of the new stroke. xii
Napkin Sketch Surfacing in 3D is a pen-based 3D modeling platform that interprets sketches as 3-Dimensional surface geometry. The interface allows for the intuitive creation of freeform surfaces through sketch based input. The system automatically interprets 2-Dimensional pen-stroke inputs in relation to the existing surface geometry in the scene and from this information it is able to modify the surface geometry to more appropriately align its silhouette with the latest stroke input in the system. Not much else is known about the software included above, but it is included in this chapter for its ability to interpret sketch input in order to create surface geometry that could potentially be used in a traditional 3D Modeling environment.
42
3.2.6 Rhonda Forever
Figure 3.11 Two screenshots of James Paterson, one of the contributors to the Rhonda Forever project, are shown
drawing a teddy bear within Rhonda. xiii
Rhonda Forever is a 3d drawing tool originally developed by Amit Pitaru around 2003 and has been contributed to by four others over the next six years. Around 2009 or 2010, development for Rhonda stopped after being shown in a number of galleries, museums, festivals, and conferences. Rhonda works in a straightforward and easily understandable way. There is always a drawing plane parallel to the screen and the user uses pen input to draw on the drawing plane. The camera controls allow the user to orbit and pan, and intersection points between geometry and the drawing plane parallel to the screen are highlighted with red dots. At any point in time the user can continue to input line geometry with the stylus or pen. One issue with this approach is the lack of control intuitively available to the user in establishing precise drawing planes for accurately drawing between two specific arbitrary points.
