Andre Hypermedie Domæner

(1)

Andre Hypermedie Domæner

2D-spatiale hypermedier

Context-aware hypermedier

Frank Allan Hansen

[email protected]

Aarhus Universitet

(2)

2D Spatial hypermedier

✦

Rationale & Definition

✦

Auqanet, VIKI, and VKB

(3)

(4)

Rationale: Hvorfor Spatial?

(5)

✦

Hvad er hypermedier?

-

Navigationelle hypermedier:

Nodes, links, and anchors,…

-

Vannevar Bush (1945): machine

support for associative memory

-

Ted Nelson (1984): non-sequential

forms of writing involving links.

-

Doug Engelbart (1984):

[Augment’s links were] textual

citations to some specific file item.

-

Conklin (1987): [Links are] the

essential feature of hypertext

(6)

✦

Navigationelle hypermedier:

Relationer mellem informationsobjekter

repræsenteret som eksplicitte links

-

Men når man arbejder med store informationsmængder er disse

relationer måske ikke kendt på forhånd

Halasz seven issues: premature sorting

-

Halasz (1991):

[nodes] explicitly or implicitly organized into one or more

structures

(7)

5

✦

Hvad er spatiale hypermedier?

-

Man arbejder stadig med struktur

-

Ikke repræsenteret med eksplicitte links (link-less), men rumligt organiseret

-

Spatial placering, former, farver og andre visuelle virkemidler

-

Maskin-understøttelse for struktur

✦

Gør det nemmere at arbejde med tvetydige, ufuldkomne og

“emergent” strukturer

✦

Simpel interaktion og ad-hoc organisering

✦

Udnytter mennesket spatiale forståelse og hukommelse

✦

Sorterings fase (første fase)

(8)

Spatial organisering af materialer

✦

Materialer ser forskellige ude

-

Farve og form kan hjælpe med at huske materialer

✦

Placering opretholder sammenhæng mellem materialer

✦

Lokation, lokation, lokation!

(9)

7

Informations Analyse

✦

Eksplorativ fortolkning og analyse af digitalt materiale

-

Ingen formalisme

-

Sortering, prioritering, analysering, meningsdannelse

-

Eks:

•

Efterretning

•

Forskning

The Central Intelligence Agency

(CIA) is a civilian intelligence

agency of the United States

government. Its primary function

is collecting and analyzing

information about foreign

governments, corporations, and

persons in order to advise public

policymakers.

(10)

(11)

9

Spatial hypermedier

✦

Repræsentation frem for navigering

-

Informationen er jo nødvendigvis ikke struktureret på

forhånd

✦

Oprettelse af struktur fremfor læsning

-

Uformel interaktion

-

Nemt at afprøve ting og re-arrangere materialer

-

Personligt struktureringsværktøj;

•

ikke forfatterværktøj eller præsentationsværktøj

✦

Visuel struktur

-

Rumlig placering (relationer)

(12)

Hypermedie Domæner

✦

The niche for spatial hypermedia

2D canvas

Single/multiple Document window(s)

No abtractions, types, regularities

Abstractions on demand

User-defined abstractions

System defined abstractions,

(Eg. Argumentation models)

Liberal

(13)

Eksempel på en argumentation efter

Toulmins model

(14)

Hypermedie Domæner

✦

The niche for spatial hypermedia

2D canvas

Single/multiple Document window(s)

No abtractions, types, regularities

Abstractions on demand

User-defined abstractions

System defined abstractions,

(Eg. Argumentation models)

Liberal

(15)

✦

(16)

(17)

15

(18)

Spatiale Strukturer

✦

Lists

-

Lister af ens objekter

✦

Stacks

-

Overlappende stakke af ens objekter

✦

Composites

-

Ens arrangementer af forskellig objekter

✦

Heaps

(19)

VIKI - data model

✦

Objekter

-

Indholdsholdene objekter (Dexter Atomic Components)

-

Hvert objekt refereres med et eller flere visuelle symboler.

-

Deklarering af pre-definerede typer ikke nødvendigt

•

Typen angives med den visuelle formatering (form, farve, osv)

✦

Collections

-

System-understøttede hierarkier

•

Kan indeholde en vilkårlig spatial arrangement af objekter eller andre

collections

-

Visuelt symbol: Scrollbare 2½D rum

✦

Composites

-

Kombination af to eller flere objekter eller collections i en bestemt

(20)

(21)

19

Spatial Support

(22)

Spatial Parser

✦

Click selection

✦

Enable

interaction

with implicit

structures

✦

Structures

need not be

explicitly

defined

✦

Works much

like text

selection in

text editors

(23)

Spatial Parser

✦

Collection suggestion

-

“Bridge the gap”

between user activities

and data model,

•

Suggest collections

based on

subdivision of

material created by

the user

-

User can iterate

through suggestions

•

Collections are

created and

(24)

✦

Composite suggestion

-

New Abstraction and

structures on demand

•

Based on recurring

spatial patterns

•

Enables user-accessible

composites types, defined

•

From parser

•

Modified from

parser suggestion

•

From scratch

-

Composite types can be

applied to material

(25)

Visual Knowledge Builder (VKB) -

Koncept og idé

✦

VKB – efterfølger til VIKI

-

Beregnet til at organisere og fortolke information fra

forskellige kilder

-

F.eks. fra desktop og WWW

-

Anvendelse:

•

Skrivning af relateret arbejde i afhandling eller artikel

•

kategorisering af information

(26)

Visual Knowledge Builder

What to do...

- start by dragging and dropping items

from the desktop, web pages, and

images into your personal or group

information space.

- then move the visual symbols into

clusters, lists, or other structures.

- categorize symbols by changing

visual attributes like color, border

color, border width, font, etc.

- as your information space enlarges,

put symbols into a hierarchy of

(27)

✦

(28)

CAOS - Maﬆer's Thesis, AU

(29)

iMapping

Heiko Haller and Andreas Abecker.

2010. iMapping: a zooming user

interface approach for personal and

semantic knowledge management.

iMapping:

a graphical approach to semi-structured knowledge modelling

This is an iMap

about iMapping

This is an

annotation

author of

works at

iMapping Examples

Nepomuk

Semantic MediaWiki

Projects

Max

Denny

Markus

Heiko

Goals

Partners

Nepomuk WorkPackages

WP4

Search

WP2

Application Integration

WP3

Processes

WP5

2p2

WP6

Architecture

WP++

Case Studies Visualisation / UI WP1 WP1 members Knowledge Articulation FZI Andreas Abecker Max VölkelHeiko Haller

KTH

Yngve Sundblad Bosse Westerlund

Rosa Gutjohnsdottir Semantic Wiki

Max Völkel

Conceptual data Structures

Heiko Haller

MAyG iMapping

MouseHouseDogCatDaddyMumCharlyFarmFarmerBulldozerHorseSpiderPigOfficeComputerPrinterPapersBossBuildingsStablesearch-filterType: AnimalMouseDogCatHorseSpider

iMapping SearchiMapping Examples

Levels Of Detail

Hide & Progressive Disclosure

Focus & Context

iMapping Design Principles

Visual Information Seeking Mantra

• overview first • zoom and filter • details on demand

7 Tasks of Information Visualisation

details-on-demandoverviewzoomfilterrelatehistoryextract

iMapping related work

Mapping Techniques

Mind-MapsConcept MapsSpatial HypertextKnowledge Mapsothermethodsetc ...LiteratureDesigning the User InterfaceThe Eyes Have It: Beyond the PlaneUsability HeuristicsZoomable User Interfaces

tools

Using Vision to Think

Website

Levels

Of

Detail

Hide &

Progressive

Disclosure

Focus &

Context

iMapping Design Principles

Visual Information

Seeking Mantra

•

overview first

•

zoom and filter

•

details on demand

7 Tasks of

Information

Visualisation

details-on-demand

overview

zoom

filter

relate

history

extract

iMapping related work

Mapping Techniques

Mind-Maps

Concept Maps

Spatial Hypertext

Knowledge Maps

other methods etc ...

Literature

Designing the User Interface

The Eyes Have It:

Beyond the Plane

Usability Heuristics

Zoomable User Interfaces

tools

iMapping

CS-TR-3665 July 1996 ISR-TR-96-66

The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations

Ben Shneiderman Department of Computer Science Human-Computer Interaction Laboratory,and Institute for Systems Research University of Maryland, College Park, Maryland 20742 USA [email protected], http://www/cs.umd.edu/projects/hcil/

Abstract

A useful starting point for designing advanced graphical user interfaces is the Visual Information-Seeking Mantra: Overview first, zoom and filter, then details-on-demand. But this is only a starting point in trying to understand the rich and varied set of information visualizations that have been proposed in recent years. This paper offers a task by data type taxonomy with seven data types (1-, 2-, 3-dimensional data, temporal and multi-dimensional data, and tree and network data) and seven tasks (overview, zoom, filter, details-on-demand, relate, history, and extract).

!"#$%!&$ !"#$%&$'&(%$#%$)$*&)(+,$-)(&.%/01.(2.0*3)(/4(&$'&(/%(/&.$%(5$,+-+*&$%2/**$2&$,(6"(%$7-&+/*)8(&"#+2-77"(*-9+1-&+/*-7(7+*3):(;.$ )+5+7-%+&"(&/(3*/<7$,1$(%$#%$)$*&-&+/*)()02.(-)(4%-5$)(-*, )$5-*&+2(*$&)(.-)(7$,(&/(502.($44/%&(+*(0)+*1(."#$%&$'&()")&$5)(4/% 3*/<7$,1$(%$#%$)$*&-&+/*(-*,($'&$*,+*1(."#$%&$'&()")&$5)(&/ 5-3$(&.$5(-67$(&/($'#%$))(5/%$:(;.+)(</%3(.-)(5$&(<+&.(7+5+&$, )022$))(,0$(&/(,+44+207&+$)(+*270,+*1(&.$(&-2+&(-*,()+&0-&$,(*-&0%$(/4 502.(3*/<7$,1$:(=*)&$-,(/4(9+$<+*1(3*/<7$,1$($'#%$))+/*(-)(-* -77(-&(/*2$($9$*&8(<$(9+$<(+&(-)(-(2/*)&%02&+9$(#%/2$))8(+:$: 3*/<7$,1$(60+7,+*1:(;.$(>+)0-7(?*/<7$,1$(@0+7,$%(A>?@B(7$&) 0)$%)($'#%$))(2/*&$*&(9+-(9+)0-7(/%(&$'&0-7(5$-*)(-*,(7-&$% 4/%5-7+C$(&.-&(2/*&$*&(+*(&.$(4/%5(/4(-&&%+60&$)8(9-70$)8(&"#$)8(-*, %$7-&+/*):(>?@(#%/-2&+9$7"()0##/%&)(&.+)(#%/2$))(&.%/01.(-()$&(/4 )011$)&+/*(-1$*&)(<./)$(+*&$%-2&+/*(<+&.(&.$(0)$%(+)(5$,+-&$,(6" &.$()011$)&+/*(5-*-1$%:(D%$7+5+*-%"($9-70-&+/*(/4(&.$()011$)&+/* 5-*-1$%(-*,()011$)&+/*(-1$*&)("+$7,)(#/)+&+9$(%$)07&)(60&(40%&.$% 2/*4+%5)(&.-&(&.$%$(+)(*/(E)+79$%(6077$&F(4/%(3*/<7$,1$($*1+*$$%+*1 GG()$5-*&+2($'#%$))+/*(+)(5/)&(7+3$7"(&/(.-##$*(,0%+*18(-*,(+) ,%+9$*(6"8(&-)3(#$%4/%5-*2$: &'()*+,-)./'01/#234)5(/6).5,-7(+,. !:H:I(J809+,:'(-+0/80(),9'5)./'01/;,).)0('(-+0KL(!"#$%&$'&M !"#$%5$,+-(G(!"#$%&'#&("')*+()'"+%))(') <)0),'=/$),:. N$)+1*8(!05-*(O-2&/%) >)?@+,1. )#-&+-7(."#$%&$'&8()011$)&+/*G6-)$,(+*&$%4-2$)8(5+'$,G+*+&+-&+9$ ,+-7/1)8(9+)0-7(7-*10-1$8()#-&+-7(#-%)$%8(+*2%$5$*&-7(4/%5-7+C-&+/* AB//CD;E%$EF$/!G6/>GHIJE6<E/ %E;%E#EG$!$8HG !"#$%&$'&(.-)(7/*1(6$$*(%$2/1*+C$,(-)(-(4/%5(/4(3*/<7$,1$ %$#%$)$*&-&+/*($5#.-)+C+*1(+*&$%,/205$*&(%$7-&+/*):(=*(-,,+&+/*(&/ 5/%$(2/55/*(#-1$G)+C$,(*/,$)(4/0*,(+*()")&$5)(7+3$(?PQ(JRK -*,(&.$(S$68(-(*056$%(/4()")&$5)(.-9$(+*270,$,()5-77$%G)+C$, +*4/%5-&+/*(2.0*3)(&/(%$#%$)$*&(4+*$%G1%-+*$,(%$7-&+/*):(T-%7" $'-5#7$)(+*270,$(&.$(%$#%$)$*&-&+/*(/4(Q$-%7$U)(V.+*$)$(W//5 -%105$*&(+*(T027+,(JXYK8(;/075+*G)&%02&0%$,(-*-7")$)(+* Z/&$V-%,)(JR[K8(-*,(=@=QG6-)$,(,$)+1*(%-&+/*-7$(+*(P+3%/#7+)(J\\K -*,(1=@=Q(JIK: ;.+)(4+%)&(%/0*,(/4(%$#%$)$*&-&+/*-7(."#$%&$'&(.$7#$,(5/&+9-&$(&.$ ,$)+1*( /4( -( *056$%( /4( )")&$5)( <+&.( 5/%$( $'#%$))+9$ %$#%$)$*&-&+/*):(=NT(JRHK($'&$*,$,(Z/&$V-%,)(&/(+*270,$(-&',-.!&'+#!"/(4/%(,$4+*+*1(&"#$)(&.%/01.(&.$(+*.$%+&-*2$(/4 -&&%+60&$)(-*,(6$.-9+/%(<+&.+*(&.$(2/*&$'&(/4(+*)&%02&+/*-7(,$)+1*: QDW=Z;(JXK8(<+&.(-*(-+5(&/($*.-*2+*1(60)+*$))(#%-2&+2$8(0)$, Q5-77;-73(5$&./,)(-&&-2.$,(&/(-(4%-5$G6-)$,(%$#%$)$*&-&+/*(&/ +*&$1%-&$(."#$%5$,+-8()$5-*&+2(*$&</%38(-*,($'#$%&()")&$5 %$#%$)$*&-&+/*):(V/*2/%,$(JRRK()0##/%&$,(3*/<7$,1$($*1+*$$%+*1 &.%/01.(-(%$#%$)$*&-&+/*(+*(<.+2.(%$7-&+/*)(6$&<$$*(*/,$)(2/07, 6$(2/*)&%-+*$,:(?*/<7$,1$()&%02&0%+*1(<-)(&.$(1/-7(/4(]^0-*$& JR_K8(<.+2.(-77/<$,(0)$%)(&/(,$4+*$(%$#%$)$*&-&+/*-7()2.$5-)(&.-& +*270,$,(/6`$2&(-*,(%$7-&+/*(&"#$)8(-&&%+60&$)8(-*,(2/*)&%-+*&):(;.$ -6/9$(%$#%$)$*&-&+/*)($5#.-)+C$(&.$($'#%$))+/*(/4(,$27-%-&+9$ 3*/<7$,1$:(D.+,+-)(J\XK($56$,,$,(#%/2$,0%-7(3*/<7$,1$(+*(&.$ ."#$%&$'&(%$#%$)$*&-&+/*8()02.(-)(+*4$%$*2$)(6-)$,(/*(4/%<-%,G 2.-+*+*18(6"(-77/<+*1(*/,$)(+*(+&)(."#$%6-)$(&/(2/*&-+*(-()&/%$, ^0$%":(]77(/4(&.$)$()")&$5)(67$*,$,(%$#%$)$*&-&+/*)(4%/5 ."#$%&$'&(-*,(-%&+4+2+-7(+*&$77+1$*2$:(O0%&.$%(,+)20))+/*)(/4(&.$ ,$)+1*(-*,(0)$(/4()02.()")&$5)(-%$(4/0*,(+*(JRaK8(JXRK8(-*,(J\HK: S/%3(/*(3*/<7$,1$G6-)$,(."#$%&$'&(6$2-5$(7$))(2/55/*(<+&. &.$($5$%1$*2$(/4(&.$(S$6:(b*$(%$-)/*(+)(&.-&(&.$(S$6U)(#%+5-%" %$#%$)$*&-&+/*)8($:1:(!;Pc8(,+,(*/&(+*270,$(0)$407(4-2+7+&+$)(4/% 5/%$(4/%5-7(%$#%$)$*&-&+/*:(P/%$(%$2$*&()&-*,-%,)(.-9$(2.-*1$, &.-&:(=*,$$,8(5-*"(/4(&.$(-6/9$(&.$5$)(-%$(6$+*1(%$9+)+&$,(+* ,+)20))+/*)(/4(&.$(EQ$5-*&+2(S$6:F(dPc(+*270,$)(5-*"(/4(&.$ 2.-%-2&$%+)&+2)(/4($-%7+$%($44/%&)(<+&.(%$1-%,)(&/(+*&$1%-&+*1(4%-5$G 6-)$,( %$#%$)$*&-&+/*)( -*,( ."#$%&$'&:( c+3$( ]^0-*$&U) %$#%$)$*&-&+/*8(+&(+)(-(5$&-G7-*10-1$(&.-&(2-*(6$(0)$,(&/($*2/,$ )#$2+4+2(3*/<7$,1$(%$#%$)$*&-&+/*(7-*10-1$):(;.$(W$)/0%2$ N$)2%+#&+/*(O%-5$</%3(AWNOB(-*,(N]Pceb=c(4/77/<(&.$ 3*/<7$,1$(+*&$%2.-*1$(4/%5-&(A?=OB(-*,(3*/<7$,1$(^0$%"(-*, 5-*+#07-&+/*(7-*10-1$(A?fPcB(-)()&-*,-%,)(4/%().-%+*1(2/55/* #%/2$,0%-7(%$#%$)$*&-&+/*)(/4(3*/<7$,1$: ;.$)$(-%$(-77(%$#%$)$*&-&+/*)(,$)+1*$,(&/(-+,(+*(&.$(0)$(-*,().-%+*1 /4(3*/<7$,1$(/*2$(+&(+)(%$#%$)$*&$,8(60&(./<(&.$(3*/<7$,1$(+) E-0&./%$,F(+*(&.$(4+%)&(#7-2$(+)(*/&(2/*)+,$%$,:(b0%(4/20)(+)(/*(./< &/()0##/%&(&.+)($'#%$))+/*8(<.+2.(<$(2-77(0123.'/4'+5(%./%146(;.+) +)(-(2/*)&%02&+9$(-2&+9+&"(<.$%$(&.$(-0&./%U)(/<*(3*/<7$,1$(+) +5#-2&$,(6"(&.$($'#%$))+/*(#%/2$)):(@$)+,$)(#%/9+,+*1(9+)0-78 )#-&+-7(-*,(&$'&0-7(5$-*)(/4(2/550*+2-&+/*8(<$(-%$(60+7,+*1 #%/-2&+9$()0##/%&(4/%(&.+)(#%/2$)): ;.$(*$'&()$2&+/*(,+)20))$)(,+44+207&+$)(<+&.(3*/<7$,1$ %$#%$)$*&-&+/*(-*,(+5#7+2-&+/*)(4/%(,$9$7/#+*1(&//7)(&/(.$7#

!"#$%&'()+$,,"%-.%/01"23"45'12'%3'%!,$&'$1*+6,"7&"8&

!"#$%&'()*+#$,&-.&/)0(#12&/33"1,&4"115#+&/#233",&6#371)&68)1(,&9#:(;&<%%#*1==) >1*#"5+1$5&3?&@3+*;51"&'0)1$01&#$=&@1$51"&?3"&5(1&'5;=A&3?&>):)5#2&B)C"#")18 D1E#8&<F/&G$)H1"8)5A @3221:1&'5#5)3$,&DI&JJKLMNMOOP&G'< QO&RJR&KSP&MPOS 8()*+#$T08.5#+;.1=; D$%5+))+/*(&/(5-3$(,+1+&-7(/%(.-%,(2/#+$)(/4(-77(/%(#-%&(/4(&.+)(</%3(4/% #$%)/*-7(/%(27-))%//5(0)$(+)(1%-*&$,(<+&./0&(4$$(#%/9+,$,(&.-&(2/#+$)(-%$ */&(5-,$(/%(,+)&%+60&$,(4/%(#%/4+&(/%(2/55$%2+-7(-,9-*&-1$(-*,(&.-&(2/#+$) 6$-%(&.+)(*/&+2$(-*,(&.$(4077(2+&-&+/*(/*(&.$(4+%)&(#-1$:(;/(2/#"(/&.$%<+)$8(/% %$#067+).8(&/(#/)&(/*()$%9$%)(/%(&/(%$,+)&%+60&$(&/(7+)&)8(%$^0+%$)(#%+/% )#$2+4+2(#$%5+))+/*(-*,M/%(-(4$$:( 789:;8(g0*$(RRGRH8(\hh\8(V/77$1$(D-%38(P-%"7-*,8(iQ]:( V/#"%+1.&(\hh\(]VP(RGH[RRXGIYYGhMh\Mhhha:::jH:hh:( 25 !"#$%!&$ !"#$%&$'&(%$#%$)$*&)(+,$-)(&.%/01.(2.0*3)(/4(&$'&(/%(/&.$%(5$,+-+*&$%2/**$2&$,(6"(%$7-&+/*)8(&"#+2-77"(*-9+1-&+/*-7(7+*3):(;.$ )+5+7-%+&"(&/(3*/<7$,1$(%$#%$)$*&-&+/*)()02.(-)(4%-5$)(-*, )$5-*&+2(*$&)(.-)(7$,(&/(502.($44/%&(+*(0)+*1(."#$%&$'&()")&$5)(4/% 3*/<7$,1$(%$#%$)$*&-&+/*(-*,($'&$*,+*1(."#$%&$'&()")&$5)(&/ 5-3$(&.$5(-67$(&/($'#%$))(5/%$:(;.+)(</%3(.-)(5$&(<+&.(7+5+&$, )022$))(,0$(&/(,+44+207&+$)(+*270,+*1(&.$(&-2+&(-*,()+&0-&$,(*-&0%$(/4 502.(3*/<7$,1$:(=*)&$-,(/4(9+$<+*1(3*/<7$,1$($'#%$))+/*(-)(-* -77(-&(/*2$($9$*&8(<$(9+$<(+&(-)(-(2/*)&%02&+9$(#%/2$))8(+:$: 3*/<7$,1$(60+7,+*1:(;.$(>+)0-7(?*/<7$,1$(@0+7,$%(A>?@B(7$&) 0)$%)($'#%$))(2/*&$*&(9+-(9+)0-7(/%(&$'&0-7(5$-*)(-*,(7-&$% 4/%5-7+C$(&.-&(2/*&$*&(+*(&.$(4/%5(/4(-&&%+60&$)8(9-70$)8(&"#$)8(-*, %$7-&+/*):(>?@(#%/-2&+9$7"()0##/%&)(&.+)(#%/2$))(&.%/01.(-()$&(/4 )011$)&+/*(-1$*&)(<./)$(+*&$%-2&+/*(<+&.(&.$(0)$%(+)(5$,+-&$,(6" &.$()011$)&+/*(5-*-1$%:(D%$7+5+*-%"($9-70-&+/*(/4(&.$()011$)&+/* 5-*-1$%(-*,()011$)&+/*(-1$*&)("+$7,)(#/)+&+9$(%$)07&)(60&(40%&.$% 2/*4+%5)(&.-&(&.$%$(+)(*/(E)+79$%(6077$&F(4/%(3*/<7$,1$($*1+*$$%+*1 GG()$5-*&+2($'#%$))+/*(+)(5/)&(7+3$7"(&/(.-##$*(,0%+*18(-*,(+) ,%+9$*(6"8(&-)3(#$%4/%5-*2$: &'()*+,-)./'01/#234)5(/6).5,-7(+,. !:H:I(J809+,:'(-+0/80(),9'5)./'01/;,).)0('(-+0KL(!"#$%&$'&M !"#$%5$,+-(G(!"#$%&'#&("')*+()'"+%))(') <)0),'=/$),:. N$)+1*8(!05-*(O-2&/%) >)?@+,1. )#-&+-7(."#$%&$'&8()011$)&+/*G6-)$,(+*&$%4-2$)8(5+'$,G+*+&+-&+9$ ,+-7/1)8(9+)0-7(7-*10-1$8()#-&+-7(#-%)$%8(+*2%$5$*&-7(4/%5-7+C-&+/* AB//CD;E%$EF$/!G6/>GHIJE6<E/ %E;%E#EG$!$8HG !"#$%&$'&(.-)(7/*1(6$$*(%$2/1*+C$,(-)(-(4/%5(/4(3*/<7$,1$ %$#%$)$*&-&+/*($5#.-)+C+*1(+*&$%,/205$*&(%$7-&+/*):(=*(-,,+&+/*(&/ 5/%$(2/55/*(#-1$G)+C$,(*/,$)(4/0*,(+*()")&$5)(7+3$(?PQ(JRK -*,(&.$(S$68(-(*056$%(/4()")&$5)(.-9$(+*270,$,()5-77$%G)+C$, +*4/%5-&+/*(2.0*3)(&/(%$#%$)$*&(4+*$%G1%-+*$,(%$7-&+/*):(T-%7" $'-5#7$)(+*270,$(&.$(%$#%$)$*&-&+/*(/4(Q$-%7$U)(V.+*$)$(W//5 -%105$*&(+*(T027+,(JXYK8(;/075+*G)&%02&0%$,(-*-7")$)(+* Z/&$V-%,)(JR[K8(-*,(=@=QG6-)$,(,$)+1*(%-&+/*-7$(+*(P+3%/#7+)(J\\K -*,(1=@=Q(JIK: ;.+)(4+%)&(%/0*,(/4(%$#%$)$*&-&+/*-7(."#$%&$'&(.$7#$,(5/&+9-&$(&.$ ,$)+1*( /4( -( *056$%( /4( )")&$5)( <+&.( 5/%$( $'#%$))+9$ %$#%$)$*&-&+/*):(=NT(JRHK($'&$*,$,(Z/&$V-%,)(&/(+*270,$(-&',-.!&'+#!"/(4/%(,$4+*+*1(&"#$)(&.%/01.(&.$(+*.$%+&-*2$(/4 -&&%+60&$)(-*,(6$.-9+/%(<+&.+*(&.$(2/*&$'&(/4(+*)&%02&+/*-7(,$)+1*: QDW=Z;(JXK8(<+&.(-*(-+5(&/($*.-*2+*1(60)+*$))(#%-2&+2$8(0)$, Q5-77;-73(5$&./,)(-&&-2.$,(&/(-(4%-5$G6-)$,(%$#%$)$*&-&+/*(&/ +*&$1%-&$(."#$%5$,+-8()$5-*&+2(*$&</%38(-*,($'#$%&()")&$5 %$#%$)$*&-&+/*):(V/*2/%,$(JRRK()0##/%&$,(3*/<7$,1$($*1+*$$%+*1 &.%/01.(-(%$#%$)$*&-&+/*(+*(<.+2.(%$7-&+/*)(6$&<$$*(*/,$)(2/07, 6$(2/*)&%-+*$,:(?*/<7$,1$()&%02&0%+*1(<-)(&.$(1/-7(/4(]^0-*$& JR_K8(<.+2.(-77/<$,(0)$%)(&/(,$4+*$(%$#%$)$*&-&+/*-7()2.$5-)(&.-& +*270,$,(/6`$2&(-*,(%$7-&+/*(&"#$)8(-&&%+60&$)8(-*,(2/*)&%-+*&):(;.$ -6/9$(%$#%$)$*&-&+/*)($5#.-)+C$(&.$($'#%$))+/*(/4(,$27-%-&+9$ 3*/<7$,1$:(D.+,+-)(J\XK($56$,,$,(#%/2$,0%-7(3*/<7$,1$(+*(&.$ ."#$%&$'&(%$#%$)$*&-&+/*8()02.(-)(+*4$%$*2$)(6-)$,(/*(4/%<-%,G 2.-+*+*18(6"(-77/<+*1(*/,$)(+*(+&)(."#$%6-)$(&/(2/*&-+*(-()&/%$, ^0$%":(]77(/4(&.$)$()")&$5)(67$*,$,(%$#%$)$*&-&+/*)(4%/5 ."#$%&$'&(-*,(-%&+4+2+-7(+*&$77+1$*2$:(O0%&.$%(,+)20))+/*)(/4(&.$ ,$)+1*(-*,(0)$(/4()02.()")&$5)(-%$(4/0*,(+*(JRaK8(JXRK8(-*,(J\HK: S/%3(/*(3*/<7$,1$G6-)$,(."#$%&$'&(6$2-5$(7$))(2/55/*(<+&. &.$($5$%1$*2$(/4(&.$(S$6:(b*$(%$-)/*(+)(&.-&(&.$(S$6U)(#%+5-%" %$#%$)$*&-&+/*)8($:1:(!;Pc8(,+,(*/&(+*270,$(0)$407(4-2+7+&+$)(4/% 5/%$(4/%5-7(%$#%$)$*&-&+/*:(P/%$(%$2$*&()&-*,-%,)(.-9$(2.-*1$, &.-&:(=*,$$,8(5-*"(/4(&.$(-6/9$(&.$5$)(-%$(6$+*1(%$9+)+&$,(+* ,+)20))+/*)(/4(&.$(EQ$5-*&+2(S$6:F(dPc(+*270,$)(5-*"(/4(&.$ 2.-%-2&$%+)&+2)(/4($-%7+$%($44/%&)(<+&.(%$1-%,)(&/(+*&$1%-&+*1(4%-5$G 6-)$,( %$#%$)$*&-&+/*)( -*,( ."#$%&$'&:( c+3$( ]^0-*$&U) %$#%$)$*&-&+/*8(+&(+)(-(5$&-G7-*10-1$(&.-&(2-*(6$(0)$,(&/($*2/,$ )#$2+4+2(3*/<7$,1$(%$#%$)$*&-&+/*(7-*10-1$):(;.$(W$)/0%2$ N$)2%+#&+/*(O%-5$</%3(AWNOB(-*,(N]Pceb=c(4/77/<(&.$ 3*/<7$,1$(+*&$%2.-*1$(4/%5-&(A?=OB(-*,(3*/<7$,1$(^0$%"(-*, 5-*+#07-&+/*(7-*10-1$(A?fPcB(-)()&-*,-%,)(4/%().-%+*1(2/55/* #%/2$,0%-7(%$#%$)$*&-&+/*)(/4(3*/<7$,1$: ;.$)$(-%$(-77(%$#%$)$*&-&+/*)(,$)+1*$,(&/(-+,(+*(&.$(0)$(-*,().-%+*1 /4(3*/<7$,1$(/*2$(+&(+)(%$#%$)$*&$,8(60&(./<(&.$(3*/<7$,1$(+) E-0&./%$,F(+*(&.$(4+%)&(#7-2$(+)(*/&(2/*)+,$%$,:(b0%(4/20)(+)(/*(./< &/()0##/%&(&.+)($'#%$))+/*8(<.+2.(<$(2-77(0123.'/4'+5(%./%146(;.+) +)(-(2/*)&%02&+9$(-2&+9+&"(<.$%$(&.$(-0&./%U)(/<*(3*/<7$,1$(+) +5#-2&$,(6"(&.$($'#%$))+/*(#%/2$)):(@$)+,$)(#%/9+,+*1(9+)0-78 )#-&+-7(-*,(&$'&0-7(5$-*)(/4(2/550*+2-&+/*8(<$(-%$(60+7,+*1 #%/-2&+9$()0##/%&(4/%(&.+)(#%/2$)): ;.$(*$'&()$2&+/*(,+)20))$)(,+44+207&+$)(<+&.(3*/<7$,1$ %$#%$)$*&-&+/*(-*,(+5#7+2-&+/*)(4/%(,$9$7/#+*1(&//7)(&/(.$7# !"#$%&'()*+$,,"%-.%/01"23"*45'12'%3*'%*!,$&'$1*+6,"7&"8& !"#$%&'()*+#$,&-.&/)0(#12&/33"1,&4"115#+&/#233",&6#371)&68)1(,&9#:(;&<%%#*1==)>1*#"5+1$5&3?&@3+*;51"&'0)1$01&#$=&@1$51"&?3"&5(1&'5;=A&3?&>):)5#2&B)C"#")18 D1E#8&<F/&G$)H1"8)5A @3221:1&'5#5)3$,&DI&JJKLMNMOOP&G'< QO&RJR&KSP&MPOS 8()*+#$T08.5#+;.1=; D$%5+))+/*(&/(5-3$(,+1+&-7(/%(.-%,(2/#+$)(/4(-77(/%(#-%&(/4(&.+)(</%3(4/% #$%)/*-7(/%(27-))%//5(0)$(+)(1%-*&$,(<+&./0&(4$$(#%/9+,$,(&.-&(2/#+$)(-%$ */&(5-,$(/%(,+)&%+60&$,(4/%(#%/4+&(/%(2/55$%2+-7(-,9-*&-1$(-*,(&.-&(2/#+$) 6$-%(&.+)(*/&+2$(-*,(&.$(4077(2+&-&+/*(/*(&.$(4+%)&(#-1$:(;/(2/#"(/&.$%<+)$8(/% %$#067+).8(&/(#/)&(/*()$%9$%)(/%(&/(%$,+)&%+60&$(&/(7+)&)8(%$^0+%$)(#%+/% )#$2+4+2(#$%5+))+/*(-*,M/%(-(4$$:( 789:;8(g0*$(RRGRH8(\hh\8(V/77$1$(D-%38(P-%"7-*,8(iQ]:( V/#"%+1.&(\hh\(]VP(RGH[RRXGIYYGhMh\Mhhha:::jH:hh:( 25

Toolkit Design for

Interactive Structured Graphics

Benjamin B. Bederson, Jesse Grosjean, and Jon Meyer

Abstract—In this paper, we analyze toolkit designs for building graphical applications with rich user interfaces, comparing polylithic and monolithic toolkit-based solutions. Polylithic toolkits encourage extension by composition and follow a design philosophy similar to 3D scene graphs supported by toolkits including Java3D and OpenInventor. Monolithic toolkits, on the other hand, encourage extension by inheritance, and are more akin to 2D Graphical User Interface toolkits such as Swing or MFC. We describe Jazz (a polylithic toolkit) and Piccolo (a monolithic toolkit), each of which we built to support interactive 2D structured graphics applications in general, and Zoomable User Interface applications in particular. We examine the trade offs of each approach in terms of performance, memory requirements, and programmability. We conclude that a polylithic approach is most suitable for toolkit builders, visual design software where code is automatically generated, and application builders where there is much customization of the toolkit. Correspondingly, we find that monolithic approaches appear to be best for application builders where there is not much customization of the toolkit.

Index Terms—Monolithic toolkits, polylithic toolkits, object-oriented design, composition, inheritance, Zoomable User Interfaces (ZUIs), animation, structured graphics, Graphical User Interfaces (GUIs), Pad++, Jazz, Piccolo.

! 1 INTRODUCTION

A

PPLICATIONdevelopers rely on User Interface (UI) toolkits such as Microsoft’s MFC and .NET Windows Forms, and Sun’s Swing and AWT to create visual user interfaces. However, while these toolkits are effective for traditional widget-based applications, they fall short when the developer needs to build a new kind of user interface component-one that is not bundled with the toolkit. These components might be simple widgets, such as a range slider or more complex objects, including interactive graphs and charts, sophisticated data displays, timeline editors, zoom-able user interfaces, or fisheye visualizations.

Developing application-specific components usually requires significant quantities of custom code to manage a range of features, many of which are similar from one component to the next. These include managing which areas of the window need repainting (called region manage-ment), repainting those regions efficiently, sending events to the internal object that is under the mouse pointer, managing multiple views, and integrating with the under-lying windowing system.

Writing this code is cumbersome, yet most standard 2D UI toolkits provide only rudimentary support for creating custom components—typically, just a set of methods for drawing 2D shapes and methods for listening to low-level events.

Some toolkits such as Tcl/Tk [19] include a “structured canvas” component, which supports basic structured

graphics. These canvases typically contain a collection of graphical 2D objects, including shapes, text, and images. These components could in principal be used to create application-specific components. However, structured can-vases are designed primarily to display graphical data, not to support new kinds of interaction components. Thus, for example, they usually do not allow the application to extend the set of objects that can be placed within the canvas. We have found that many developers bypass these structured canvas components and follow a “roll-your-own” design philosophy, rewriting large quantities of code and increasing engineering overhead, particularly in terms of reliability and programmability. There are also commer-cial toolkits available such as Flash [6] and Adobe SVG Viewer [2]. But, these approaches are often difficult to extend and integrate into an application.

We believe future user interface toolkits must address these problems by providing higher-level libraries for supporting custom interface components. However, there is still an open question regarding which design philosophy to adopt for these higher-level toolkits. The core issue we address here is whether toolkits should be designed so that the inevitable complexity and extension of the components are supported primarily through composition (which we call polylithic) or inheritance (which we call monolithic). In this paper, we consider these two design approaches for interactive structured graphics toolkits through two toolkits we built: Jazz,1 a polylithic toolkit; and Piccolo,2 a IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 30, NO. 8, AUGUST 2004 1

. The authors are with the Human-Computer Interaction Laboratory,Institute for Advanced Computer Studies, Computer Science Department,University of Maryland, College Park, MD 20742. E-mail: {bederson, jesse, meyer}@cs.umd.edu. Manuscript received 16 Sept. 2003; accepted 16 Mar. 2004. Recommended for acceptance by D. Weiss. For information on obtaining reprints of this article, please send e-mail to: [email protected], and reference IEEECS Log Number TSE-0145-0903.

1. The name Jazz is not an acronym, but rather is motivated by the music-related naming conventions that the Java Swing toolkit started. In addition, the letter “J” signifies the Java connection, and the letter “Z” signifies the zooming connection. Jazz is open source software according to the Mozilla Public License, and is available at: http://www.cs.umd.edu/ hcil/jazz. 2. The name Piccolo is motivated by the music connection of Jazz and Swing, and because it is so small (approximately one tenth the size of Jazz). Piccolo is open source software according to the Mozilla Public License, and is available at: http://www.cs.umd.edu/hcil/piccolo. 0098-5589/04/$20.00! 2004 IEEEPublished by the IEEE Computer Society

Mouse House Dog Cat Daddy Mum Charly FarmFarmer Bulldozer Horse Spider Pig Office Computer Printer Papers Boss Buildings Stable search-filterType: Animal Mouse Dog Cat Horse Spider

iMapping Search

iMapping Examples

superordinate Node new Text...

when [enter] is pressed while editing a node, editing is anded and a Textcursor is set to the position below the node - where a new paragraph would start.

entering text

iMapping core principles

•

spatial layout

•

nesting, zooming

•

graphical authoring

•

allow vague structures

•

capture implicit semantics

•

minimize cognitive overhead

•

use Conceptual Data Structures

Abstract

iMapping is a technique for visually structuring information

objects. It supports the full range from informal note taking over

semi-structured personal information management to formal

knowledge models. With iMaps, users can easily go from

overview to fine-grained structures while browsing editing or

refining the knowledge base in one comprehensive view.

An iMap is comparable to a large white-board where information

items can be positioned like post-its but also nested into each

other. Spatial browsing and zooming as well as graphical editing

facilities make it easy to structure content in an intuitive way.

iMapping builds on a zooming user interface approach to

facilitate navigation and to help users maintain an overview in

the knowledge space. While a first implementation is being

developed, iMapping is still in a conceptual stage. In this paper

we describe the iMapping approach and how it tries to combine

and extend the advantages of other approaches.

Conceptual Data Structures

relation

annotation

tag

type

context/detail

subclass

part of

order

CDS is a lightweight top-level ontology about relations

that naturally occur in common knowledge artefacts.

It is designed to bridge the gap between unstructured

content like informal notes and formal semantics like

ontologies by allowing the use of vague semantics and

by subsuming arbitrary relation types under more

general ones.

MaxVölkel

ignorable PopUps

DenkWerkzeug2006_CDS [event]

DennyVrandecic

participant_of

organiser_of

met_at

MaxVölkel

HeikoHaller

is developed in

display search results

Mouse

House

Dog

Cat

Daddy

Mum

Charly

Farm

Farmer

Bulldozer

Horse

Spider

Pig

Office

Computer

Printer

Papers

Boss

Buildings

Stable

search-filter

Type: Animal

Mouse

Dog

Cat

Horse

Spider

SurroundingWikiPage

SomePeople

Rudi

Heiko

Max

NepomukProject

knows

wo

rks

_fo

r

wo

rks

_fo

r

Max

Eyal

NepoWorkPackages

Heiko Haller