Fundamental Data Structures

2

Fundamental

Data Structures

17 February 2021

Sebastian Wild

Outline

2

Fundamental Data Structures

2.1 Stacks & Queues 2.2 Resizable Arrays 2.3 Priority Queues 2.4 Binary Search Trees 2.5 Ordered Symbol Tables 2.6 Balanced BSTs

Abstract Data Types

abstract data type (ADT)

I list of supported operations

I whatshould happen

I not: how to do it

I not: how to store data

≈ _Javainterface

(with Javadoc comments)

vs.

data structures

I specify exactly

how data is represented

I algorithms for operations

I has concrete costs

(space and running time)

≈ _Javaclass

(non abstract)

Why separate?

I Can swap out implementations “drop-in replacements”)

reusable code!

I (Often) better abstractions

Stacks

Stack ADT

I top()

Return the topmost item on the stack Does not modify the stack.

I push(𝑥)

Add𝑥onto the top of the stack.

I pop()

Remove the topmost item from the stack (and return it).

I isEmpty()

Returnstrueiff stack is empty.

I create()

Linked-list implementation for Stack

Invariants:

I maintaintoppointer to topmost element

I each element points to the element below it

(ornullif bottommost)

Linked stacks:

I requireΘ(𝑛)space when𝑛elements on stack

Array-based implementation for Stack

Can we avoid extra space for pointers? array-based implementation

Invariants:

I maintain arraySof elements, from bottommost to topmost

I maintain indextopof position of topmost element inS.

What to do if stack is full uponpop?

Array stacks:

I requirefixed capacity𝐶(known at creation time)!

I requireΘ(𝐶)space for a capacity of𝐶elements

Digression – Arrays as ADT

Arrays can also be seen as an ADT! . . . but are commonly seen as specific data structure

Array operations:

I create(𝑛) Java:A = new int[𝑛];

Create a new array with𝑛cells, with positions0, 1, . . . , 𝑛 − 1

I get(𝑖) Java:A[𝑖]

Return the content of cell𝑖

I set(𝑖,𝑥) Java:A[𝑖] = 𝑥;

Set the content of cell𝑖to𝑥.

Arrays have fixed size (supplied at creation).

Usually directly implemented by compiler + operating system / virtual machine.

Difference to others ADTs: Implementation usually fixed to “a contiguous chunk of memory”.

Doubling trick

Can we have unbounded stacks based on arrays? Yes!

Invariants:

I maintain arraySof elements, from bottommost to topmost

I maintain indextopof position of topmost element inS

I maintain capacity𝐶 =S.lengthso that1₄𝐶 ≤ 𝑛 ≤ 𝐶

can always push more elements!

How to maintain the last invariant?

I beforepush

If𝑛 = 𝐶, allocate new array of size2𝑛, copy all elements.

I afterpop

If𝑛 < 1₄𝐶, allocate new array of size2𝑛, copy all elements.

“Resizing Arrays

an implementation technique, not an ADT!

Amortized Analysis

I Any individual operationpush/popcan be expensive!

Θ(𝑛)time to copy all elements to new array.

I But: An one expensive operation of cost𝑇meansΩ(𝑇)next operations are cheap!

Formally:consider “credits/potential”

distance to boundary

Φ =min{𝑛 − 1₄𝐶, 𝐶 − 𝑛} ∈ [0, 0.6𝑛]

I amortized cost of an operation = actual cost (array accesses) − 4·_{change inΦ}

I cheappush/pop: actual cost1array access, consumes≤ 1credits amortized cost≤ 5

I copyingpush: actual cost2𝑛 + 1array accesses, creates1₂𝑛 + 1credits amortized cost≤ 5

I copyingpop: actual cost2𝑛 + 1array accesses, creates1₂𝑛 − 1credits amortized cost5

sequenceof𝑚operations: total actual cost≤_{total amortized cost}+_{final credits}

Queues

Operations:

I enqueue(𝑥)

Add𝑥at the end of the queue.

I dequeue()

Remove item at the front of the queue and return it.

Bags

What do Stack and Queue have in common?

They are special cases of aBag!

Operations:

I insert(𝑥)

Add𝑥to the items in the bag.

I delAny()

Remove any one item from the bag and return it. (Not specified which; any choice is fine.)

I roughly similar to Java’sCollection

Sometimes it is useful to state that order is irrelevant Bag

Priority Queue ADT – min-oriented version

Now: elements in the bag have differentpriorities.

(Max-oriented) Priority Queue (MaxPQ):

I construct(𝐴)

Construct from from elements in array𝐴.

I insert(𝑥,𝑝)

Insert item𝑥with priority𝑝into PQ.

I max()

Return item with largest priority.(Does not modify the PQ.)

I delMax()

Remove the item with largest priority and return it.

I changeKey(𝑥,𝑝0)

Update𝑥’s priority to𝑝0.

Sometimes restricted to increasing priority.

I isEmpty()

PQ implementations

Elementary implementations

I unordered list Θ(1)insert, butΘ(𝑛)delMax

I sorted list Θ(1)delMax, butΘ(𝑛)insert

Can we get something between these extremes? Like a “slightly sorted” list? Yes! Binary heaps.

Array view

Heap = array𝐴with

∀𝑖 ∈ [𝑛] : 𝐴[b𝑖/2c] ≥ 𝐴[𝑖]

≡

store nodes in level order

in𝐴[1..𝑛]

Tree view

Heap = tree that is

(i)a complete binary tree

all but last level full last level flush left

(ii)heap ordered

Why heap-shaped trees?

Why complete binary tree shape?

I only one possible tree shape keep it simple!

I complete binary trees have minimal height among all binary trees

I simple formulas for moving from a node to parent or children:

For a node at index𝑘in𝐴

I parent atb𝑘/2c

I left child at2𝑘

I right child at2𝑘 + 1

Why heap ordered?

I Maximum must be at root! max()is trivial!

I But: Sorted only along paths of the tree; leaves lots of leeway for fast

how? . . . stay tuned

Analysis

Height of binary heaps:

I heightof a tree: # edges on longest root-to-leaf path

I depth/levelof a node: # edges from root root has depth0

I How many nodes on first𝑘fulllevels?

𝑘 Õ

ℓ =0

2ℓ =2𝑘+1− 1

Height of binary heap: ℎ = min𝑘s.t.2𝑘+1− 1 ≥ 𝑛 = blg(𝑛)c

Analysis:

I insert: new element “swims” up ≤ ℎ_{steps (}ℎ_cmps)

I delMax: last element “sinks” down ≤ ℎ_{steps (2ℎcmps)}

I constructfrom𝑛elements:

cost = cost of letting each node in heap sink!

≤ 1 · ℎ + 2 · (ℎ − 1) + 4 · (ℎ − 2) + · · · + 2ℓ · (ℎ − ℓ ) + · · · + 2ℎ−1_{· 1 + 2}ℎ_{· 0} = ℎ Õ ℓ =0 2ℓ(ℎ − ℓ ) = ℎ Õ 𝑖=0 2ℎ 2𝑖𝑖 = 2 ℎ ℎ Õ 𝑖=0 𝑖 2𝑖 ≤ 2 · 2 ℎ ≤ 4𝑛

Binary heap summary

Operation Running Time

construct(𝐴[1..𝑛]) 𝑂(𝑛) max() 𝑂(1) insert(𝑥,𝑝) 𝑂(log 𝑛) delMax() 𝑂(log 𝑛) changeKey(𝑥,𝑝0) 𝑂(log 𝑛) isEmpty() 𝑂(1) size() 𝑂(1)

Symbol table ADT

Symbol table / Dictionary / Map

Java:java.util.Map<K,V>

/ Associative array / key-value store:

I put(𝑘,𝑣) Pythondict:d[𝑘] = 𝑣

Put key-value pair(𝑘, 𝑣)into table

I get(𝑘) Pythondict:d[𝑘]

Return value associated with key𝑘

I delete(𝑘)

Remove key𝑘(any associated value) form table

I contains(𝑘)

Returns whether the table has a value for key𝑘

I isEmpty(),size()

I create()

Most fundamental building block in computer science.

Symbol tables vs mathematical functions

I similar interface

I but: mathematical functions are static (never change their mapping)

(Different mapping is a different function)

I symbol table = dynamicmapping

Elementary implementations

Unordered (linked) list:

Fastput

Θ(𝑛)time forget

Too slow to be useful

Sortedlinked list:

Θ(𝑛)time forput

Θ(𝑛)time forget

Too slow to be useful

Binary search

It does help . . . if we have a sorted array! Example:search for69

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 11 12 17 28 35 55 57 63 69 77 79 80 82 85 88 97 ℓ 𝑚 𝑟 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 11 12 17 28 35 55 57 63 69 77 79 80 82 85 88 97 ℓ 𝑚 𝑟 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 11 12 17 28 35 55 57 63 69 77 79 80 82 85 88 97 ℓ 𝑚 𝑟 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 11 12 17 28 35 55 57 63 69 77 79 80 82 85 88 97 ℓ Binary search: I halve ±1 remaining list in each step

≤ blg 𝑛c + 1cmps in the worst case

Binary search trees

Binary search trees (BSTs) ≈ _{dynamic sorted array}

I binary tree

I Each node has left and right child

I Either can be empty (null)

I Keys satisfy search-tree property

BST example & find

BST insert

BST delete

I Easy case: remove leaf, e. g.,11 replace bynull

I Medium case: remove unary, e. g.,69 replace by unique child

I Hard case: remove binary, e. g.,85 swap with predecessor, recurse

11 12 17 28 35 55 57 63 69 77 79 80 82 85 97 88 11 12 17 28 35 55 57 63 69 77 79 80 72 85 88 97

BST summary

Operation Running Time

construct(𝐴[1..𝑛]) 𝑂(𝑛 ℎ) put(𝑘,𝑣) 𝑂(ℎ) get(𝑘) 𝑂(ℎ) delete(𝑘) 𝑂(ℎ) contains(𝑘) 𝑂(ℎ) isEmpty() 𝑂(1) size() 𝑂(1)

Ordered symbol tables

I min(),max()

Return the smallest resp. largest key in the ST

I floor(𝑥), b𝑥c =ℤ.floor(𝑥)

Return largest key𝑘in ST with𝑘 ≤ 𝑥.

I ceiling(𝑥)

Return smallest key𝑘in ST with𝑘 ≥ 𝑥.

I rank(𝑥)

Return the number of keys𝑘in ST𝑘 < 𝑥.

I select(𝑖)

Return the𝑖th smallest key in ST (zero-based, i. e.,𝑖 ∈ [0..𝑛))

With select, we can simulate access as in a truly dynamic array!.

Augmented BSTs

Rank

Select

Balanced BSTs

Balanced binary search trees:

I imposes shape invariant that guarantees𝑂(log 𝑛)height

I adds rules to restore invariant after updates

I many examples known

I AVL trees(height-balanced trees)

I red-black trees

I weight-balanced trees(BB[𝛼]trees)

I . . .

Other options:

I amortization: splay trees, scapegoat trees

I’d love to talk more about all of these . . . (Maybe another time)

BSTs vs. Heaps

Balanced binary search tree

Operation Running Time

construct(𝐴[1..𝑛]) 𝑂(𝑛 log 𝑛) put(𝑘,𝑣) 𝑂(log 𝑛) get(𝑘) 𝑂(log 𝑛) delete(𝑘) 𝑂(log 𝑛) contains(𝑘) 𝑂(log 𝑛) isEmpty() 𝑂(1) size() 𝑂(1)

min()/max() 𝑂(log 𝑛) 𝑂(1)

floor(𝑥) 𝑂(log 𝑛)

ceiling(𝑥) 𝑂(log 𝑛)

rank(𝑥) 𝑂(log 𝑛)

select(𝑖) 𝑂(log 𝑛)

Binary heaps Strict Fibonacci heaps

Operation Running Time

construct(𝐴[1..𝑛]) 𝑂(𝑛) insert(𝑥,𝑝) 𝑂(log 𝑛) 𝑂(1) delMax() 𝑂(log 𝑛) changeKey(𝑥,𝑝0) 𝑂(log 𝑛) 𝑂(1) max() 𝑂(1) isEmpty() 𝑂(1) size() 𝑂(1)

I apart from fasterconstruct,

BSTs always as good as binary heaps

I MaxPQ abstraction still helpful