Conditions for Strong Synchronization In Concurrent Data Types

Conditions for Strong Synchronization

In Concurrent Data Types

Maged Michael

IBM TJ Watson Research Center

Dagstuhl Seminar on Consistency in Distributed Systems

February 2013

Joint work with Hagit Attiya, Rachid Guerraoui, Danny

Hendler, Petr Kuznetsov, and Martin Vechev

Idempotent Work Stealing

Work Stealing

Load Balancing

T1

T2

Work stealing is a load balancing technique

T1’s Work

T2’s Work

Put

Take

Steal

T1

T2

Three operations:



Put

a task in own work set



Take

a task from own work set

Idempotent Work Stealing



Observation:

Some application semantics can tolerate the repetition of tasks.

Such tasks are

idempotent tasks

.

Take

T1

Example:



Take and steal extract same

task



Inserted

once

, extracted

twice

T1’s Work

t

Steal

T2



Conventional work stealing

–

Each inserted task is eventually extracted

exactly once



Idempotent work stealing

Work Stealing Algorithms



Arora+ 1998, Frigo+ 1998, Hendler+ 2002, 2006, Chase-Lev 2005

public Object popBottom() { 20 long b = this.bottom; 21 CircularArray a = this.activeArray; 22 b = b - 1; 23 this.bottom = b; 24 long t = this.top; ...

Example from Chase-Lev 2005

The owner’s

take

operation

store load



Prior algorithms require a

store-load ordering

in the owner’s critical path



Store-load fence instructions and atomic instructions are typically slower than

regular memory access instructions

Opportunity



Design

idempotent work stealing

to exploit relaxed application semantics

Idempotent Work Stealing Algorithms



No lost tasks

Guarantees:

Three Algorithms with three extraction policies

T a k e S t e a l LIFO T  P u t H 



No garbage tasks extracted

H  FIFO T  T a k e S t e a l P u t T a k e Double Ended H  T  S t e a l P u t



Owner never extracts the same task twice



LIFO: Owner and thieves extract tasks from tail



FIFO: Owner and thieves extract tasks from head



Double-Ended: Owner extracts from tail.

Thieves extract from head.

LIFO Algorithm

anchor: <integer,integer> // <tail,tag> tasks: task array

1 <t,g> := anchor 2 if (t == tasks.size) EXPAND ... 3 tasks.array[t] := task 4 anchor := <t+1,g+1> 1 <t,g> := anchor 2 if (t == 0) return EMPTY 3 task := tasks.array[t-1] 4 anchor := <t-1,g> 5 return task

Structures

Put (task)

Take ()

4 task := a.array[t-1] Order read in 4 before CAS in 5

5 if !CAS(anchor,<t,g>,<t-1,g>) CONFLICT ... 6 return task

Steal ()

tail tag

packed word

only thieves need atomic ops

No StoreLoad order

LIFO Algorithm – Losing Steals



How

steals

may be lost

W X

2 Steal X

Y

The steal of X is lost

Similarly, steals concurrent with a slow

take

may be lost

2 100 1 100

Tail Tag

Order write in 3 before write in 4 4 anchor := <t+1,g+1>

Put (Y)

FIFO Algorithm

head: integer tail: integer tasks: task array

1 h := head 2 t := tail

3 if (t == h + tasks.size) EXPAND ... 4 tasks.array[t % tasks.size] := task 5 tail := t + 1

1 h := head 2 t := tail

3 if (t == h) return EMPTY

4 task := tasks.array[h % tasks.size] 5 head := h + 1 6 return task

Structures

Put (task)

Take ()

Order read in 1 before read in 2 2 t := tail

3 if (t == h) return EMPTY Order read in 1 before read in 4 4 a := tasks

5 task := a.array[h % a.size] Order read in 5 before CAS in 6

6 if !CAS(head,h,h+1) CONFLICT ... 7 return task

Steal ()

head 

tail 

No packed tags. No size limit

No StoreLoad order and

no atomic ops by owner

Double-Ended Algorithm

anchor: <integer,integer,integer> // <head,size,tag> tasks: task array

1 <h,s,g> := anchor

2 if (s == tasks.size) EXPAND ...

3 tasks.array[h+s % tasks.size] := task Order write in 3 before write in 4

4 anchor := <h,s+1,g+1>

1 <h,s,g> := anchor

2 if (s == 0) return EMPTY

3 task := tasks.array[h+s-1 % tasks.size] 4 anchor := <h,s-1,g> 5 return task

Structures

Put (task)

Take ()

4 task := a.array[h % a.size] 5 h2 := h+1 % MAXSIZE

Order read in 4 before CAS in 6

6 if !CAS(anchor,<h,s,g>,<h2,s-1,g>) CONFLICT 7 return task

Steal ()

head size tag

1 2 9876

head 

head + size 

packed word

limited max size

No StoreLoad order and

Conditions for Strong Synchronization

Joint work with

Hagit Attiya, Rachid Guerraoui, Danny Hendler, Petr

Kuznetsov, and Martin Vechev

Motivation

There are good algorithms with good features … except for

the requirement of strong synchronization

Are there conditions under which the avoidance of both RAW

and AWAR is impossible?

Atomic Write After Read (AWAR)

Read After Write (RAW) Order

StoreLoad order

Strong synchronization is typically slower than regular

instructions

Strong Non-Commutativity



Given a sequential specification Spec, a complete invocation s

of a

method m

is strongly non-commutative (SNC) if there exist a method

m

, histories

base and

s

, such that:

–

s

is a complete invocation of m

–

processes executing s

and s

differ

–

base is a complete sequential history

–

base



Spec, base



s



Spec , base



s



Spec

–

base



s



s



Spec, base



s



s



Spec

Any SNC invocation must contain strong synchronization

s

₁

influences

s

₂

AND

s

₂

influences

s

₁

Acknowledgement to

Sebastian Burckhardt

for suggesting

the improved form of the SNC definition

Common Examples of SNC Invocations



Set

–

Add(v) : true // Returns true iff v was not in the set

–

Remove(v) : true // Returns true iff v was in the set



LIFO Stack

–

Pop() : Nonempty Value



FIFO Queue

–

Dequeue() : Nonempty Value



CAS Data Type

–

CAS(expected,newval) : true



Work Stealing

–

Take() : Nonempty Task

–

Steal() : Nonempty Task



Counter

Avoiding SNC: Limited Concurrency

–

E.g., single-consumer FIFO queue

•

Successful multi-consumer

dequeue

is SNC

Avoiding SNC: Limited API

–

E.g., Set Add without return value

•

Set Add : true is SNC

•

Set Add : void is not SNC

–

E.g., Counter Add without returning value

•

FetchAndAdd : integer is SNC

Avoiding SNC: Idempotent Types

E.g., idempotent work stealing

•

Conventional (non-idempotent) Take is SNC

–

SNC with Steal

Implications



Algorithm Design

–

Guidance on when avoiding RAW/AWAR is futile



Hardware Design

–

Added motivation to lower overheads of RAW/AWAR



API Design

–

Sometimes return values in APIs dictate RAW/AWAR



Specifications and Correctness Conditions

–

Motivation to examine requirements for linearizability



Formal Verification and Algorithm Synthesis

–

Avoid useless work on non-RAW/AWAR algorithms when

RAW/AWAR is required