Roohollah Abdipour Processes Chapter 3

(1)

Chapter 3

Processes

Roohollah Abdipour

[email protected]

(2)

Outline

Basic Ideas

Threads

Virtualization

Code Migration

(3)

Basic ideas(1)

 To execute a program, an operating system creates a number of virtual processors, each one for running a different program.

 To keep track of these virtual processors, the operating system has a process table, Containing entries to store CPU register Values, memory maps, open files, etc.

 A process is often defined as a program in

execution, that is, a program that is currently

being executed on one of the operating system's

(4)

Basic ideas(2)

 An important issue is that the operating system takes great care to ensure that independent processes cannot maliciously or inadvertently affect the correctness of each other's behavior .

 In other words, the fact that multiple processes

may be concurrently sharing the same CPU and

other hardware resources is made transparent.

(5)

Basic ideas(3)

 This concurrency transparency comes at a relatively high price.

 Each time a process is created, the operating

system must create a complete independent

address space. Allocation can mean initializing

memory segments by, for example, zeroing a data

segment, copying the associated program into a

text segment, and setting up a stack for temporary

data.

(6)

Basic ideas(4)

 switching the CPU between two processes may be relatively expensive as well.

 Apart from saving the CPU context (which

consists of register values, program counter,

stack pointer, etc.), the operating system will also

have to modify registers of the memory

management unit (MMU) and invalidate address

translation caches such as in the translation

lookaside buffer (TLB).

(7)

Basic ideas(5)

 if the operating system supports more processes than it can simultaneously hold in main memory, it may have to swap processes between main memory and disk before the actual switch can take place.

 Like a process, a thread executes its own piece of code, independently from other threads.

 However, in contrast to processes, no attempt is made to achieve a high degree of concurrency transparency if this would result in performance degradation.

 Therefore, a thread system generally maintains only the

minimum information to allow a CPU to be shared by

several threads.

(8)

Basic ideas(6)

We build virtual processors in software, on top of physical processors:

Processor: Provides a set of instructions along with the capability of automatically executing a series of those instructions.

Thread: A minimal software processor in whose context a series of instructions can be executed. Saving a thread context implies stopping the current execution and saving all the data needed to continue the execution at a later stage.

Process: A software processor in whose context one or

more threads may be executed. Executing a thread, means

executing a series of instructions in the context of that thread.

(9)

Outline

Basic Ideas

Threads

Virtualization

Code Migration

(10)

• Observation 1: Threads share the same address space.

• Observation 2: Process switching is generally more expensive. OS is involved.

– e.g., trapping to the kernel.

• Observation 3: Creating and destroying threads is cheaper than doing it for a process.

Threads

(11)

Multithread Benefits(1/3)

 The most important benefit comes from the fact that in a single-threaded process, whenever a blocking system call is executed, the process as a whole is blocked.

 consider an application such as spreadsheet program, and assume that a user continuously and interactively wants to change values

 Another advantage of multithreading is that it becomes possible to exploit parallelism when executing the program on a multiprocessor system

 In that case, each thread is assigned to a different CPU while

shared data are stored in shared main memory.

(12)

Multithread Benefits(2/3)

 Multithreading is also useful in the context of large

applications. Such applications are often developed as a collection of co-operating programs, each to be executed by a separate process.

 Cooperation between programs is implemented by means of inter process communication (IPC)

mechanisms

 The major drawback of all IPC mechanisms is that

communication often requires extensive context

switching, shown at three different points in next

slide Figure

(13)

Context Switching in Large Apps

A large app is commonly a set of cooperating processes,

communicating via IPC, which is expensive.

(14)

Multithread Benefits(3/3)

 Different parts of an application can be implemented using isolated threads. These parts can communicate with each other via shared data space.

 Context switch between thread can be done in user space

 Low overhead in CS  performance gain

 A SW engineering reason: Many applications are easier to be designed and developed as a collection of cooperating threads

 e.g., using separate threads for input validation, grammar check, paging, TOC generation and etc. in a word processing program.

 Because threads are not automatically protected against each

other the way processes are, development of multithreaded

applications requires additional intellectual effort (i.e.,

additional care from the programmer).

(15)

Threads and OS (1/2)

Main Issue: Should an OS kernel provide threads or should they be implemented as part of a user-level package?

User-space solution:

 Pros:

 Low cost of thread creation and deletion (only needs to create and release a stack for thread)

 Low cost of context switch between threads (only needs storing and retrieving CPU registers’ values, no need for TLB clearance, …)

 Cons:

 Everything done by a thread affects the whole process.

So what happens when a thread blocks on a syscall?

 The whole process will be blocked!

(16)

Threads and OS (2/2)

Kernel solution: Kernel implements threads.

Everything is system call.

 Pros:

 Operations that block a thread are no longer a problem: kernel schedules another.

 External events are simple: the kernel (which catches all events) schedules the thread associated with the event.

 Cons:

 Less efficient: All thread-related operations (e.g., creation, deletion, coordination, etc.) should be done by kernel  Needs system calls  CS between threads is as expensive as CS between processes!)

A solution: Try to mix user-level and kernel-level threads into a single concept:

Light Weight Processes (LWP)

(17)

Hybrid (Solaris)

Use two levels. Multiplex user threads on top of LWPs (kernel threads).

•Threads are created and managed by programmer

•Kernel is not aware of threads

•All thread-related operations (creation, deletion, coordination, Mutex, conditional variables, etc.) are performed in user-level

•Kernel dynamically assigns some LWPs to each process

•Programmer is not aware of LWPs and cannot manage them.

•Each thread is run in the context of a LWP

(18)

When a user-level thread does a blocking syscall:

Executions change from user-level to level (but in the context of the same LWP)

 the LWP blocks (So, only the thread bounded to that LWP is blocked).

Kernel schedules another LWP.

Another thread is executed

This means a Context switch occurs (at user-level)

Hybrid (Solaris)

(19)

Hybrid (Solaris)

• Pros:

– Low cost of thread creation, deletion and coordination (all done in user-level)

– Blocking system calls do not block the whole process

– The programmer is not aware of the LWP

– LWPs can be run on simultaneously on different CPUs, transparently from the application.

• Cons:

– It needs to create and delete LWPs (kernel-level threads) which has high cost.

• But, LWP creating and deletion occoures rarely.

(20)

Threads in Distributed Systems

Server N threads

Input-output

Client

Thread 2 makes

T1 Thread 1

requests to server

generates results

Requests

Receipt &

queuing

Client and server with threads

(21)

Threads in Distributed Systems

Multithreaded clients(1/2)

• Main issue is hiding network latency.

• Multithreaded web client:

– Browser scans HTML, and finds more files that need to be fetched.

– Each file is fetched by a separate thread, each issuing an HTTP request.

– As files come in, the browser displays them.

• Multiple request-response calls to other machines (RPC):

– Client issues several calls, each one by a different thread.

– Waits till all return.

– If calls are to different servers, will have a linear speedup.

(22)

Fetching Images

 Suppose there are ten images in a page. How should they be fetched?

 Sequentially

fetch_sequential() {

for (int i = 0; i < 10; i++) { int sockfd = ...;

write(sockfd, "HTTP GET ...");

n = read_till_socket_closed(sockfd, jpeg[i], 100K);

} }

 Concurrently

fetch_concurrent() { int thread_ids[10];

for (int i = 0; i < 10; i++) {

thread_ids[i] = start_read_thread(urls[i], jpeg[i]);

}

for (int i = 0; i < 10; i++) {

wait_for_thread(thread_ids[i]);

} }

Threads in Distributed Systems

Multithreaded clients(2/2)

(23)

Threads in Distributed Systems

Multithreaded Servers(1/2)

• Improve performance:

– Starting a thread to handle an incoming request is much cheaper than starting a process.

– Single-threaded server can’t take advantage of multiprocessor.

– Hide network latency. Other work can be done while a request is coming in.

• Better structure:

– Using simple blocking I/O calls is easier.

– Multithreaded programs tend to be simpler.

(24)

Threads in Distributed Systems

Multithreaded Servers(2/2)

A multithreaded server organized in a dispatcher/worker model.

• Single-Threaded Servers:

• The server can not respond to clients while is doing a blocking system call (e.g., waiting for the disk)

• A solution: Implementing the server as a Finite-State-Machine.

(25)

Threads in Distributed Systems

A Finite State Machine(1/2)

 Suppose that threads are not available but the system designers find the performance loss due to single threading unacceptable.

 A third possibility is to run the server as a big finite-state machine.

 When a request comes in, the one and only thread examines it.

 If it can be satisfied from the cache, fine, but if

not, a message must be sent to the disk.

(26)

 However, instead of blocking, it records the state of the current request in a table and then goes and gets the next message.

 The next message may either be a request for new work or a reply from the disk about a previous operation.

 If it is new work, that work is started.

 If it is a reply from the disk, the relevant information is fetched from the table and the reply processed and subsequently sent to the client.

 In this scheme, the server will have to make use of non- blocking calls to send and receive.

Threads in Distributed Systems

A Finite State Machine(2/2)

(27)

Threads in Distributed Systems

• Three ways to construct a server:

Model Characteristics

Threads Parallelism, blocking system calls

Single-threaded process No parallelism, blocking system calls

Finite-state machine Parallelism, nonblocking system calls

(28)

Outline

Basic Ideas

Threads

Virtualization

Code Migration

(29)

The Traditional Server Concept

Web Server Windows

IIS

App Server Linux Glassfish

DB Server Linux MySQL

EMail

Windows

Exchange

(30)

And if something goes wrong ...

Web Server Windows

IIS

App Server DOWN!

DB Server Linux MySQL

EMail

Windows

Exchange

(31)

The Traditional Server Concept

• System Administrators often talk about servers as a whole unit that includes the hardware, the OS, the storage, and the applications.

• Servers are often referred to by their function i.e. the Exchange server, the SQL server, the File server, etc.

• If the File server fills up, or the Exchange server

becomes overtaxed, then the System Administrators

must add in a new server.

(32)

The Traditional Server Concept

• Unless there are multiple servers, if a service experiences a hardware failure, then the service is down.

• System Admins can implement clusters of servers to make them more fault tolerant.

However, even clusters have limits on

their scalability, and not all applications

work in a clustered environment.

(33)

The Traditional Server Concept

• Pros

– Easy to conceptualize – Fairly easy to deploy – Easy to backup

– Virtually any

application/service can be run from this type of setup

• Cons

– Expensive to acquire and maintain hardware

– Not very scalable – Difficult to replicate

– Redundancy is difficult to implement

– Vulnerable to hardware outages

– In many cases, processor

is under-utilized

(34)

Virtual Machine Monitors (VMMs)

...

Virtual Machine Monitor (VMM) VM_n VM₀ VM₁

Platform HW

I/O Devices Processor/CS

Memory

App_n App₀

Guest OS₀

App₁

Guest OS₁ Guest OS_n

(35)

The Virtual Server Concept

• Virtual servers seek to encapsulate the server software away from the hardware

– This includes the OS, the applications, and the storage for that server.

• Servers end up as mere files stored on a physical box, or in enterprise storage.

• Virtual servers can still be referred to by

their function i.e. email server, database

server, etc.

(36)

The Virtual Server Concept

• Virtual servers can be scaled out easily.

– If the administrators find that the resources supporting a virtual server are being taxed too much, they can adjust the amount of resources allocated to that virtual server

• Server templates can be created in a virtual

environment to be used to create multiple, identical virtual servers

• Virtual servers themselves can be migrated from

host to host almost at will.

(37)

The Virtual Server Concept

• Pros

– Resource pooling – Highly redundant – Highly available

– Rapidly deploy new servers

– Easy to deploy

– Reconfigurable while services are running – Optimizes physical

resources by doing more with less

• Cons

– Slightly harder to conceptualize

– Slightly more costly (must

buy hardware, OS, Apps,

and now the abstraction

layer)

(38)

Virtualization

•Related Concepts:

•Isolation

•Consolidation

•Migration

(39)

Virtualization: Isolation

HW

App₂ App₁

OS

VMM HW

App₁ App₂

OS OS

(40)

Virtualization: Consolidation

HW₁ HW₂

App₂ App₁

OS₁ OS₂

VMM HW

App₂ App₁

OS₁ OS₂

(41)

Virtualization: Migration

VMM HW₁

App

HW₂ VMM

OS

VMM HW₁

App

HW₂ VMM

OS

(42)

Code Migration

 Approaches to Code Migration

 Migration and Local Resources

 Migration in Heterogeneous Systems

(43)

Approaches

 Why code migration?

Moving from heavily loaded to lightly loaded.

Also, to minimize communication costs.

Moving code to data, rather than data to

code.

(44)

A framework of Code Migration

A process consists of three segments

• Code segment: contains the actual code

• Resource segment: contains references to external resources needed by the process

– E.g., files, printers, devices, other processes

• Execution segment: store the current

execution state of a process, consisting of

private data, the stack, and the program

counter

(45)

Two Notions of Code Mobility

• Weak mobility: only code, plus maybe some init data (and start execution from the beginning) after migration:

– Examples: Java applets

• Strong mobility: Code and execution segment are moved

– Migration: move the entire object from one machine to the other – Cloning: simply start a clone, and set it in the same execution

state.

• Initiation

– Sender-initiated migration

– Receiver-initiated migration

(46)

Alternatives for code migration

(47)

Migrating Local Resources (1/3)

• Problem: A process uses local resources that may or may not be available at the target site.

• Process-to-resource binding

– Binding by identifier: the process refers to a resource by its identifier (e.g., a URL)

– Binding by value: the object requires the value of a resource (e.g., a library)

– Binding by type: the object requires that only a type of resource is available (e.g., local devices, such as

monitors, printers, and so on)

(48)

Migrating Local Resources (2/3)

• Resource-to-machine binding

– Unattached: the resource can easily be moved along with the object (small files, e.g. a cache)

– Fastened: the resource can, in principle, be migrated but only at high cost (possibly larger)

• E.g., local databases and complete Web sites

– Fixed: the resource cannot be migrated, such as local hardware (bound to the machine)

• E.g., a local communication end point

(49)

Actions to be taken with respect to the references to local resources when migrating code to another machine.

Process-to- resource binding

Resource-to-machine binding

Unattached Fastened Fixed

By identifier By value By type

MV (or GR)

CP ( or MV, GR) RB (or GR, CP)

GR (or MV) GR (or CP)

RB (or GR, CP)

GR GR

RB (or GR) GR: Establish a global system-wide reference

MV: Move the resource

Migrating Local Resources (3/3)

(50)