DRAFT. Paternoster of Programmers Reloaded. C, C++, Java, Python and AspectJ Case Studies. Ed. Lecture notes, version

(1)

DRAFT

Paternoster of Programmers Reloaded AUTHOR’S EDITION i

Paternoster of Programmers Reloaded

C, C++, Java, Python and AspectJ Case Studies

(2)

DRAFT

Paternoster of Programmers Reloaded AUTHOR’S EDITION ii

The lecture notes were supported by the TÁMOP-4.1.2.A/1-11/1-2011-0103 project. The project has been supported by the European Union, co-financed by the European Social Fund.

The author endeavors to maintain and to keep the author’s editions of this book, and the othersin the seriesup-to-date. These author’s editions can be found at the pagehttp://www.inf.unideb.hu/~nbatfai/konyvek/.

(3)

DRAFT

Paternoster of Programmers Reloaded AUTHOR’S EDITION iii

COLLABORATORS

TITLE:

Paternoster of Programmers Reloaded

ACTION NAME DATE SIGNATURE

WRITTEN BY Dr. Norbert Bátfai March 9, 2014

Szakmai lektor András Keszthelyi March 9, 2014

Language reviewer Ildikó Novák March 9, 2014

REVISION HISTORY

NUMBER DATE DESCRIPTION NAME

0.0.1 31 May 2012 I have started to write this book (called POPR book in the following) from the version 0.0.11 of the Hungarian edition, entitled „Programozó Páternoszter újratöltve: C, C++, Java, Python és AspectJ esettanulmányok” (PROP könyv).

Bátfai

0.0.2 10 June 2012 I have started the section Simplification of the AI-based simulation model of RCSS.

Bátfai

0.0.3 3 July 2012 The chapter titled The simplified down protocol of 2D RCSS is reorganized.

Bátfai

0.0.4 8 Sept 2013 The legal notice paragraph is translated from Hungarian to English.

Bátfai

0.0.9 9 Sept 2013 The first two case studies have been translated. Bátfai

0.0.19 24 Sept 2013 The preface has been translated. Bátfai

(4)

DRAFT

Paternoster of Programmers Reloaded AUTHOR’S EDITION iv

REVISION HISTORY

NUMBER DATE DESCRIPTION NAME

0.1.2 7 Feb 2014 Theenvironment of the bookhas been updated with the book called The Yearbook of the Programmers of University of Debrecen.

Bátfai

0.1.4 13 Feb 2014 Incorporation of improvements made by the language reviewer.

Bátfai

(5)

DRAFT

Paternoster of Programmers Reloaded AUTHOR’S EDITION v

Contents

1 Introduction 1

1.1 The Paternoster of Programmers . . . 3

1.2 About these lecture notes . . . 3

1.2.1 The environment of this book . . . 3

1.2.2 The courses of the book . . . 4

1.2.2.1 High-Level Programming Languages 1 . . . 4

1.2.2.2 Other courses . . . 4

1.2.3 About the author . . . 5

1.2.4 About the peer reviewers . . . 5

I

C Case Studies

6

2 MINIX kernel hacking: analysing microkernel’s IPC 8 2.1 Introduction: theMINIXwhich acted as the detonator ofopen source . . . 10

2.1.1 TheMINIXmicrokernel and the MINIX IPC . . . 10

2.2 The installation ofMINIX3system . . . 14

2.2.1 Installation in VirtualBox. . . 14

2.2.2 The first MINIX kernel hacking . . . 17

2.3 Analysis ofMINIX3IPC . . . 21

2.3.1 A solution with modifying the PCB . . . 22

2.3.2 Another solution by introducing a new system call . . . 29

3 GNU/Linuxkernel hacking: making entries in the/procvirtual file system 34 3.1 The monolithic kernel ofLinux . . . 36

3.1.1 Linux kernel compiling. . . 36

3.1.2 Kernel modules . . . 43

3.2 Making entries in the/procvirtual file system . . . 45

3.2.1 Creating the module in VirtualBox. . . 45

(6)

DRAFT

Paternoster of Programmers Reloaded AUTHOR’S EDITION vi

4 Berkeley socket API, Sys V IPC and I/O multiplexing 54

4.1 Berkeley socket API, Sys V IPC, I/O multiplexing. . . 56

4.2 A simple client/server sample. . . 56

4.2.1 The client side . . . 56

4.2.2 The server side . . . 59

4.2.2.1 Discussing the source code of the server side . . . 63

4.2.2.2 Protecting the memory of the processes . . . 68

4.2.3 Testing of the example . . . 70

4.2.3.1 Testing on localhost . . . 70

4.2.3.1.1 Compiling and running the server. . . 70

4.2.3.1.2 Compiling and running the client . . . 70

4.2.3.1.3 There is some disturbance in the force . . . 71

4.2.3.2 Testing on two machines . . . 72

4.2.3.2.1 Portability . . . 72

4.2.3.2.2 Testing on a virtualized machine . . . 73

II

C++ Case Studies

76

5 A simplified protocol of 2D RCSS 78 5.1 Emasculation of the AI-based simulation model of RCSS . . . 80

5.1.1 Introducing a new positioning command for RCSS client protocol . . . 80

5.1.1.1 Testing of the newly introduced command . . . 80

5.1.1.2 The response of the simplified server . . . 84

5.2 The team called Debrecen Great Forest FC++--> . . . 84

5.2.1 The implementation of the throw-in . . . 86

5.2.1.1 Testing the throw-in . . . 88

5.2.2 The introduction of the tactical lineups . . . 91

5.2.2.1 The implementation of the corner kick . . . 93

5.2.2.1.1 Testing the corner kick . . . 94

5.2.2.1.2 The evaluation of the team Debrecen Great Forest FC++ . . . 94

5.3 The team called Debrecen Deep Forest FC++ . . . 95

5.3.1 The evaluation of the team Debrecen Deep Forest FC++ . . . 98

5.4 The team called Debrecen Round Forest FC++ . . . 100

5.4.1 The additional command line arguments of the simplified server . . . 100

5.4.1.1 The response of the simplified server . . . 100

5.4.1.2 Receiving the response of the simplified server . . . 101

(7)

DRAFT

Paternoster of Programmers Reloaded AUTHOR’S EDITION vii

III

Java Case Studies

105

6 Community consciousness net 106

6.1 The Seventh Eye mobile game . . . 108

6.1.1 The Seventh Eye and the „Community consciousness net” . . . 108

IV

Python Case Studies

111

7 A virtual librarian 113 7.1 Kálmán Könyves . . . 115

7.1.1 The structure of the AIML files . . . 118

V

AspectJ Case Studies

128

8 What is the mother tongue of the object oriented programs? 130 8.1 An analytical weaving . . . 132

8.2 A robot soccer weaving . . . 132

VI

Bibliography

134

8.3 Quotes . . . 135 8.4 Operating systems. . . 135 8.5 Programming . . . 135 8.6 Football . . . 136 8.7 Math. . . 137 8.8 For children . . . 137 8.9 Background . . . 137

9

Index

138

(8)

DRAFT

Paternoster of Programmers Reloaded AUTHOR’S EDITION viii

List of Figures

2.1 SEND and RECEIVE message passing primitives.. . . 11

2.2 The sender is blocked while the receiver is not ready to receive.. . . 12

2.3 The receiver is blocked while the message is not being received. . . 13

2.4 Giving the name of theMINIXsystem that will be installed. . . 14

2.5 Launching the virtualizedMINIX. . . 15

2.6 Starting the installation. . . 15

2.7 Loading the system from disk. . . 16

2.8 We accept defaults, mutatis mutandis. . . 16

2.9 MINIX starts from hard disk now. . . 17

2.10 Opening the source filekernel/main.cwith vi. . . 18

2.11 The modification of the functionannounce()in thekernel/main.csource file. . . 19

2.12 The booting of the newly compiled kernel. . . 20

2.13 The message from the kernel. . . 20

2.14 The first step of managing packages. . . 21

2.15 . . . 22

2.16 Printing out the size of the PCB and the size of the process table. . . 23

2.17 Read the size of the PCB and the size of the process table. . . 23

2.18 Extending the MINIC PCB.. . . 24

2.19 Counting massages. . . 25

2.20 The kernel process table. . . 25

2.21 Connecting a debug function to a function key. . . 26

2.22 The beginning of the functionuzenetszam_dmp. . . 26

2.23 The midst of the functionuzenetszam_dmp. . . 27

2.24 The end of the functionuzenetszam_dmp. . . 27

2.25 The prototype of the functionuzenetszam_dmp. . . 28

2.26 Displaying the IPC matrix. . . 28

2.27 List of the non-empty slots from the process table.. . . 29

2.28 An array to store the „IPC matrix”. . . 29

2.29 Incrementing of the appropriate element of theuzenetszamarray. . . 30

(9)

DRAFT

Paternoster of Programmers Reloaded AUTHOR’S EDITION ix

2.31 Usage of thesys_getmatrixsystem call that will be newly developed. . . 31

2.32 Printing the matrix. . . 31

2.33 Defining thesys_getmatrixmacro. . . 32

2.34 TheGET_MATRIXmacro. . . 32

2.35 Supplying thedo_getinfosystem call. . . 33

2.36 Displaying the IPC matrix in the second solution. . . 33

3.1 Downloading the kernel sources. . . 37

3.2 Making the.configwith commandmake menuconfig. . . 38

3.3 Setting the optionKernel .config support. . . 39

3.4 Starting the compiling. . . 40

3.5 Starting the installation. . . 41

3.6 The version of the running kernel. . . 42

3.7 The updated GRUB menu. . . 42

3.8 The new kernel. . . 43

3.9 Switching off the optionEnable loadable module support. . . 44

3.10 The size of the kernel image. . . 45

3.11 The circular doubly linked list of PCBs in Linux. . . 49

4.1 Compiling the client and the server. . . 73

4.2 Running the client and the server.. . . 74

4.3 Checking the results. . . 75

5.1 The LightFC++ team assembles before kick off using the move command. . . 83

5.2 After kick off the players move using the pos command. . . 83

5.3 Testing the throw-in: player 11 kicks the ball across the touch line. . . 88

5.4 Testing of the throw-in: player 4 starts to move towards the ball. . . 89

5.5 Testing the throw-in: player 4 is closer than 30 meters. . . 89

5.6 Testing of the throw-in: player 4 has just arrived to the ball. . . 90

5.7 Testing of the throw-in: player 4 passes the ball to player 3. . . 90

5.8 Testing the throw-in: the ball is moving towards player 3. . . 91

5.9 The 4-4-2 formation. . . 92

5.10 Corner kick formations. . . 92

5.11 Player 3 and player 6 should go to the ball. . . 95

5.12 An opposing player gains possession of the ball. . . 96

5.13 A portion of the soccerwindow2’s log file. . . 101

5.14 The Debrecen Round Forest FC++’s team logo in the soccerwindow2 and the rcssmonitor. . . 104

6.1 The starting icon of the Seventh Eye. . . 108

6.2 The splash screen of the Seventh Eye. . . 109

(10)

DRAFT

Paternoster of Programmers Reloaded AUTHOR’S EDITION x

List of Examples

2.1 Number of processes . . . 23

2.2 The size of the PCB . . . 24

2.3 Non-empty slots. . . 28

3.1 The contents of the PCB . . . 48

3.2 Playing the game with thecurrentmacro . . . 52

3.3 Drawing a memory map . . . 53

4.1 30 processes, with using a paper and pen . . . 58

5.1 Placing a player beside a goalpost . . . 92

5.2 Free kicks . . . 95

5.3 Do not pass backward. . . 99

5.4 A good starting . . . 99

5.5 Create the your own XPM team logo . . . 102

6.1 Running the Seventh Eye on a real mobile phone . . . 108

6.2 The client side of the „Community consciousness net” . . . 109

6.3 The server side of the „Community consciousness net” . . . 109

6.4 A community-based exercise . . . 110

6.5 A community portal based on the mental fingerprints . . . 110

7.1 Kálmán Könyves on IRC using theProgram W . . . 117

7.2 Kálmán Könyves on the web using theProgram D . . . 117

7.3 Create your own chatterbot . . . 127

8.1 Weaving this aspect into ALICE . . . 132

(11)

DRAFT

Paternoster of Programmers Reloaded AUTHOR’S EDITION xi

Dedication

(12)

DRAFT

Paternoster of Programmers Reloaded AUTHOR’S EDITION xii

Preface

The book that the dear reader is holding in his hands shows seven more or less well designed greater or smaller programming case studies. The level of elaboration of the examples also depends on the course in which the given example is taught. In the case of the subjects of the first three semesters of BSc studies, for example, for the courseHigh level programming languages 1-2, the case studies are highly developed. In contrast, in the case of the latter subjects, such asC, C++ Cas e Studies,Programming in GNU/Linux environment,Java Case StudiesandMobile Programming only the specification of the tasks is dominant. Topics of the case studies are scattered in a broad spectrum, because C, C++, Java, Python and AspectJ examples will be shown. Even for this reason alone these lecture notes are not regarded as a programmed introduction to a given particular field of information technology.

Let’s see the first case study about MINIX. For the understanding of this, the reader must be familiar with the textbook [OS]. The MINIX case study and this Tannenbaum book are very closely linked, because the case study itself is a solution of an exercise of the book [OS].

In other cases the link between the special literature and our lecture notes is not so close. However, even in these cases, close relations can be found inthe environment of these notes. For example, here we only briefly outline the structure of the famous RCSS team Agent2D [HELIOS], because the detailed introduction can be found in the book entitledMesterséges intelligencia a gyakorlatban: bevezetés a robotfoci programozásba, http://www.inf.unideb.hu/~nbatfai/konyvek/MIRC/mirc.book.xml.pdf, [MIRC]. But, we attach great importance to developing an our own RCSS team, which has been started from the sampleclient of rcssserver, therefore this example is well-developed here.

We can also find such example that was developed mainly in the environment of these notes. For example, the developing the Sevent Eye case study can be found in the bookMobil programozás, Nehogy már megint a mobilod nyomkodjon Téged!

http://www.inf.unideb.hu/~nbatfai/konyvek/MOBP/mobp.book.xml.pdf, [MOBP].

Finally, it may be noted that this work can be regarded as a continuation and an extension of the lecture notes Programozó Páternoszter[PP].

(13)

DRAFT

Paternoster of Programmers Reloaded AUTHOR’S EDITION 1 / 141

Chapter 1

(14)

DRAFT

Abstract

We are not in a difficult situation when we need to write the introduction to this book, because the „original” Paternoster of Programmers can be a good muse for this, where in a minimalist style, the Éric Lévénez’s timelines were mentioned. Now, we will try to give a more pathetic one.

(15)

DRAFT

„Facebook is not your friend.”

—Richard Stallman stallman.org/facebook.html

1.1

The Paternoster of Programmers

„A lot of people misinterpret that, as if I don’t care about revenue or profit or any of those things. But what not being just a company means to me is not being just that — building something that actually makes a really big change in the world.”

—Mark Zuckerberg [FBB] David Kirkpatrick: The Facebook Effect: The Inside Story of the Company That Is Connecting the World

On the one hand, the background to the name Paternoster is the metaphor that anybody can travel some floors. It means exactly that the reader can choose an arbitrary task of current interest from the wide spectrum of the tasks of the book and is able to reproduce the solution that is also fully explained in the book Paternoster. On the other hand, the Latin namePater noster has distinctly religious overtone, because it means the Lord’s Prayer. This direction is also confirmed by the fun question asked in the lab sessions „Who can celebrate the source code”. The basic message of the usage of the name paternoster is that programmers must program every day.

But also there is one other motivation... The great ones of science give meaning to intuitive notions such as • changing(Newton,1643-1727, mathematics, physics)

• infinity(Cantor,1845-1918, mathematics) • space and time(Einstein,1876-1955, physics). • computability(Turing,1912-1954, informatics).

The programming also has its own legendary hackers like Richard Stallman, Mark Zuckerberg or Linus Torvalds. But who will be the next genius who is able to give meaning to the following exciting notion standing on the shoulders of the giants. And what will be the next concept? The intuitive notion of love was reinterpreted by Jesus in the New Testament, but it has not still been developed deductively, in sense that it has no mathematical background. I believe, the most promising candidates are imagination and reality. This was the reason why the Penrose-Hameroff „Orch OR” model of consciousness was mentioned in the „original” Paternoster of Programmers [PP] as part of the introduction to quantum informatics.

Many programmers know the feeling that we really ought to write a good computer program. The root of this is that the programmers can create whole worlds in their programs. In this sense, beginner programmers have already participated in a sort of a genesis. It may be actually true for only a few professions. Programming something is a very constructive process, as Chaitin said in [METAMATH] „you understand something only if you can program it”. Based on Turing’s work on computing machinery, in my humble opinion, programmers may do the next scientific conceptual breakthrough. For example, to define random infinite sequences of 0 and 1 is a hard mathematical statistical issue, but the same result is achieved, relatively easily, by using the algorithmically complexity.

When reading this book, it is important for dear readers to keep in mind that neither this book nor other books can give the excitement of writing programs. This book can be only the preliminary step in knowing the real nature of programming. First, try the examples in the book, then write your own modifications to these examples. Certainly you must work on real computers from the start of reading the book.

1.2

About these lecture notes

1.2.1

The environment of this book

This book has been continuously improved together with other books in the following series

(16)

DRAFT

• Bátfai Norbert, Juhász István:Javát tanítok, Bevezetés a programozásba a Turing gépekt˝ol a CORBA technológiáig http://www.tankonyvtar.hu/-hu/tartalom/tkt/javat-tanitok-javat, [JAVATTANITOK].

• Bátfai Norbert: Mobil programozás, Nehogy már megint a mobilod nyomkodjon Téged! http://www.inf.unideb.hu/~nbatfai/-konyvek/MOBP/mobp.book.xml.pdf, [MOBP].

• Bátfai Norbert: Mesterséges intelligencia a gyakorlatban: bevezetés a robotfoci programozásba http://www.inf.unideb.hu/-~nbatfai/konyvek/MIRC/mirc.book.xml.pdf, [MIRC].

• Bátfai Norbert:Párhuzamos programozás GNU/Linux környezetben: SysV IPC, P-szálak, OpenMP http://www.inf.unideb.hu/-~nbatfai/konyvek/PARP/parp.book.xml.pdf, [PARP].

• Bátfai Norbert:Programozó Páternoszter újratöltve: C, C++, Java, Python és AspectJ esettanulmányok http://www.inf.unideb.hu/-~nbatfai/konyvek/PROP/prop.book.xml.pdf(ez a jelen könyv).

• Bátfai Norbert et al.: The Yearbook of the Programmers of University of Debrecenhttp://sourceforge.net/projects/udprog/, [UDPROG].

All these (only Hungarian language) books in this series try to support each other with background information and know-how. It is not too hard to organize because these books contain many common programming themes. Moreover, in many cases, it occurs that an example of one book is further developed in another one.

1.2.2

The courses of the book

In this section, we briefly introduce the courses that are based on using this book.

1.2.2.1 High-Level Programming Languages 1

This is a fundamental and the first hard core Programming class on the software engineering B.Sc. at University of Debrecen. The programming tasks for this course can be found in an exercise workbook entitled The Yearbook of the Programmers of University of Debrecen. It can be found on Sourceforge under the project nameudprogthat contains the book itself in DocBook 5.1 and a git repository to maintain the source codes of solutions of exercises. But some of the exercises ofudprogare developed in details here, in the present lecture notes.

Availability of the source codes and the book itself

In the spirit of „Release Early, Release Often” [KATEDRALIS], these lecture notes have already been downloadable since the beginning of its writing. It can be downloaded in DocBook XML 4.4 source format and in some other formats such as PDF and EPUB as well from theauthor’s homepage.

The source codes were copied from this book itself for testing, so all programs will work properly, at least in theory. In addition, the usage of the source codes is shown in detail for all examples.

The evolution of the lecture notes

This book is an informatics one so its basic nature is changing. Therefore we are continuously maintaining it as an author’s edition at the pagehttp://www.inf.unideb.hu/~nbatfai/konyvek/.

1.2.2.2 Other courses

The examples of these lecture notes can be naturally used in other courses. For example, in the author’s (only Hungarian language) courses, by the following breakdowns of exercises:

(17)

DRAFT

• Programozás GNU/Linux környezetben(PTI, GI M.Sc. lecture and lab), • Java esettanulmányok(PTI, GI B.Sc. lab),

• XML, HTML(PTI B.Sc. lab),

• Mobil programozás(PTI, GI B.Sc. lab).

• Magas szint˝u programozási nyelvek 2(PTI, MI, GI B.Sc. lecture and lab)

1.2.3

About the author

Norbert Bátfai is working as an assistant professor in the Faculty of Informatics at the University of Debrecen, Hungary. He received his M.Sc. (summa cum laude) in Computer Science in 1998 at the Kossuth Lajos University (KLTE), Debrecen, Hungary. In 1999, he won the first prize in the Java Programming Contest organized by Hungarian Java Alliance: Sun, IBM, Oracle, Novell and IQSoft. In 2004, his company won the first prize in the Hungarian Mobile Java Developer Contest organized by Sun Hungary and Nokia Hungary. In 2008, the Hungarian Chief Information Officers’ Association selected him as anIT trainer of the year. He received the Ph.D. degree in 2011. He won the Pollák-Virág award from Scientific Association for Infocommunications Hungary in 2012.

1.2.4

About the peer reviewers

András Keszthelyi, PhD., [email protected], Óbuda University.

(18)

DRAFT

Part I

(19)

DRAFT

This part is divided into three chapters.

• In the first chapter we solve aMINIX kernel exercise.

• In the next chapter we are going to show how to create entries to Linux/procvirtual file system froman our own kernel module.

• Finally, in the last chapter, we presenta parallel network serverthat is based on I/O multiplexing and use shared memory as IPC.

(20)

DRAFT

Chapter 2

MINIX kernel hacking: analysing microkernel’s

IPC

(21)

DRAFT

Abstract

In this chapter we have made a lifelong friend with a beautifully simple microkernel calledMINIX. We solve an exercise of Tanenbaum and Woodhull’s operating systems book,

• first with the modification of a part of PCB (Process Control Block) defined in proc.h, • then with the creation of a new system call.

(22)

DRAFT

„Re 2: your job is being a professor and researcher: That’s one hell of a good excuse for some of the brain-damages of minix.”

—Linus Benedict Torvalds Linus-Tanenbaumvita

2.1

Introduction: the

MINIX

which acted as the detonator of

open source

Eric Steven Raymond (esr) changed the termfree softwaretoopen sourceinThe Cathedral and the Bazaarin 1998. More than ten years back, the MINIX had been released in 1987, so obviously it was not open source.

GNU GPL

It may be noted that the first version of GNU GPL was released in 1989 and the first Linux had already been licensed under GNU GPL version 2, released in 1991.

At the beginning of the informatics profession, the C source codes of UNIX systems could have been used in university courses, but it lasted only a short period of time due to UNIXs had become a very valuable product quickly and then it was no longer possible to teach the operating systems in its C source form. The MINIX was born in this environment. The purpose of it was specifically to allow the usage of the source code of a UNIX-like operating system in education again.

From this aspect, the MINIX can be regarded as a pioneer of open source operating systems and, ultimately, we can say that the open source was brought into existence by the closing of the source code [OS3].

2.1.1

The

MINIX

microkernel and the MINIX IPC

The IPC (Inter-Process Communication) determines a communication method in which programs can exchange data with each other. There are several IPC mechanisms, from pipes through anonymous, local and network sockets to using shared memory (from these methods semaphore arrays and the shared memory will be used in thethird case studies). The MINIX IPC is based on the message passing. The massage passing takes place between two appropriately synchronised (by the rendezvous principle) processes, so we would not buffer the messages [OS], [OS3], as shown in the following figures.

(23)

DRAFT

(24)

DRAFT

(25)

DRAFT

(26)

DRAFT

2.2

The installation of

MINIX3

system

There is a great experience for speed lovers to use the MINIX installed on a primary partition, but it is simpler if the first encounter with MINIX takes place on a virtualized machine. So we are going to installMINIX3in VirtualBox. For example, let’s choose the latest development snapshot imageminix3_2_1_ide_20131118.iso, that should be written to a CD disk, but it is a better way to mount this ISO image directly.

2.2.1

Installation in VirtualBox

Figure 2.4: Giving the name of theMINIXsystem that will be installed.

Creating a virtual machine in VirtualBox is a very simple, but necessary subtask. After some clicks with using the default settings the new virtual MINIX system will be launched soon.

(27)

DRAFT

Figure 2.5: Launching the virtualizedMINIX.

At first startup, we must give from where we want to boot. All we have to do is to choose the downloaded image file.

Figure 2.6: Starting the installation.

(28)

DRAFT

Figure 2.7: Loading the system from disk.

We must follow the instructions which appeared on the screen: the installation will start after we have logged in as root and enteredsetup.

(29)

DRAFT

During the next installation steps we may accept the defaults by pressing Enter, mutatis mutandis. Then (after unmounting the installer disk image in the menu item Devices/CD/DVD Device) we are able to boot the new MINIX system from the hard disk.

Figure 2.9: MINIX starts from hard disk now.

In the next section we are going to take possession of the newly installed MINIX.

2.2.2

The first MINIX kernel hacking

The first kernel compilation is an obligatory task for all students. Let’s do it together now!

Using the command cd /usr/srcwe enter the directory that contains the MINIX kernel-tree. Then we open the source file kernel/main.cwith vi editor. The usage of vi is not masochism, because the installed MINIX has no other editors now.

(30)

DRAFT

Figure 2.10: Opening the source filekernel/main.cwith vi.

In this source file, search for the functionannounce().

/*===========================================================================*

* announce *

*===========================================================================*/ static void announce(void)

{

/* Display the MINIX startup banner. */ printf("\nMINIX %s.%s. "

#ifdef _VCS_REVISION

"(" _VCS_REVISION ")\n"

#endif

OS_RELEASE, OS_VERSION);

printf("MINIX is open source software, see http://www.minix3.org\n"); printf("nORBi a kernelben\n");

printf("%c%s",0x1B,"[47;30m"); }

(31)

DRAFT

Figure 2.11: The modification of the functionannounce()in thekernel/main.csource file.

All we did was to write a message during booting. The Hungarian language messagenORBi a kernelbenroughly means „the author is in the kernel” that is similar to a „Hello, World!„ message from the kernel. This message will be printed in boot time. In addition, the video mode of the terminal is changed to gray background and black foreground. The description of the used ANSI escape codes, for example, can be found inWikipedia.

After saving the source (using Shift+zz key combination in vi) themake installcommand compiles the kernel sources, then the system must be rebooted by the commandshutdown -r.

(32)

DRAFT

Figure 2.12: The booting of the newly compiled kernel.

In the case of the newly compiled kernel we can see the message from the kernel and the changed video mode as well.

(33)

DRAFT

Installing the packages

Handling packages and the packages themselves are always changing... At first in Minix3, we could usepackman package manager program and for example the emacs editor could be installed from package. At this moment MINIX’s package manager ispkginand emacs cannot be installed from package.

Before listing the available packages we must enter the commandpkgin update, after this we are able to choose from packages using commandpkgin av|more.

Figure 2.14: The first step of managing packages.

Customize Your Minix!

For example, I like to set the bash’sPS1environment variable to show the host name, the user name and the working directory. I usually add the following lines to the bottom of the.profilefile.

export PS1="\[\033[47;30m\][minix@\u]\w > " clear

bash exit

If we have already set the video mode in kernel, we do the same in the shell too. Here we may notice that using .bashrcto setPS1is a more robust solution than using.profile

export PS1="\[\033[47;30m\][minix@\u]\w > " clear

because in this case themake world will not delete this file.

2.3

Analysis of

MINIX3

IPC

The examples of this section are the solutions of exercise 44 on page 219 of book [MINIX3_OS_BOOK]. The exact wording of the exercise, cited from the book [MINIX3_OS_BOOK], is the following: „Add code to the MINIX 3 kernel to keep track of the number of messages sent from process (or task) i to process (or task) j. Print this matrix when the F4 key is hit.”

(34)

DRAFT

In the author’s Operating System courses between 2008 and 2010, the solutions of this task were used as typical examples to illustrate the Minix kernel programming. At that time, the Minix kernel was 3.1, and our students were helped to solve this exercise by the following notes: DEIK_MIPPOS_2008tavasz_BN_KiemeltOttoni_OR168_38.pdf. But remember, Minix is a living system, its code is changing rapidly, and therefore we may find several points, in this linked document, where the source code has already changed. This is a common phenomenon in informatics. For example, being restricted to Minix only, in the Minix book we can read about managing penalty points insched()and we can see it in source code in the appendix of the book, but it is not included in the current version of the Minix kernel.

2.3.1

A solution with modifying the PCB

In this section we are working withMINIX 3.2.1. At the time of the writing of this book, it is the current version and it has just been installed in the section „The installation of MINIX3 system”

The PCB is an abstraction of the processes of the operating system, which is typically realised as a C structure. In Minix this structure is thestruct procthat is defined inkernel/proc.hsource file. The process table in Minix is simply a structure array ofNR_TASKS + NR_PROCSPCB elements.

Acquiring routine

If you have no experience in Minix kernel programming we suggest that you should suspend the elaboration of this example and first start by solving the following exercise.

300 processes

When you log into the machine a special process called the shell will be created. If you enter a command in the shell, a (child) process will be created to execute the command. Web servers may fork processes to handle client requests and the list goes on. Many situations can be listed in which processes play significant roles. Actually, it is not suprising because processes are abstractions of computer programs. But it is obvious that the number of processes is limited by the size of the process table array anyway. Therefore, it is a legitimate exercise to increase the size of the process table: set the number of possible processes to 300 (from 256).

We may set the value of NR_PROCS in the source fileinclude/minix/sys_config.h 1. Set the value of NR_PROCS to 300 from 256.

(35)

DRAFT

2. After setting the NR_PROCS, it will be interesting to see the size of the process table. It can be done easily by following the previous exercise in SectionThe first MINIX kernel hacking.

Figure 2.16: Printing out the size of the PCB and the size of the process table.

Example 2.1Number of processes We have also done this exercise ourselves.

After the above two steps we can see in the next figure that the size of the PCB is equal to 1664 bytes and the size of the process table is 507520 (=1664*(5+300)). If the reader completes this exercise by himself/herself, he or she will see that the result will be different due to the fact that we have already used a modified version of PCB. Because our PCB has already contains a vector in which the number of elements is dependent on the value of NR_PROCS too.

(36)

DRAFT

Example 2.2The size of the PCB

Print the size of the PCB and the size of the process table from kernel. This exercise was solved as an implicit part of the previous exercise.

Returning to the original exercise, the MINIX PCB is being extended with an array of integers (called in Hungrianuzenetszam

that means the number of messages). The i-th element of this array says how many messages were sent to the i-th process.

Figure 2.18: Extending the MINIC PCB.

In principle, we know where to store the number of sent messages, now we are going to program the counting procedure itself. ++(caller_ptr->uzenetszam[dst_ptr->p_nr + NR_TASKS]);

The appropriate element of theuzenetszamarray in the caller’s PCB is being incremented. Here, it is necessary to add+ NR_TASKSto the array index because the numbering of processes starts from -NR_TASKS.

To avoid complication (in the first approach), this increment statement has been placed into the source filekernel/proc.c, into the begining of the functionmini_send.

(37)

DRAFT

Figure 2.19: Counting massages.

We have already mentioned that the numbering of processes starts from -NR_TASKS. It can be seen not only in the kernel sources but also if you press F1 key:

Figure 2.20: The kernel process table.

InGNU/Linuxsystems, the/procvirtual file system serves for monitoring the running kernel. Minix kernel has no such tool like this. We can obtain information about the running kernel over the Information Server (is), which is located in the server part of the Minix microkernel architecture. If we are going to debug the running kernel (in our case to print out the added PCB vectors), we need to use Information Server. We can link debug functions to function keys in the source fileservers/is/dmp.c, right now to the F4.

(38)

DRAFT

Figure 2.21: Connecting a debug function to a function key.

The debug functionuzenetszam_dmpfirst prints the process names of the non-empty slots of the process table as a header line. We may have noticed that we are in the server layer of the kernel tree to be more precise we have worked in theis server, where to the process table is copied from the bottom layer of the kernel by the system callsys_getproctab. The uzenetszam_dmpfunction works this copied instance of the process table, because the bottom layer of the kernel cannot be seen from the layer of servers.

Figure 2.22: The beginning of the functionuzenetszam_dmp.

(39)

DRAFT

Figure 2.23: The midst of the functionuzenetszam_dmp.

At the end of printing the matrix, themennyitol(in English it means „from number”) variable will be incremented by 21, it implements a kind of paging for displaying the „IPC matrix”. To be more precise, if the F4 key has been pressed, a submatrix of size 22x9 will be printed from the actual value ofmennyitol.

Figure 2.24: The end of the functionuzenetszam_dmp.

Since the functionuzenetszam_dmpis also used outside of the source fileservers/is/dmp_kernel.c, we must declare it, therefore we give its prototype in the source fileservers/is/proto.h.

(40)

DRAFT

Figure 2.25: The prototype of the functionuzenetszam_dmp.

After this, all we have to do is to compile the kernel, then to boot the new kernel. And after the login, if we press F4 the following debug screen appears:

Figure 2.26: Displaying the IPC matrix.

Example 2.3Non-empty slots

Print out the non-empty slots of the kernel process table in the form of [process name]-[p_quantum_size_ms]-[p_cpu_time_left] when F1 key is pressed (that is using theisInformation Server). By solving this exercise, you must know the struct proc structure membersp_quantum_size_msandp_cpu_time_leftdefined in the kernel part of Minix PCB in the source file kernel/proc.h.

(41)

DRAFT

Figure 2.27: List of the non-empty slots from the process table.

2.3.2

Another solution by introducing a new system call

In the previous section, we modified the PCB. Then (using thesys_getproctabsystem call) we copied the kernel process table to the layer of server processes. Now, outside the PCB, but inside the kernel we create a data structure: a matrix of integers to store the „IPC matrix”. To access this matrix, we create a new system call.

Define a 2 dimensional array of size (NR_TASKS + NR_PROCS)*(NR_TASKS + NR_PROCS) outside the PCB structure

Figure 2.28: An array to store the „IPC matrix”.

Counting of IPC messages will be done in the same manner as described in the previous section. But now, theuzenetszam array will be accessed directly and not through the PCB.

(42)

DRAFT

Figure 2.29: Incrementing of the appropriate element of theuzenetszamarray.

Theuzenetszamarray of the kernel is also defined with the same name in the server layer.

Figure 2.30: This is theuzenetszamthat we defined in the server layer.

Use thesys_getmatrixsystem call that will be newly developed for copying the IPC matrix from kernel level to the server level.

(43)

DRAFT

Figure 2.31: Usage of thesys_getmatrixsystem call that will be newly developed.

Printing the matrix has changed very little only in one line:

printf("%7d|", uzenetszam[rp->p_nr + NR_TASKS][i]);

Figure 2.32: Printing the matrix.

In the source fileinclude/minix/syslib.h, write thesys_getmatrixmacro. This macro will call the system call do_getinfothat will do the real work. Thesys_getmatrixmacro helps simplify using thedo_getinfo.

(44)

DRAFT

Figure 2.33: Defining thesys_getmatrixmacro.

TheGET_MATRIXconstant (used in thesys_getmatrixmacro) is defined in sourceinclude/minix/com.h, its value is chosen to be equal to 255.

Figure 2.34: TheGET_MATRIXmacro.

There is not much for us to do to add a case branch to the switch ofdo_getinfoin order to handle the calls of thesys_get matrix.

(45)

DRAFT

Figure 2.35: Supplying thedo_getinfosystem call.

Now we are ready to recompile and boot the new kernel. Then, after pressing F4, we can see the fruits of our labour.

Figure 2.36: Displaying the IPC matrix in the second solution.

It is interesting to compare the two solutions. For example, what happens if processes are created then destroyed after they have been finished and so on.

The availability of this case study.

The two presented solutions are available in a VirtualBox virtual machine that can be found at http://www.inf.unideb.hu/-~nbatfai/MINIX3.2.ova

(46)

DRAFT

Chapter 3

GNU/Linux

kernel hacking: making entries in

(47)

DRAFT

Abstract

In this chapter we have made a lifelong friend with a beautifully sophisticated monolithic kernel calledLinux. • First, we are going to configure and compile a customLinuxkernel.

(48)

DRAFT

„LINUX is obsolete”

—Andy Tanenbaum (ast) Linus-Tanenbaumvita

3.1

The monolithic kernel of

Linux

„<odd>.x.x: Linus went crazy, broke absolutely _everything_, and rewrote the kernel to be a microkernel using a special message-passing version of Visual Basic.”

—Linus Torvalds RFD: Kernel release numbering(Linux Kernel Mailing List)

In the previous case study, we had some experience with the operation of the microkernel, where architectural components are organized in different hierarchical levels. This section drives us towards the other end of the spectrum, to the world of monolithic kernels. What do monolithic and micro kernels mean in practice? Think about the previousMINIXcase study, where the data structures of the lower levels of the kernel could not be accessed from the higher levels. For example, from the Minix Information Server, we could not access the vector added to PCB, we needed to copy it up to the server level. In a monolithic kernel, the data structures and functions of the kernel can be accessed from anywhere in the kernel code.

In the known history of mankind, there have been several examples where huge masses of people were cooperating to achieve some mutual goals or were involved in achieving some goals. Some examples may be the building of the pyramids, the religious communities and the large mass armies of the 19th century.

If we investigate only pure intellectual purposes, we will find that we are not an embarrassment of riches. A present day example may be the building of the LHC (Large Hadron Collider), that was built by10.000technicians, engineers and scientist from all over the world. This may be comparable to writing the Linux kernel that is being maintained by more than8000developers [KERNELDEV]. And if we focus not only on the kernel itself but the whole GNU/Linux system (including editors, browsers, games, etc.), then we will get a much greater number than 10000 volunteers.

3.1.1

Linux kernel compiling

We believe that compiling the kernel should be a fundamental experience for all programming students. If the reader has never done it, we will do it now, in this section.

Compiling the kernel is a relatively safe activity, but it is better to do it in a virtualized Linux system. So if the reader has never done it before, we will compile it now, using the virtualized imageBatfai_Prog1.ova.

The Linux kernel is a large C program, in order to compile it we need to download its source. The actual kernel sources can be downloaded fromkernel.org. Here, at the time of writing, the 3.5 is the actual version. In the beginning, a simple version numbering scheme was used: the unstable versions were numbered by odd numbers and the stable ones by even numbers. But it was too „mathematical”, now a more sophisticated model was used.

Kernel version numbering

But joking apart, the rethinking of the numbering concept was suggested by Linus Torvalds in his email subjected „RFD: Kernel release numbering”.

The 2.6 kernels had already been numbered by the next schema. A stablemajor.minor.revisionis followed by a sequence of major.minor.revision.patchlevelstable versions that contain bugfixes and security fixes. In parallel with these versions, themajor.minor.revision+1"-rc-patchlevel"is the development (unstable) version, where new things are introduced. The next stable versionmajor.minor.revision+1will be evolved from the development versions.

On the Linux’s 20th birthday, the version numbering was bumped from 2.6 to 3.0. Now, therevisionshows the fixes, the development versions have thercprefix.

We have chosen the current version, that is 3.5. The whole source tree is roughly 70-80 MB (it is not sufficient to download the patch that is only some kilobytes of large.) The unpacked size will be roughly a half gigabyte, but it should be noted that the size of the compiled kernel tree may be larger than 5 GB.

(49)

DRAFT

Using the commandwget http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.5.tar.bz2, here we are downloading the kernel sources.

Figure 3.1: Downloading the kernel sources.

The root of the unpacked kernel tree contains aREADMEfile, we follow the procedure descripted in this. First, unpack the sources with commandtar xvjf linux-3.5.tar.bz2or cited from the README, with thebzip2 -dc linux-3.5.tar.bz2 | tar xvf -command. The compiling is controlled by the.configfile of the kernel tree that file also can be found in the running system’s/boot folder. We will use the commandmake menuconfigin order to customize the kernel. Themake menuconfigworks in the terminal screen and its output is the file.config. If you have already compiled the kernel, use the commandmake mrproper. This program will clean up the config and the object files from the source tree. As a best practice, the reader can copy the existing file from the/bootdirectory into the root of the source tree, then uses the commandmake oldconfig, because in this case it is enough to make decisions about the new features only.

(50)

DRAFT

Figure 3.2: Making the.configwith commandmake menuconfig.

Reading all kernel options inmake menuconfigis a whole day’s work, here we can see only one: under the General setup menu, we are going to set theKernel .config supportoption that will save the.configfile into the running kernel itself.

(51)

DRAFT

Figure 3.3: Setting the optionKernel .config support.

Linux kernel compiling may be coming now. Use the makecommand to compile the sources (here we do not follow the README, the argumentO=output/diris simply omitted, because we do not want to use a separate build directory).

(52)

DRAFT

Figure 3.4: Starting the compiling.

The compilation has already finished, now we sudo into root and give the commandmake modeules_install install. With this command, we install the modules and the kernel.

(53)

DRAFT

Figure 3.5: Starting the installation.

(54)

DRAFT

Figure 3.6: The version of the running kernel.

We should now reboot the computer. After this, it can be seen that the grub menu has also been updated properly.

Figure 3.7: The updated GRUB menu.

(55)

DRAFT

Figure 3.8: The new kernel.

3.1.2

Kernel modules

In the previous section, you can see that options marked by a star (*) will be compiled into the kernel. If we mark an option with M, it will be compiled into the kernel as a module.

(56)

DRAFT

Figure 3.9: Switching off the optionEnable loadable module support.

Without module support, the size of the kernel image is much larger than the original one. Let’s also see it after creating and booting a new pure monolithic kernel:

[norbert@BatfaiProg1 ~]$ ls -l /boot/vmlinuz-‘uname -r‘

(57)

DRAFT

Figure 3.10: The size of the kernel image.

3.2

Making entries in the

/proc

virtual file system

As a first step we present we are going to the whole solution of the exercise, then we will „celebrate the source code”.

3.2.1

Creating the module in VirtualBox.

The solution is presented using the virtual machineBatfai_Prog1.ova. Exercise: making entries in the/proc

Write a kernel module that prints debug information about processes into a file under/proc.

This example already appeared in the original Paternoster [PP] (pp. 177). Further background information can be found in the books [LDD] and [KMGUIDE] and especially in the following articles

• Randy Dunlap: Linux kernel seq_file HOWTO • Driver porting: The seq_file interface

1. Now we have made the module. Its source code calledsbejegyzes.ccan be found in thePROP_peldak/sajat_ bejegyzes_moduldirectory of thevirtualized system.

(58)

DRAFT

#include <linux/init.h> #include <linux/module.h> #include <linux/kernel.h> #include <linux/sched.h> #include <linux/list.h> #include <linux/proc_fs.h> #include <linux/stat.h> #include <linux/seq_file.h>

MODULE_DESCRIPTION ("Ez a sajat bejegyzes (PROP konyv) kernel modul"); MODULE_AUTHOR ("Bátfai Norbert ([email protected])");

MODULE_LICENSE ("GPL"); static int

taszk_lista (struct _{seq_file *m,} void _*v) {

int i = 0;

struct _{task_struct *task;} struct _{list_head *p;}

// a kernel/sched/core.c-ben latott modon iratjuk ki // a betut (szemben a fs/proc/array.c-ban lathatoval) // stat_nam[task->state ? __ffs(task->state) + 1 : 0] static const char stat_nam[] = TASK_STATE_TO_CHAR_STR;

seq_puts (m, "sajat taszk lista (negyedik kernel modul, PROP konyv)\n"); seq_printf (m, "%-9s %-16s %-6s %-12s %-6s %-3s\n",

"#", "CMD", "PID", "FLAGS", "ST1", "ST2"); list_for_each (p, current->tasks.next)

{

task = list_entry (p, struct task_struct, tasks);

seq_printf (m, "%-9i %-16s %-6i %-12u %-6li %-3c\n", ++i, task->comm, task->pid, task->flags, task->state,

stat_nam[task->state ? __ffs (task->state) + 1 : 0]); }

return 0; }

static int

sajat_open (struct _{inode *inode,} struct _{file *file)} {

return single_open (file, taszk_lista, NULL); }

static struct file_operations sajat_fajl_muveletek = { .owner = THIS_MODULE, .open = sajat_open, .read = seq_read, .llseek = seq_lseek, .release = single_release };

static struct _{proc_dir_entry *sajat_proc;} static int

sbejegyzes_init_module (void) {

struct _{proc_dir_entry *sajat_proc_fajl;}

if (!(sajat_proc = proc_mkdir ("sajat", NULL))) {

remove_proc_entry ("sajat", NULL);

printk (KERN_NOTICE "/proc/sajat/ letrehozas sikertelen\n"); return -1;

(59)

DRAFT

printk (KERN_NOTICE "/proc/sajat/ letrehozva\n"); if ((sajat_proc_fajl =

create_proc_entry ("taszk_stat", S_IFREG | S_IRUGO, sajat_proc))) {

sajat_proc_fajl->proc_fops = &sajat_fajl_muveletek;

printk (KERN_NOTICE "/proc/sajat/taszk_stat letrehozva\n"); return 0;

} else

{

remove_proc_entry ("taszk_stat", sajat_proc); remove_proc_entry ("sajat", NULL);

printk (KERN_NOTICE "/proc/sajat/taszk_stat letrehozas sikertelen\n"); return -1; } } static void sbejegyzes_exit_module (void) {

remove_proc_entry ("taszk_stat", sajat_proc);

printk (KERN_NOTICE "/proc/sajat/taszk_stat torolve\n"); remove_proc_entry ("sajat", NULL);

printk (KERN_NOTICE "/proc/sajat torolve\n"); }

module_init (sbejegyzes_init_module); module_exit (sbejegyzes_exit_module);

2. The module has been created into the kernel object filesbejegyzes.kousing the commandmake. [norbert@matrica sajat_bejegyzes__modul]$ make

make -C /lib/modules/‘uname -r‘/build M=‘pwd‘ modules

make[1]: Entering directory ‘/usr/src/kernels/2.6.42.12-1.fc15.x86_64’ CC [M] /home/norbert/kernelfa/sajat_bejegyzes__modul/sbejegyzes.o Building modules, stage 2.

MODPOST 1 modules

CC /home/norbert/kernelfa/sajat_bejegyzes__modul/sbejegyzes.mod.o LD [M] /home/norbert/kernelfa/sajat_bejegyzes__modul/sbejegyzes.ko make[1]: Leaving directory ‘/usr/src/kernels/2.6.42.12-1.fc15.x86_64’

We assume that the Makefile is in the same directory assbejegyzes.c. The reader can be familiar with Makefiles for kernel modules from theDocumentation/kbuild/modules.txtof the kernel source tree.

obj-m += sbejegyzes.o all:

make -C /lib/modules/‘uname -r‘/build M=‘pwd‘ modules clean:

make -C /lib/modules/‘uname -r‘/build M=‘pwd‘ clean rm *~

3. The created module can be loaded as root using the commandinsmod sbejegyzes.ko

[root@matrica sajat_bejegyzes__modul]# insmod sbejegyzes.ko

4. We can see your entries under the/proc/sajat. Let’s try it. Enter the commandmore /proc/sajat/taszk_stat

[root@matrica sajat_bejegyzes__modul]# more /proc/sajat/taszk_stat sajat taszk lista (negyedik kernel modul, PROP konyv)

# CMD PID FLAGS ST1 ST2

(60)

DRAFT

2 kthreadd 2 2149613632 1 S 3 ksoftirqd/0 3 2216722496 1 S 4 migration/0 6 2216722496 1 S 5 watchdog/0 7 2216722752 1 S 6 migration/1 8 2216722496 1 S 7 ksoftirqd/1 10 2216722496 1 S 8 watchdog/1 12 2216722752 1 S 9 migration/2 13 2216722496 1 S 10 ksoftirqd/2 15 2216722496 1 S 11 watchdog/2 16 2216722752 1 S 12 migration/3 17 2216722496 1 S 13 ksoftirqd/3 19 2216722496 1 S 14 watchdog/3 20 2216722752 1 S 15 cpuset 21 2216722496 1 S 16 khelper 22 2216722496 1 S 17 kdevtmpfs 23 2149613888 1 S 18 netns 24 2216722496 1 S 19 sync_supers 25 2149613632 1 S 20 bdi-default 26 2157969472 1 S 21 kintegrityd 27 2216722496 1 S 22 kblockd 28 2216722496 1 S 23 ata_sff 29 2216722496 1 S 24 khubd 30 2149580864 1 S ...

Example 3.1The contents of the PCB

The Linux PCB is defined in thestruct task_structstructure in theinclude/linux/sched.hsource file. Investi-gate the elements of the PCB and add further columns to/proc/sajat/taszk_stat.

3.2.1.1 „Celebrating” the source code

The essential part of the work of the module is done bytaszk_listafunction, in which the variableicounts the processes (PCBs) in the system and a struct task_struct * pointertaskis used to iterate through the processes. To be more precise a struct list_head * pointerpis used because the next and back links of the circular doubly linked list of PCBs are organised around of the structure struct list_head as shown in the next figure.

(61)

DRAFT

Figure 3.11: The circular doubly linked list of PCBs in Linux.

Handling linked list in the kernel

The macros inlinux/list.hcan be used to manage the linked list of PCBs. The link member of the PCB list is a list_head structure member that is defined inlinux/types.h.

struct list_head {

struct _{list_head *next, *prev;} };

the usage of these link members were shown in the previous figure. These elements are part of the PCB as it can be seen in the next code snippet cited from the code of PCB fromlinux/sched.h.

struct task_struct {

volatile long state; _{/* -1 unrunnable, 0 runnable, >0 stopped */} ...

struct list_head tasks;

We have used the

list_for_each(pos, head) macro defined inlinux/list.h: /**

* list_for_each - iterate over a list

* @pos: the &struct list_head to use as a loop cursor. * @head: the head for your list.

*/

#define list_for_each(pos, head) \

for (pos = (head)->next; pos != (head); pos = pos->next)

(62)

DRAFT

list_for_each(pos, head)

iterates a struct list_head * pointerposthat is passed as the first parameter through the list from the elementheadthat is passed as the second parameter.

The current macro

It should be noted that in the code snippet

list_for_each (p, current->tasks.next) {

we have used thecurrentmacro that points to the process that is being executed. What is thecurrentmacro exactly? Its definition can be found inasm-generic/current.h

#define get_current() (current_thread_info()->task) #define current get_current()

Thecurrent_thread_infofunction is defined in(arch/x86/include/)asm/thread_info.h. #ifndef __ASSEMBLY__

/* how to get the current stack pointer from C */

register unsigned long current_stack_pointer asm("esp") __used;

/* how to get the thread information struct from C */ static inline struct _{thread_info *current_thread_info(}void) {

return (struct _{thread_info *)}

(current_stack_pointer & ~(THREAD_SIZE - 1)); }

#else _{/* !__ASSEMBLY__ */}

/* how to get the thread information struct from ASM */ #define GET_THREAD_INFO(reg) \

movl $-THREAD_SIZE, reg; \ andl %esp, reg

in the kernel, the thread_union union is on the bottom of the memory area of a process. If we filled the entire stack, then the stack pointer would point to the bottom of the stack and the memory area of thread_union would be overwritten. This union is 8 KB, so we can easily calculate the address of struct thread_info of the running process. The stack pointer’s low 25 bits must be set to zero. It can be seen in the following code snippet from the(arch/x86/include/)asm/ thread_info.h: current_stack_pointer & ~(THREAD_SIZE - 1), where the value ofTHREAD_SIZEis equal 8 KB #define THREAD_SIZE 8192 _{//(2*PAGE_SIZE)}

as can be seen in the filearch/xtensa/include/)asm/thread_info.h. The address of the thread_union and the address of the thread_info are the same:

union thread_union {

struct thread_info thread_info;

unsigned long stack[THREAD_SIZE/sizeof(long)]; };

and thetaskmember of the struct thread_info points to PCB of the process. struct thread_info {

struct task_struct *task; /* main task structure */ See also the exercise at the end of this section.

Returning to the code snippet of the module to be developed, thelist_for_eachmacro iterates through the list of PCBs, and accesses the PCBs using the

(63)

DRAFT

list_entry(ptr, type, member) macro that defined in thelinux/list.hheader. /**

* list_entry - get the struct for this entry * @ptr: the &struct list_head pointer.

* @type: the type of the struct this is embedded in. * @member: the name of the list_struct within the struct. */

The printed debug information also includes a status character indicating the state of the given process. These possible characters are collected in the following macro that is defined inlinux/sched.h.

#define TASK_STATE_TO_CHAR_STR "RSDTtZXxKW"

static int

taszk_lista (struct _{seq_file *m,} void *v) {

int i = 0;

struct _{task_struct *task;} struct _{list_head *p;}

// a kernel/sched/core.c-ben latott modon iratjuk ki // a betut (szemben a fs/proc/array.c-ban lathatoval) // stat_nam[task->state ? __ffs(task->state) + 1 : 0] static const char stat_nam[] = TASK_STATE_TO_CHAR_STR;

seq_puts (m, "sajat taszk lista (negyedik kernel modul, PROP konyv)\n"); seq_printf (m, "%-9s %-16s %-6s %-12s %-6s %-3s\n",

"#", "CMD", "PID", "FLAGS", "ST1", "ST2");

list_for_each (p, current->tasks.next) {

seq_printf (m, "%-9i %-16s %-6i %-12u %-6li %-3c\n", ++i, task->comm, task->pid, task->flags, task->state,

stat_nam[task->state ? __ffs (task->state) + 1 : 0]); }

return 0; }

Making entries under the/procis done by using the next two functions defined inlinux/proc_fs.h extern struct _{proc_dir_entry *create_proc_entry(}const char *name, umode_t mode,

struct _{proc_dir_entry *parent);}

extern struct _{proc_dir_entry *proc_mkdir(}const char *,struct _{proc_dir_entry *);}

of which the first is the more interesting one if ((sajat_proc_fajl =

create_proc_entry ("taszk_stat", S_IFREG | S_IRUGO, sajat_proc)))

For example, according thelinux/stat.h, theS_IRUGO #define S_IRUGO (S_IRUSR|S_IRGRP|S_IROTH)

gives read permission to user (USR), group (GRP) and other (OTH). # ls -l /proc/sajat/taszk_stat

(64)

DRAFT

Example 3.2Playing the game with thecurrentmacro

Modify the third example module (that can be found in the virtualized system) to print the address pointed by thecurrent macro. #include <linux/init.h> #include <linux/module.h> #include <linux/kernel.h> #include <linux/sched.h> #include <linux/list.h> #include <linux/thread_info.h>

MODULE_DESCRIPTION ("Ez a harmadik kernel modulom - modositasa"); MODULE_AUTHOR ("Bátfai Norbert ([email protected])");

MODULE_LICENSE ("GPL"); static int

current_and_sp (void) {

struct _{task_struct *task;} struct _{list_head *p;} struct _{thread_info *ti;}

register unsigned long esp asm ("esp");

ti = (struct _{thread_info *) (esp & ~(}sizeof (union thread_union) - 1)); list_for_each (p, current->tasks.next)

{

printk (KERN_NOTICE "nORBi a kernelben: %p\n", ti->task);

printk (KERN_NOTICE "nORBi a kernelben, current: %p\n", current);

printk (KERN_NOTICE "%s %i %u %li\n", task->comm, task->pid, task->flags, task->state);

} return 0; } static int harmadik_init_module (void) { return current_and_sp (); } static void harmadik_exit_module (void) { current_and_sp (); } module_init (harmadik_init_module); module_exit (harmadik_exit_module);

After compiling and loading/removing the module we can see the following in the kernel logs. [ 8379.261436] nORBi a kernelben: f0f49920

[ 8379.261438] nORBi a kernelben, current: f0f49920 [ 8379.261440] systemd 1 4202752 1

[ 8379.261441] nORBi a kernelben: f0f49920

[ 8379.261442] nORBi a kernelben, current: f0f49920 [ 8379.261443] kthreadd 2 2129984 1

(65)

DRAFT

[ 8379.261444] nORBi a kernelben, current: f0f49920 [ 8379.261445] ksoftirqd/0 3 69238848 1 [ 8379.261446] nORBi a kernelben: f0f49920 . . . [ 8379.261848] nORBi a kernelben: f0f49920

[ 8379.261849] nORBi a kernelben, current: f0f49920 [ 8379.261850] kworker/0:1 2486 69238880 1

[ 8379.261852] nORBi a kernelben, current: f0f49920 [ 8379.261853] flush-253:1 3846 10485824 1

[ 8379.261854] nORBi a kernelben, current: f0f49920 [ 8379.261855] kworker/1:1 6477 69238880 1

[ 8379.261857] nORBi a kernelben, current: f0f49920 [ 8379.261858] insmod 6913 4202752 0

[ 8393.314942] nORBi a kernelben: f0d28000

[ 8393.314945] nORBi a kernelben, current: f0d28000 [ 8393.314947] systemd 1 4202752 1

[ 8393.314949] nORBi a kernelben, current: f0d28000 [ 8393.314951] kthreadd 2 2129984 1

[ 8393.314953] nORBi a kernelben, current: f0d28000 [ 8393.314955] ksoftirqd/0 3 69238848 1 [ 8393.314956] nORBi a kernelben: f0d28000 . . . [ 8393.315565] nORBi a kernelben: f0d28000

[ 8393.315566] nORBi a kernelben, current: f0d28000 [ 8393.315567] bash 1733 4202496 1

[ 8393.315570] nORBi a kernelben, current: f0d28000 [ 8393.315571] kworker/0:1 2486 69238880 1

[ 8393.315573] nORBi a kernelben, current: f0d28000 [ 8393.315574] flush-253:1 3846 10485824 1

[ 8393.315576] nORBi a kernelben, current: f0d28000 [ 8393.315578] kworker/1:1 6477 69238880 1

[ 8393.315580] nORBi a kernelben, current: f0d28000 [ 8393.315582] rmmod 6915 4202752 0

Example 3.3Drawing a memory map

Draw a sketch of the memory to show how to compute the value of thecurrentmacro. See also the Figure 3.2 of the book [ULK], but use the number of the previous exercise.

(66)

DRAFT

Chapter 4

Berkeley socket API, Sys V IPC and I/O

multiplex-ing

(67)

DRAFT

Abstract

In this chapter we are going to create a sample program from the examples of [PP] to help the reader get acquainted with • Berkeley socket API

• I/O multiplexing • forking processes • mutual exclusion • and semaphore arrays.

(68)

DRAFT

„We should select any person from the 1.5 billion inhabitants of the Earth - anyone, anywhere at all. He bet us that, using no more than five individuals, one of whom is a personal acquaintance, he could contact the selected individual using nothing except the network of personal acquaintances.”

—Frigyes Karinthy CHAIN-LINKS (1929, Everything is Different)

„Tessék egy akármilyen meghatározható egyént kijelölni a Föld másfél milliárd lakója közül, bármelyik pontján a Földnek - ˝o fogadást ajánl, hogy legföljebb öt más egyénen keresztül, kik közül az egyik neki személyes ismer˝ose, kapcsolatot tud létesíteni az illet˝ovel, csupa közvetlen - ismeretség - alapon,”

—Karinthy Frigyes Láncszemek

4.1

Berkeley socket API, Sys V IPC, I/O multiplexing

4.2

A simple client/server sample

The first versions of this example were created for the courseHigh level programming 1. These were based on merging the source files of [PP]. This sample program itself is a simple client/server program, its client side is based on thekliens.c of [PP] (see pp. 133). Now, we have complemented the client to wait for characters plus (+) or minus (-) to be typed in the command line.

The code of the server is placed into a source file calledszerver.cthat is more complex than the client. It is based on a parallel and multiplexed example of [PP] (see pp. 123) that uses POSIX signal handling. The subject matter of the server and all other servers in [PP] are substantially identical. They implement a simple echo-like server. In the present example, this functionality is complemented by a counter that counts the actions of clients.

The usage of the sourcesszerver.candkliens.cis shown in the following YouTube video:http://youtu.be/J--bRSmNjZg

4.2.1

The client side

The whole source code of the server is the following. #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <string.h> //#include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #define SZERVER_PORT 2012 #define BUFFER_MERET 256 int kliens (char b) {

int kapu, olvasva;

struct sockaddr_in szerver; char buffer[BUFFER_MERET];

memset ((void *) &szerver, 0, sizeof (szerver)); szerver.sin_family = AF_INET;

inet_aton ("127.0.0.1", &(szerver.sin_addr)); szerver.sin_port = htons (