Building Programs with Make. What does it mean to Build a Program?

(1)

Building Programs with

Make

John Pormann, Ph.D.

jbp1@duke.edu http://www.oit.duke.edu/scsc/

http://wiki.duke.edu/display/SCSC scsc@duke.edu

What does it mean to “Build” a Program?

Often, a single executable program is compiled and linked from multiple source-code files and multiple libraries

We could type:

But this would be:

• A lot to type each time (and typos will lead to errors)

• Inefficient -- regardless of what files may have changed, all 4 *.c files will be re-compiled each time

• What if file3.c calls functions in file4.c (that are changing) ?

• What if file1.c cannot be compiled with -O ?

• What if we want to split file2.c into file2a.c and file2b.c ?

• What if we don’t “control” file4.c, we have to copy it from someone else?

% gcc -O -o prog.exe file1.c file2.c file3.c file4.c -llib1 -llib2

(2)

Program-File Dependencies

We can think of the final program as being “dependent” on the source code files and libraries

If a file changes, the dependent pieces must be re-compiled

file1.c

file2.c

file3.c

file4.c

lib1.a

lib2.a

prog.exe

Dependencies, cont’d

Digging deeper, we know that C-code is compiled to Obj-code; then the Obj-code is linked (with libraries) into the EXE-code

file1.o

file2.o file3.o

file4.o

lib1.a

lib2.a

prog.exe file2.c

file1.c

file3.c

file4.c

(3)

Efficiency

Now, if file1.c changes, we see that file1.o must be rebuilt (recompiled)

And because file1.o changes, prog.exe must also be rebuilt ... but (existing) file2.o, file3.o, and file4.o are reused in the link- stage for prog.exe

• MUCH FASTER !!

We can also do an efficient rebuild with the more complex relationships (file3/file4)

... but we have to be careful to explain what the relationship is

Including Header Files

file1.o

file2.o file3.o

file4.o

lib1.a

lib2.a file2.c

file1.c

file3.c

file4.c file3n4.h

Global header files can be problematic

(4)

The Make Program

Unix/Linux include a utility called ‘make’ that can help build your program, given a set of dependency criteria

You may have used this with other open-source projects

‘configure’ is another tool that can be used to create a “makefile”

for different computers/architectures/operating systems

‘make’ will build the library or executable

• You will probably see a lot of ‘gcc’ commands fly by

% ./configure

% make

Make Program, cont’d

By default, ‘make’ will look for a file called ‘makefile’ or ‘Makefile’

in the current directory

You can also specify another filename with ‘-f myprog.mk’

The ‘makefile’ will define “rules” and dependencies

“rules” allow for default operations on some class of file-types

• E.g. defines basic compilation steps for .c to be converted to .o

“dependencies” identify which files have to be re-built if other files are touched; and what command is used to build them

You can also include other ‘makefiles’ in the current one

• Useful for OS-specific customization

Also allows you to define and use environment variables

(5)

Makefile Example

Note: the “action” lines MUST start with a tab

In this example, no defaults are used, no simplifications are made This level of detail can be tedious if your program uses 10’s or 100’s of files

# comment lines

# can be added for clarity

prog.exe: file1.o file2.o file3.o file4.o

gcc -o prog.exe file1.o file2.o file3.o file4.o -llib1 -llib2 file1.o: file1.c

gcc -o file1.o -c file1.c file2.o: file2.c

gcc -o file4.o -c file4.c

By default, builds the first

“target” in the makefile

Makefile Example, cont’d

To build the program, we just type:

You’ll see a lot of ‘gcc’ lines go by:

That’s it, your program is ready to run What if you edit ‘file2.c’ ??

% make

gcc -o file1.o -c file1.c gcc -o file2.o -c file2.c gcc -o file3.o -c file3.c gcc -o file4.o -c file4.c

gcc -o prog.exe file1.o file2.o file3.o file4.o -llib1 -llib2

% make

(6)

A Quick Warning

Make is reasonably intelligent

It will not build files that it does not think need to be built

It looks at the Unix file-access times to determine if a given .c and .o file are “the same”

But ‘make’ only knows what the OS tells it !!

• File-access times can sometimes be wrong when you scp/sftp a file from another system

• It is easy to forget about header files in your dependency lists

 If a function changes arguments, some compiled files may be using the old arguments, and some use the new arguments

If you’re seeing strange behavior, try ‘rm *.o’ and start over

• This is often put in the makefile as a target named ‘clean’

Multiple “Targets” in a Makefile

A single makefile can hold commands for multiple “target” outputs E.g. prog1.exe and prog2.exe use lots of similar functions and files, so they can be both put into a single makefile

To identify which one to use:

# ...

prog1.exe: file1.o file2.o

$(CC) $(LFLAGS) -o prog1.exe file1.o file2.o prog2.exe: file1.o file3.o

$(CC) $(LFLAGS) -o prog2.exe file1.o file3.o

# ...

% make prog1.exe

% make prog2.exe

(7)

Multiple “Targets”, cont’d

It is often useful to add “Phony” targets which only group related build items

# ...

.PHONY: all

# ...

all: prog1.exe prog2.exe

# really nothing to do for ‘all’,

# just build prog1 and prog2 prog1.exe: file1.o file2.o

$(CC) $(LFLAGS) -o prog1.exe file1.o file2.o prog2.exe: file1.o file3.o

$(CC) $(LFLAGS) -o prog2.exe file1.o file3.o clean:

rm *.o prog1.exe prog2.exe

# ... % make clean

% make prog1.exe

% make prog2.exe

% make

% make all

More Complex Commands

If you need to do more work for a given build-target, you can put them on subsequent, tabbed lines:

# basic makefile

# file3 gets no optimization flag

# and we’ll print the diff relative to

# some previous copy of the file file3.o: file3.c

diff file3.c file3_orig.c

$(CC) $(DEF) -o file3.o -c file3.c

# otherwise, compilation will be optimized .c.o:

$(CC) $(CFLAGS) -o $@ -c $<

(8)

Compiler Arguments/Flags

What about compiler options/flags/arguments?

If needed, you can always include them, as usual, in the ‘gcc’ lines for each individual file:

# comment lines

# can be added for clarity

gcc -O -o prog.exe file1.o file2.o file3.o file4.o -llib1 -llib2 file1.o: file1.c

gcc -O -D_MY_C_DEFINE -o file1.o -c file1.c

# file2 has lots of small functions, let’s inline them for speed file2.o: file2.c

gcc -O -finline-functions -o file2.o -c file2.c

# file3 will crash with -O file3.o: file3.c

gcc -O -o file4.o -c file4.c

Again, for lots of files this can get tedious.

Simplifying the Makefile

IF all (or most) of your files are compiled the same way, you can define a default “.c.o” rule

I.e. how should ‘make’ convert a .c file to a .o file?

In the “.c.o” rule, we use $@ for the name of the target .o file, and

$< for the name of the input file

You can define “.f.o” for Fortran, “.cpp.o” or “.cc.o” for C++

• You can even create your own suffixes if needed

# basic makefile

gcc -o prog.exe file1.o file2.o file3.o file4.o -llib1 -llib2 .c.o:

gcc -o $@ -c $<

(9)

Simplifying, the Makefile cont’d

IF a dependency line exists for a specific file, then that line is used and NOT the default rule

So you can define a base compilation, then override when needed

Running ‘make’:

# basic makefile

# file3 gets no optimization flag file3.o: file3.c

gcc -O -o $@ -c $<

% make -f makefile2 gcc -O -o file1.o -c file1.c gcc -O -o file2.o -c file2.c gcc -o file3.o -c file3.c gcc -O -o file4.o -c file4.c

gcc -o prog.exe file1.o file2.o file3.o file4.o -llib1 -llib2 Note, we’ve now added ‘-O’ to ALL default compilations!

Common Options/Flags

If you use the same set of compiler flags a lot, you can declare them in a variable and re-use them through the variable name

This makes it easy to change one line and recompile everything for debug or production code

# basic makefile

CFLAGS = -O -D_MY_C_DEFINE LFLAGS = -O

gcc $(LFLAGS) -o prog.exe file1.o file2.o file3.o file4.o \ -llib1 -llib2

gcc $(CFLAGS) -o $@ -c $<

Continues on next line

(10)

Common Options/Flags, cont’d

To change to “debug” mode, we only modify the top lines of the makefile:

Note that ‘make’ will NOT realize that you need to recompile everything now (for the ‘-g’ to take effect everywhere)

# basic makefile

CFLAGS = -g -D_MY_C_DEFINE LFLAGS = -g

gcc $(LFLAGS) -o prog.exe file1.o file2.o file3.o file4.o \ -llib1 -llib2

gcc $(CFLAGS) -o $@ -c $<

A “Usual” Makefile

# basic makefile CC = gcc OPT = -O

DEF = -D_MY_C_DEFINE CFLAGS = $(OPT) $(DEF) LFLAGS = -O

OBJS = file1.o file2.o file3.o file4.o LIBS = -llib1 -llib2

prog.exe: $(OBJS)

$(CC) $(LFLAGS) -o prog.exe $(OBJS) $(LIBS)

$(CC) $(DEF) -o file3.o -c file3.c

$(CC) $(CFLAGS) -o $@ -c $<

(11)

A “Usual” Makefile, cont’d

Why use $(OPT) and $(DEF) and $(CFLAGS) ?

What if -D_MY_C_DEFINE changes?

• We’d have to change it in two location in the makefile

• ... will you really remember to do that?

• In previous makefile:

 Everything above “OBJS =” is flexible

 Everything below “OBJS =” is fixed

. . .

$(CC) -D_MY_C_DEFINE -o file3.o -c file3.c

$(CC) $(CFLAGS) -o $@ -c $<

Including other Makefiles

An makefile may include other makefiles or makefile-fragments

# basic makefile include ./debug.mk

#include ./optimize.mk

prog.exe: $(OBJS)

$(CC) -o file3.o -c file3.c

$(CC) $(CFLAGS) -o $@ -c $<

# debug.mk # optimize.mk

(12)

Cross-Platform Builds

‘make’ can actually handle cross-platform builds fairly smoothly Although you need some help (from the OS or the user) to pick a good set of options

# header for gcc on Linux CC = gcc

# header for cc on SGI CC = cc

CFLAGS = -fast -D_MY_OS=IRIX LFLAGS = -fast

# header for Intel compilers CC = icc

CFLAGS = -O3 -D_MY_OS=LINUX

LFLAGS = -O3 It is often easy to

compartmentalize the changes

Cross-Platform Builds, cont’d

# basic makefile

# OS-specific stuff (CC, CFLAGS) defined in

# separate file include ./os_stuff.mk

prog.exe: $(OBJS)

$(CC) -o file3.o -c file3.c

$(CC) $(CFLAGS) -o $@ -c $<

YOU have to set os_stuff.mk properly (or copy from one of several pre-defined files)

(13)

Recursive Builds

You can also have ‘make’ enter a subdirectory and do a ‘make’

Useful for projects that require libraries that require libraries ...

Why the parens (...) ?

• If you cd into a subdirectory, you would have to remember to cd out

 And what if one of the commands inside that subdirectory do a cd and forget to “cd ..”?

• The parens execute the commands, then return to the original directory

 Usually separate commands by semi-colons

# top-level makefile prog.exe:

( cd lib1 ; make ) ( cd lib2 ; make )

$(CC) -o prog.exe lib1/lib1.a lib2/lib2.a

Recursive Builds, cont’d

topdir

lib1

lib2

# topdir makefile all:

( cd lib1; make ) ( cd lib2; make )

# lib1 makefile lib1: lib11.o lib12.o

$(CC) -o lib1.a lib11.o lib12.o .c.o:

$(CC) -o $@ -c $<

# lib2 makefile lib2: lib21.o lib22.o

(14)

Putting It All Together

How to do multiple targets, multiple subdirectories, cross-platform program building?

Top-level directory

Makefile - has targets like ‘all’ and ‘clean’, but doesn’t really do the work

buildenv.mk - includes OS-specific variables, copied from one of several build_linux.mk, build_sgi.mk, build_intel.mk, etc.

buildtype.mk - includes production vs. debug variables, copied from one of build_opt.mk or build_debug.mk

Component-specific subdirectories

Makefile - has targets for program-components, files, etc.

• Each one includes ../buildenv.mk and ../buildtype.mk

 For sub-sub-dirs, include ../../buildenv.mk, etc.

Putting It All Together, cont’d

topdir

lib1

lib2

# topdir makefile include ./buildenv.mk include ./buildtype.mk all:

( cd lib1; make ) ( cd lib2; make ) clean:

( cd lib1; make clean ) ( cd lib2; make clean ) rm *.o

# lib1 makefile include ../buildenv.mk include ../buildtype.mk lib1: lib11.o lib12.o

$(CC) $(LFLAGS) -o lib1.a lib11.o lib12.o clean:

rm *.o .c.o:

# build_linux.mk CC = gcc OPT = -O DEBUG = -g

# build_debug.mk CFLAGS = $(DEBUG) LFLAGS = $(DEBUG)

# build_opt.mk CFLAGS = $(OPT) LFLAGS = $(OPT)

# build_intel.mk CC = icc OPT = -fast DEBUG = -g buildenv.mk

buildtype.mk

(15)

Suffix Rules

Make supports a lot of standard compilations as suffix rules .c.o .cpp.o .f.o

But this is somewhat OS- or version-specific

You can create your own using the special variable .SUFFIXES

.SUFFIXES: .f77 .java .class . . .

.f77.o:

$(F77) -o $@ -c $<

.java.class:

$(JAVAC) -o $@ -c $<

Other Makefile Tricks

Adding an ‘@’ sign to the beginning of a command hides the command

Adding a ‘-’ sign to the beginning of a command ignores errors

# NOTE: this is not a .phony target!!

all: prog1.exe prog2.exe

@echo All done with build

Without the ‘@’, make would print the line ‘echo ...’ then execute the

line, which would print the ‘...’

# remove old files clean:

-rm prog.exe *.o

Without the ‘-’, make might see an error from ‘rm’ if ‘prog.exe’

did not already exist

(16)

Non-Programming Uses for Make

While ‘make’ is particularly useful for building programs, it really is just a dependency/rule processing engine

You can use it to automate many other tasks Perl-documentation to Man page conversion:

.SUFFIXES: .pod .3 .html

MAN_FILES = abc.3 def.3 ghi.3

HTML_FILES = abc.html def.html ghi.html

final_man: $(MAN_FILES) echo DONE with MAN final_html: $(HTML_FILES)

echo DONE with HTML

.pod.3:

pod2man $<

.pod.html:

pod2html $<

Building Programs with Make. What does it mean to Build a Program?