When searching for a Lua file,require uses a path that is a little different from typical paths. The typical path is a list of directories wherein to search for a given file. However, ANSI C (the abstract platform where Lua runs) does not have the concept of directories. Therefore, the path used by require is a list of templates, each of them specifying an alternative way to transform a module name (the argument torequire) into a file name. More specifically, each template in the path is a file name containing optional question marks. For each template, require replaces the module name for each ‘?’ and checks whether there is a file with the resulting name; if not, it goes to the next template. The templates in a path are separated by semicolons (a character seldom used for file names in most operating systems). For instance, if the path is
?;?.lua;c:\windows\?;/usr/local/lua/?/?.lua
then the callrequire"sql" will try to open the following Lua files:
sqlsql.lua
c:\windows\sql
/usr/local/lua/sql/sql.lua
Therequire function assumes only the semicolon (as the component separator) and the question mark; everything else, including directory separators and file extensions, is defined by the path itself.
The path thatrequire uses to search for Lua files is always the current value of variablepackage.path. When Lua starts, it initializes this variable with the value of the environment variableLUA_PATH_5_2. If this environment variable is not defined, Lua tries the environment variable LUA_PATH. If both are not defined, Lua uses a compiled-defined default path.2 When using the value of an
2In Lua 5.2, the command-line option-E prevents the use of those environment variables and forces the default.
15.1 Therequire Function 155
environment variable, Lua substitutes the default path for any substring “;;”.
For instance, if you setLUA_PATH_5_2 to “mydir/?.lua;;”, the final path will be the template “mydir/?.lua” followed by the default path.
The path used to search for a C library works exactly in the same way, but its value comes from variable package.cpath (instead of package.path).
Similarly, this variable gets its initial value from the environment variables LUA_CPATH_5_2 or LUA_CPATH. A typical value for this path in UNIX is like this:
./?.so;/usr/local/lib/lua/5.2/?.so
Note that the path defines the file extension. The previous example uses.so for all templates; in Windows, a typical path would be more like this one:
.\?.dll;C:\Program Files\Lua502\dll\?.dll
Functionpackage.searchpath encodes all those rules for searching libraries.
It receives a module name and a path, and looks for a file following the rules described here. It returns either the name of the first file that exists or nil plus an error message describing all files it unsuccessfully tried to open, as in the next example:
> path = ".\\?.dll;C:\\Program Files\\Lua502\\dll\\?.dll"
> print(package.searchpath("X", path)) nilno file '.\X.dll'
no file 'C:\Program Files\Lua502\dll\X.dll'
Searchers
In reality,require is a little more complex than we have described. The search for a Lua file and the search for a C library are just two instances of a more general concept of searchers. A searcher is simply a function that receives the module name and returns either a loader for that module or nil if it cannot find one.
The arraypackage.searchers lists the searchers that require uses. When looking for a module,require calls each searcher in the list passing the module name, until one of them finds a loader for the module. If the list ends without a positive response,require raises an error.
The use of a list to drive the search for a module allows great flexibility to require. For instance, if you want to store modules compressed in zip files, you only need to provide a proper searcher function for that and add it to the list.
However, more often than not, programs do not change the default contents of package.searchers. In this default configuration, the searcher for Lua files and the searcher for C libraries that we described earlier are respectively the second and the third elements in the list. Before them, there is the preload searcher.
The preload searcher allows the definition of an arbitrary function to load a module. It uses a table, calledpackage.preload, to map module names to loader functions. When searching for a module name, this searcher simply looks for the
given name in the table. If it finds a function there, it returns this function as the module loader. Otherwise, it returns nil. This searcher provides a generic method to handle some non-conventional situations. For instance, a C library statically linked to Lua can register itsluaopen_ function into the preload table, so that it will be called only when (and if) the user requires that module. In this way, the program does not waste time opening the module if it is not used.
The default content ofpackage.searchers includes a fourth function that is relevant only for submodules. We will discuss it at Section 15.4.
15.2 The Basic Approach for Writing Modules in Lua
The simplest way to create a module in Lua is really simple: we create a table, put all functions we want to export inside it, and return this table. Listing 15.1 illustrates this approach. Note how we defineinv as a private function simply by declaring it local to the chunk.
Some people do not like the final return statement. One way of eliminating it is to assign the module table directly intopackage.loaded:
local M = {}
package.loaded[...] = M
<as before>
Remember thatrequire calls the loader passing the module name as the first argument. So, the vararg expression ... in the index results in that name.
After this assignment, we do not need to return M at the end of the module:
if a module does not return a value, require will return the current value of package.loaded[modname] (if it is not nil). Anyway, I still prefer to write the final return, because it looks clearer.
Another variant for writing a module is to define all functions as locals and build the returning table at the end, as in Listing 15.2. What are the advantages of this approach? You do not need to prefix each name with M.
or something similar; there is an explicit export list; and you define and use exported and internal functions in the same way inside the module. What are the disadvantages? The export list is at the end of the module instead of at the beginning, where it would be more useful as a quick documentation; and the export list is somewhat redundant, as you must write each name twice.
(This last disadvantage may become an advantage, as it allows functions to have different names inside and outside the module, but I think programmers seldom do this.) I particularly like this style, but tastes may differ.
Anyway, remember that no matter how a module is defined, users should be able to use it in a standard way:
local cpx = require "complex"
print(cpx.tostring(cpx.add(cpx.new(3,4), cpx.i))) --> (3,5)
15.2 The Basic Approach for Writing Modules in Lua 157
Listing 15.1. A simple module for complex numbers:
local M = {}
function M.new (r, i) return {r=r, i=i} end -- defines constant 'i'
M.i = M.new(0, 1) function M.add (c1, c2)
return M.new(c1.r + c2.r, c1.i + c2.i) end
function M.sub (c1, c2)
return M.new(c1.r - c2.r, c1.i - c2.i) end
function M.mul (c1, c2)
return M.new(c1.r*c2.r - c1.i*c2.i, c1.r*c2.i + c1.i*c2.r) end
local function inv (c) local n = c.r^2 + c.i^2 return M.new(c.r/n, -c.i/n) end
function M.div (c1, c2) return M.mul(c1, inv(c2)) end
function M.tostring (c)
return "(" .. c.r .. "," .. c.i .. ")"
end return M
Listing 15.2. Module with export list:
local function new (r, i) return {r=r, i=i} end -- defines constant 'i'
local i = complex.new(0, 1)
<other functions follow the same pattern>
return {
new = new,
i = i,
add = add,
sub = sub,
mul = mul,
div = div,
tostring = tostring, }
15.3 Using Environments
One drawback of those basic methods for creating modules is that it is all too easy to pollute the global space, for instance by forgetting a local in a private declaration.
Environments offer an interesting technique for creating modules that solves this problem. Once the module main chunk has an exclusive environment, not only all its functions share this table, but also all its global variables go to this table. Therefore, we can declare all public functions as global variables and they will go to a separate table automatically. All the module has to do is to assign this table to the_ENV variable. After that, when we declare function add, it goes toM.add:
local M = {}
_ENV = M
function add (c1, c2)
return new(c1.r + c2.r, c1.i + c2.i) end
Moreover, we can call other functions from the same module without any prefix.
In the previous code,add gets new from its environment, that is, it calls M.new.
This method offers a good support for modules, with little extra work for the programmer. It needs no prefixes at all. There is no difference between calling an exported and a private function. If the programmer forgets a local, he does not pollute the global namespace; instead, a private function simply becomes public.
Nevertheless, currently I still prefer one of the two methods that we dis-cussed in the previous section. They may need more work, but the resulting
15.4 Submodules and Packages 159
code states clearly what it does. To avoid creating a global by mistake, I use the simple method of assigning nil to_ENV. After that, any assignment to a global name will raise an error.
What is missing, of course, is access to other modules. Once we change the value of_ENV, we lose access to all previous global variables. There are several ways to recover this access, each with its pros and cons.
One solution is inheritance:
local M = {}
setmetatable(M, {__index = _G}) _ENV = M
(You must callsetmetatable before assigning to _ENV; why?) With this construc-tion, the module has direct access to any global identifier, paying a small over-head for each access. A funny consequence of this solution is that, conceptually, your module now contains all global variables. For instance, someone using your module can call the standard sine function by writing complex.math.sin(x).
(Perl’s package system has this peculiarity, too.)
Another quick method of accessing other modules is to declare a local vari-able that holds the original environment:
local M = {}
local _G = _G
_ENV = M -- or _ENV = nil
Now you must prefix any global name with_G., but the access is a little faster, because there is no metamethod involved.
A more disciplined approach is to declare as locals only the functions you need or, at most, the modules you need:
-- module setup local M = {}
-- Import Section:
-- declare everything this module needs from outside local sqrt = math.sqrt
local io = io
-- no more external access after this point _ENV = nil -- or _ENV = M
This technique demands more work, but it documents your module dependencies better. It also results in code that runs a little faster than code with the previous schemes, because of its usage of local variables.
15.4 Submodules and Packages
Lua allows module names to be hierarchical, using a dot to separate name levels.
For instance, a module namedmod.sub is a submodule of mod. A package is a complete tree of modules; it is the unit of distribution in Lua.
When you require a module called mod.sub, require queries first the table package.loaded and then the table package.preload using the original module name “mod.sub” as the key; here, the dot is just a character like any other in the module name.
However, when searching for a file that defines that submodule, require translates the dot into another character, usually the system’s directory separa-tor (e.g., ‘/’ for UNIX or ‘\’ for Windows). After the translation, require searches for the resulting name like any other name. For instance, assume ‘/’ as the directory separator and the following path:
./?.lua;/usr/local/lua/?.lua;/usr/local/lua/?/init.lua The callrequire"a.b" will try to open the following files:
./a/b.lua
/usr/local/lua/a/b.lua /usr/local/lua/a/b/init.lua
This behavior allows all modules of a package to live in a single directory. For instance, if a package has modulesp, p.a, and p.b, their respective files can be p/init.lua, p/a.lua, and p/b.lua, with the directory p within some appropriate directory.
The directory separator used by Lua is configured at compile time and can be any string (remember, Lua knows nothing about directories). For instance, sys-tems without hierarchical directories can use a ‘_’ as the “directory” separator, so thatrequire"a.b" will search for a file a_b.lua.
Names in C cannot contain dots, so a C library for submodulea.b cannot ex-port a functionluaopen_a.b. Here require translates the dot into another char-acter, an underscore. So, a C library nameda.b should name its initialization functionluaopen_a_b. We can use the hyphen trick here too, with some subtle results. For instance, if we have a C librarya and we want to make it a submod-ule ofmod, we can rename the file to mod/v-a. When we write require"mod.v-a", require correctly finds the new file mod/v-a as well as the function luaopen_a inside it.
As an extra facility,require has one more searcher for loading C submodules.
When it cannot find either a Lua file or a C file for a submodule, this last searcher searches again the C path, but this time looking for the package name.
For example, if the program requires a submodulea.b.c this searcher will look fora. If it finds a C library for this name, then require looks into this library for an appropriate open function,luaopen_a_b_c in this example. This facility allows a distribution to put several submodules together into a single C library, each with its own open function.
From the point of view of Lua, submodules in the same package have no explicit relationship. Requiring a module a does not automatically load any of its submodules; similarly, requiring a.b does not automatically load a. Of course, the package implementer is free to create these links if she wants. For instance, a particular modulea may start by explicitly requiring one or all of its submodules.
Exercises 161
Exercises
Exercise 15.1: Rewrite the code in Listing 13.1 as a proper module.
Exercise 15.2: What happens in the search for a library if the path has some fixed component (that is, a component without a question mark)? Can this behavior be useful?
Exercise 15.3: Write a searcher that searches for Lua files and C libraries at the same time. For instance, the path used for this searcher could be something like this:
./?.lua;./?.so;/usr/lib/lua5.2/?.so;/usr/share/lua5.2/?.lua (Hint: usepackage.searchpath to find a proper file and then try to load it, first withloadfile and next with package.loadlib.)
Exercise 15.4: What happens if you set a metatable for tablepackage.preload with an__index metamethod? Can this behavior be useful?