S
tack winding is a powerful technique for assembly programmers that allows us to modify an application's stack to do weird and wonderful things that can be extended into C/C++ with very little work. While the days of writing every line of game code in hand-optimized machine language are over, sometimes it is worth the effort to dip into the realm of the arcane to get that extra bit of speed and elegance in a game.In this gem, we cover one particular type of stack winding that I call the "tempo-rary return." This is the bare minimum form that we will build upon in subsequent examples until we have a thunked temporary return. The code examples have been tested with Microsoft's MASM and Visual C++ compiler. I have personally used stack winding in a number of projects for the GameBoy Color, PC, and Xbox.
Simple TempRet
Stack winding, as its name implies, is a technique for modifying the stack to make it do unexpected things. The term stack winding comes from the idea of inserting values in an existing stack frame to change its normal and expected behavior.
Listing 1.13.1 The TempRet routine
0
16
In Listing 1.13.1, we see the first building block of stack winding: the TempRet rou-tine. Let's take a function (call it MyFunc) and say it calls _TempRetEg. The latter then calls two functions: fnO and fnl. It then hits the lines:
pop edx call edx
Now we know that the way the CPU handles the assembly CALL instruction on line 24 is to push the address of the next line (25) and execute a JUMP to line 8. Line 15 pops that address off the stack and stores it in a CPU register. Now we CALL that address. This pushes line 20 onto the stack and executes a JUMP to line 25. The lat-ter does nothing but execute a CPU return, which pops an address off the stack and jumps there.
The rest of _TempRetEg then continues and when it returns, we do not return to MyFunc but to whatever function called MyFunc in the first place. It is an interesting little trick, but why would it be important? The power comes when we consider the functions FNO through FN3.
Let's say that FNO opens a file, FN1 allocates a buffer and reads the file into memory, FN2 frees that memory, and FN3 closes the file. Thus, MyFunc no longer has to worry about the release steps. It doesn't have to close the file or worry about freeing up the memory associated with the file. Functionally the process of opening a file, reading it into memory, freeing that memory, and closing the file is all contained within a single block of code. MyFunc only has to call _TempRetEg, use the buffer, and return.
TempRet Chains
The TempRet example comes of age when we chain functions together. Let's take a classic problem: the initialization and destruction of DirectX 7. This usually takes a number of steps, but it's incredibly important to release the components of DX in reverse order, which can sometimes become horribly complicated.
So, let's expand our first example to illustrate this:
Listing 1.13.2 Winding multiple routines onto the stack
10 ; open the window 11 TempRet
12 ; close it 13 net
14 setCooperativeLevel:
15 ; set to exclusive 16 TempRet
25 ; create primary surface 26 ; get attached back 27 TempRet
By performing numerous TempRets in succession we effectively have wound four routines onto the stack so that when _SomeUserRunFunc returns, we will bounce back through createSurfaces, changeDisplayMode, setCooperativeLevel, and cre-ateWindow at the line after the TempRet in reverse order.
So far, we've been using assembly language, but it's not necessary to write assem-bly modules to use this technique. We will cover two mechanisms in Microsoft's Visual C++ in the final section that aid us in stack winding: inline assembly and naked functions.
Thunking
The ideas discussed so far need to be translated into C/C++. As stated previously, Visual C++ has a handy mechanism for doing diis, but what about other compilers? If naked functions are not supported, then we will have to dip into assembly language because the presence of a stack frame really complicates things. It is not impossible, just difficult.
Thunking is a technique popularized by Microsoft for slipping a piece of code between two others. In effect, program flow is hurtling along through our code until thunk!—it crashes into that layer. Thunks are a great way of implementing a stack-winding paradigm in C++. Let's look at an example that performs the same task of set-ting up DirectX as we saw earlier:
Listing 1.13.3 Visual C++ example using TempRet
#define TempRet\
_ asm{pop edx}\
__ asm{call edx}
tfdefine NAKED void _ declspec(naked) tfdefine JUMP _ asm jmp
tfdefine RET _ asm ret static NAKED createWindow(){
// open the window TempRet
// close it RET
static NAKED setCooperativeLevel(){
// set to exclusive TempRet
// restore RET
static NAKED changeDisplayMode(){
// set 640x480 16 bpp TempRet
// restore RET
static NAKED createSurfaces(){
// create primary surface // get attached back TempRet
// restore RET
NAKED SetUpDX7(){
createWindow();
setCooperativeLevel();
changeDisplayMode();
createSurfaces();
JUMP run
Recursion
As a final example of the power of stack winding, we will explore a solution for a clas-sic problem with recursive searching: how to roll back die recursion. In regular C we would simply return repeatedly, walking back through the stack until we reach the top. If our recursion is over 100 calls deep, however, this might take a little time. To fix this, here is a pair of utility functions called SafeEnter. Incidentally, the code works just as well from a C++ object as a global function.
Listing 1.13.4 The SafeEnter and SafeExit functions that aid recursion
.586
.model flat .code
public SafeEnter,SafeExit
; struct SAFE{
; void*_reg[8];
; void* ret;
I }
; assembly for SafeEnter routine _SafeEnter:
pop edx ; return address mov eax,[esp] ; safe
mov [eax].safe. ret,edx mov [eax].safe. ebx.ebx n)ov [eax].safe. ebp,ebp mov [eax].safe. esp,esp mov [eax].safe. esi,esi mov [eax].safe. edi.edi
pop eax ; safe pointer pop edx ; call function push eax ; safe pointer
mov ebp,eax call edx mov eax,ebp jmp sex _SafeExit:
pop edx ; return pop eax ; regs context
mov edi,[eax].safe. edi mov esi,[eax].safe. esi mov esp,[eax].safe. esp mov ebp,[eax].safe. ebp mov ebx,[eax].safe. ebx mov edx,[eax].safe. ret mov eax,[eax].safe. eax
jmp edx end
SafeEnter works by saving off to a SAFE structure a copy of crucial CPU regis-ters. It then calls our recursive function. As far as the function is concerned, no extra work is necessary. Now the cool part comes when we find the piece of data we're look-ing for. We simply call SafeExit() and pass it the register context we built earlier. We are instantly transported back to the parent function.
Now, if the unthinkable happened and the search routine did not meet its search criteria, then the function can simply return in the normal way, all the way up the chain.
Listing 1.13.5 Recursive example using SafeEnter and SafeExit static void search(SAFE&safe,void*v){
if(<meets_requirement>) SafeExit(safe);
// do stuff search(safe,v);
return;
}
int main(){
SAFE safe;
SafeEnter(
safe, search,
<some_pointer>)