Fast multiplication and marginalization

First we explain existing techniques described in [Murphy, 2002b; Huang and Darwiche, 1996].

If M ⊂ N then a mapping between φMand φNis simply the set of indices

{A, C} and {A, B, C, D} equals {0, 2}. Note that in general the variables need not appear in the same order in both sets.

To perform multiplication and marginalization we can, for each row in the superset, calculate the superset states using GetStates. Then using a mapping and GetRow we can calculate the equivalent row in the subset, and perform the required operations. As an alternative, we could use GetState instead of GetStates, or a hybrid scheme, however in all cases mapping between the indices of the superset and subset is very expensive.

5.3.1. Performance considerations

In practice when performing multiplication or marginalization, we must con- sider how efficient our sequences of lookups into memory are, and which type of memory we are accessing. In general the smaller the memory resource, the faster the retrieval, so the processor caches are ideal for fast operations. The way we access arrays of data is crucial, preferring sequential-access over random-access, to maximize the number of cache hits.

Consider a subset φMand superset φN. Most algorithms for multiplication

and marginalization access φNsequentially and φMin random order, or vice

versa. Since φNis always at least two times larger than φMit is preferable to

access φNsequentially. Hybrid schemes are also possible, in order to improve

upon the randomness of access to φM. For example, we might perform two

sequential passes of φN visiting even rows in the first pass and odd rows in

the second.

Some of the algorithms for multiplication and marginalization that follow require supporting arrays in addition to the superset and subset. Although the supporting arrays are accessed sequentially, which is fast, they are com- peting with the superset and subset for fast memory resources, which can increase the number of cache misses, thus reducing their effectiveness.

5.3.2. Indexes

For multiplication and marginalization, the task is to map between the equivalent rows in the superset and subset. Table 5.2 illustrates a one- dimensional version of the marginalized table in Table 2.2, and Table 5.3 represents Table 5.1 with an extra column showing the equivalent row in Table 5.2.

Table 5.2.: One-Dimensional Marginal A C Row Value T rue T rue 0 0.0822 T rue F alse 1 0.0178 F alse T rue 2 0.2898 F alse F alse 3 0.6102

The equivalent row column is called an index, and is similar in concept to a database index. Any procedure capable of multiplication or marginalization can be adapted to build an index, as they already compute the equivalent rows.

If we pre-compute this index we can then use it to perform marginalization and multiplication. With this technique we access the superset sequentially, which is preferred, however we must also access the index, albeit sequen- tially, which means extra competition for fast memory resources. We must also take into account the additional memory required to store the index, which has the same number of entries as the superset.

5.3.3. Index cache

Once the index is built it can be stored in an index cache and reused. Note that the same index could be used for both multiplication and marginaliza- tion, and could be used over different sets of tables if their state counts and mapping are equivalent. Inference algorithms use caching schemes of vary- ing complexity, some storing indexes along with internal structures, such as junction-trees, reusing them over a number of queries, and others which store them for very short periods of time within single queries, or a hy- brid approach. There is a clear trade-off between memory consumption and performance.

5.3.4. Index maps

To reduce the substantial memory overhead of an index, we can compute the equivalent entries in the index on the fly by way of two smaller separate indexes which here we call an index map.

Table 5.3.: Equivalent Rows

A B C D Row Equivalent row T rue T rue T rue T rue 0 0 T rue T rue T rue F alse 1 0 T rue T rue F alse T rue 2 1 T rue T rue F alse F alse 3 1 T rue F alse T rue T rue 4 0 T rue F alse T rue F alse 5 0 T rue F alse F alse T rue 6 1 T rue F alse F alse F alse 7 1 F alse T rue T rue T rue 8 2 F alse T rue T rue F alse 9 2 F alse T rue F alse T rue 10 3 F alse T rue F alse F alse 11 3 F alse F alse T rue T rue 12 2 F alse F alse T rue F alse 13 2 F alse F alse F alse T rue 14 3 F alse F alse F alse F alse 15 3

Consider the tables φMand φN with M ⊂ N. The first index, which here

we call the start index, is constructed in the same way as a normal index except that we only iterate over N ∩ M in φN. It therefore contains the

same number of entries as the subset. The second, which we call the offset index, is constructed similarly except that we iterate over N\M (called the difference) in φN.

The idea is that instead of iterating over all of the superset variables to build an index, we build one for the subset variables and one for the difference. When we then perform multiplication or marginalization, we effectively perform the missing iterations that would have been needed to build an index, by iterating over the second index for each value in the first, and sum the two values. The pseudo-code in algorithm 7 illustrates Multiplication using this technique.

An advantage of this approach is that the combined size of the start and offset indexes is much smaller than that of the equivalent index. Not only is the memory overhead reduced, but the time to build them is significantly reduced.

Algorithm 7: Multiply (Index Map)

Input: superset, subset, startIndex, offsetIndex // no need to initialize subset to zero for i=0 ; i < length(startIndex) ; i++ do

startValue = start[i]; sum = 0.0;

for j=0 ; j < length(offsetIndex) ; j++ do sum += superset[startValue + offsetIndex[j]]; subset[i] =sum;

The disadvantages are that we access the subset sequentially, which is fast, but access the superset in random order which is slow. In addition we must access the two index tables which are therefore also competing for fast memory resources.

Once again these index maps can be built using any procedures capable of multiplication or marginalization, and can also be stored in an index cache and reused. In fact the start and offset indexes can be stored in the cache independently and may be re-used separately.