5.9 Experimental Evaluation
5.9.6 Operation
We executed our algorithms and the two baseline comparisons on an Intel Core 2 Q6600 quad core processor with 3GB of available memory. We repeated the execution of the algorithms 5 times and present the average of the computation times in our results.
Fge-CNF Evaluation Both FGE-BDD and FGE-CNF operate on input derived from
dependencies—constraints expressed as propositional formulas and not as a set of configurations and constructs a feature graph. FGE-BDD was implemented in Java
and uses the JavaBDD 1.0b2 library1. FGE-CNF was implemented using the Scala
programming language[OAC+06] with the core SAT4J library2. SAT4J is a widely used open-source Java interface to SAT solvers that implements the initial Minisat specifi- cation[ES03]. Using Scala for FGE-CNF was a choice based on language preference
instead of a performance advantage. Both Java and Scala run on the Java Virtual Machine (JVM). The performance of the PIG algorithm for computing OR-groups in
FGE-CNF is heavily dependent on the expense of forward and backward subsumption
and we have implemented the algorithms by Zhang[Zha05].
1http://javabdd.sourceforge.net/ 2http://sat4j.org/
For input derived from the Linux variability model, both FGE-BDD and FGE-CNF were
not able to computeOR-groups. We were unable to construct a BDD representing the
constraints of the variability model within our memory constraints. FGE-CNF timed
out when computing OR-groups. We used the alternative XOR-group computation
(Section5.3) in order to compute XOR-groups for FGE-CNF without a dependency on OR-groups.
Variable ordering affects the performance and space consumption of both SAT solvers and BDDs. We performed some limited experiments with the variable ordering heuris- tics for BDDs as proposed by Mendonca[Men09]. However, even with Mendonca’s
heuristics, we were still unable to build a BDD for the input derived from the Linux kernel. We did not use any specific variable ordering for FGE-CNF.
We measured the computation times for computing the implication graph, mutex graph, andOR-groups. We used JavaBDD 1.0b2 for the BDD-based implementation and set the
timeout to 90 seconds for FGE-BDD. For any computation that timed out on FGE-BDD,
we recorded a time of 90 seconds.
Fge-DNF Evaluation We evaluate FGE-DNF against the formal concept analysis (FCA)-
based approach by Ryssel et al.[RPK11]. We presented the algorithm according to
the structure of the FGE algorithm as FGE-FCA in Section 5.7. Both FGE-DNF and
FGE-FCA constructs an identical feature graph and use a set of configurations as input.
While the inputs and output is the same for FGE-DNF and FGE-FCA, the procedure for
computing feature groups is vastly different. Whereas FGE-DNF identifies feature groups
by translating a DNF formula to a BIP problem, FGE-FCA identifies feature groups by
translating a formal context to a set cover problem. Both BIP and set cover problems are NP-hard, thus are in the same class of theoretical complexity. However, as often the case with NP-hard problems, the performance of these algorithms will differ in practical scenarios.
The approaches for identifying dead features, implications, and mutual exclusions are essentially equivalent in both FGE-DNF and FGE-FCA. For dead features, both approaches
search for features that appear in no configurations. In FGE-DNF, implications and
mutual exclusions are computed by checking for the presence of a pair of features across all terms inϕ. FGE-FCA operates on the same principle by checking whether
the configurations of one feature are a subset of another. Since both approaches are equivalent for these components, we focus the evaluation on the computation ofOR-
groups where the approaches are quite different. Furthermore,OR-group computation
FGE-DNF was implemented in Java using the core SAT4J library. Our implementation of
FGE-DNF uses the PRIME algorithm to solve the BIP problem for identifyingOR-groups.
PRIME is an extension by Andersen [And09] of the MIN_PRIME algorithm [EPP95].
PRIME incrementally enumerates all prime implicants of a CNF formula. PRIME first
finds a prime implicant using MIN_PRIME, then adds it as a constraint to find the
next prime implicant. This process repeats until all prime implicants are found. An advantage of using SAT4J for our implementation is that it natively supports the cardinality constraints needed for PRIME. We did not use a specific variable ordering
for FGE-DNF. FGE-FCA was implemented by Ryssel in Java and uses the Colibri library
for operations related to formal concept analysis3 and a custom implementation for calculating minimal set covers needed for feature group computation.