A major problem associated with mathematical modeling of sepsis is the sheer number of potential biomarkers and the myriad of interactions between them [39]. Compounded with the sparsity of human sepsis data, it is often difficult to pinpoint why few mathematical models of sepsis have been validated against literature: model structure problems, failure of the inverse-problem algorithms, or a combination of both.
Boolean Linear-Programming (Boolean-LP) is introduced as a tool to assist with the initial steps of model creation. Boolean-LP was created in an effort to identify the underlying inflammatory pathways between elderly and young mice (Mochan, Zhang, et al, Discrete
dynamical modeling of influenza infection suggests age-dependent differences in immunity; submitted to the Journal of Virology).
Model structure development is one of the initial phases of mathematical modeling, where the modeler combines literature knowledge and intuition to propose a mathematical characterization of certain dynamical behaviors. This begins an iterative process of fitting data to this proposed model and then making structural model changes to accommodate missing behaviors or other issues. Boolean-LP speeds up this approach by taking a set of proposed network pathways between model states and identifying the network pathway that best describes the given data.
Boolean-LP first requires the user to first discretize the data into Boolean variables. Second, the user needs to propose a set of model pathways via Boolean rules. For example, the rule C(t + 1) = A(t)and not B(t) may be proposed, describing the production of state C by state A but this process is inhibited by state B. A set of rules describing the relationships between states A, B, and C and data are provided to Boolean-LP and it returns the rule that best describes the data. In the case of a solution pool (multiple rules yield the same objective function), Boolean-LP will return all of the possibilities.
To find the optimal set of Boolean rules that best described the data, the rule discovery problem may be formulated as a mixed integer linear programming (MILP) problem. A similar formulation has been previously reported [111]. We define our objective function as the minimum difference between the model and the measured data, given by equation (6.1).
minX
s∈S
X
t∈Ts
|Dt,s− Mt,s| (6.1)
This was later reformulated as a linear objective function using dummy variables (see Section 6.1.2, specifically equation 6.17) in order to conver the problem into a mixed-integer linear programming (MILP). Dt,s and Mt,s represents the measured data and model, respectively,
for state s at time t.
All potential Boolean rules were expressed as a series of logical equivalences (if and only if statements: clause1 ↔ clause2). These rules were expanded into their equivalent
This form allows us to represent each of the OR clauses with a single linear inequality [112]. As long as each of these linear inequalities are satisfied, the overall conjunction expression is satisfied. The result is:
Q1∧ Q2∧ · · · ∧ Qn
Q1 = P11 ∨ P12 ∨ · · · =⇒ y11 + y12+ · · · ≥ DV
Q2 = P21 ∨ P22 ∨ · · · =⇒ y21 + y22+ · · · ≥ DV
(6.2)
where yij represents the Boolean value of expression Pij and DV is a Boolean decision
variable. The decision variable allows the optimizer to apply this constraint (DV = 1) or turn it off (DV = 0).
6.1.1 MILP Formulation of Network Optimizer Problem
Specifically, all Boolean Rules can be expressed in the conjunctive normal form, which com- prises of a series of overarching AND clauses consisting of OR operators:
RuleR = Q1∧ Q2∧ · · · ∧ Qn (6.3)
Qi = Pi1∨ Pi2· · · ∨ Pir (6.4)
Pi1∈ 0, 1 (6.5)
where Qi is a series of inclusive OR operators. Let yi represent the Boolean value of clause
Pi. Each of the Qi logical OR constraints can be expressed as
Pi1∨ Pi2· · · ∨ Pir ⇒ y1+ y2+ · · · + yr ≥ 1 (6.6)
The AND constraint, R, does not need to be explicitly constrained because (6.6) ensures that each of its sub-clauses, Qi, are true. NOT clauses, ¬P1, can be expressed as:
1 − y1 (6.7)
Implications, e.g. P1 ⇒ P2, can be expressed as ¬P1∨ P2, which is an OR constraint:
Using this framework, we can formulate this as a MILP. Potential rules are always of the form: Si,t+1 ← St, where St represents a series of logical operations acting upon the states
at the current time t. This logical clause will generate an update to the ith state, Si, at time
t + 1. If this rule were true, then Si,t+1⇔ Stfor all time, t. Applying the above equivalences,
we obtain:
¬Si,t+1∨ St
¬St∨ Si,t+1
(6.9)
which we expand into the conjunctive normal form and apply the appropriate linear con- straints.
Finally, in order to perform rule optimization, Boolean decision variables, DVij, are
initialized for every proposed rule j in each state i. The k OR constraints generated from the conjunctive normal form of Rule ij is now represented as:
Constraint 1: y1+ y2+ · · · + yr ≥ DVij
Constraint 2: y1+ y2+ · · · + yr ≥ DVij
· · ·
Constraint k: y1+ y2+ · · · + yr ≥ DVij
(6.10)
which represents a slight modification from equation (6.6) in order to allow the optimizer to turn a constraint on or off. If DVij is 0, the values of yi are unconstrained and potential rule
ij does not apply. If DVij is 1, the rule applies. A final constraint is set:
J
X
j=1
DVij == 1 (6.11)
6.1.2 Detailed Formulation Example
As a detailed example, take a rule governing Interleukin-6, activated macrophages (ActiveM), and Interleukin-8 (IL-8, a neutrophil attractant): ActiveM (t+1) ← IL8(t)∧IL6(t) Working out one implication at a time, this is equivalent to constraints (6.13),(6.14), (6.16).
ActiveM(t + 1) ⇒IL8(t) ∧ IL6(t) = ¬ActiveM(t + 1) ∨ IL8(t) ∧ IL6(t) =¬ActiveM(t + 1) ∨ IL8(t) ∧ ¬ActiveM(t + 1) ∨ IL6(t)
(6.12)
1 − ActiveM(t + 1) + IL8(t) ≥ DV (6.13)
1 − ActiveM(t + 1) + IL6(t) ≥ DV (6.14)
IL8(t) ∧ IL6(t) ⇒ ActiveM(t + 1) = ¬IL8(t) ∧ IL6(t) ∨ ActiveM(t + 1) = ActiveM(t + 1) ∨ ¬IL8(t) ∨ ¬IL6(t)
(6.15)
ActiveM (t + 1) + 1 − IL8(t) + 1 − IL6(t) ≥ DV (6.16)
To further simplify the problem, the nonlinear objective function from equation (6.1) may be linearized via the introduction of dummy variables At,s:
minimize X s∈S X t∈Ts At,s subject to Dt,s− Mt,s≤ At,s Dt,s− Mt,s≥ −At,s (6.17)
where Dt,s and Mt,s represents the measured data and model, respectively, for state s at
time t.
A Python (version 3.5) package was written to accept Boolean data and a list of potential rules for each state. This package reformulates the inputs into a MILP problem for use with the Python Optimization Modeling Objects package (Pyomo) [85,86]. Pyomo then converts this script into a solver-friendly file, which was then solved by the IBM ILOG CPLEX optimization studio. CPLEX was set to populate all optimum solutions via its solution pool
feature. Finally, our Python package parses through this solution pool and returns all valid rules that yield the minimum objective function value. The GLPK solver was successfully tested, but it was significantly slower then CPLEX and lacked the ability to populate a solution pool in the case of multiple solutions. Boolean LP is available as an open source project on GitLab.
6.1.3 Applications in Age-related Immune Pathways
The BooleanLP algorithm was created to address optimization problems in a project quan- tifying the immunosenescence (age-related) differences in immune pathways between adult and elderly mice. Adult (12-16 weeks) and elderly (72-76 weeks) BALB/c mice were subject to an innocula of the Influenza virus [113]. Near daily sacrifices (in triplicate) were taken from each group for measurements of inflammatory markers (chemokines and cytokines) and white blood cells, including day zero baseline measurements. An ANOVA analysis was conducted (p ≤ 0.05) to convert the entire dataset into zero (off) and one (on). A library of possible immune pathway rules were generated and BooleanLP was applied to identify the optimum inflammatory network that best described the data. BooleanLP indicated no age-dependent changes in macrophage recruitment between the elderly and adult mice, but macrophage cytokine expressions were different. Cytokine and chemokine pathways differed vastly between the age groups and was responsible for the two day delay in the immune response within elderly mice. The manuscript of this work is currently being updated after the first round of reviewers.