The Pattern-HCG Algorithm

Pattern-HCG combines the three components of the preceding subsections (reach allocation, pattern assignment, and pattern generation) in an integrated, iterative fashion. At a high level, the idea is to rst solve (RA-δ) to produce an aggregate reach allocation with maximum aggregate quality, and then use column generation to generate and assign patterns to maximize disaggregate quality while maintaining the aggregate quality attained by (RA-δ). In the process, there are two substantial challenges that must be overcome. First, we need a way to construct an initial set of patterns so we can start with a feasible solution to (PA).

Second, while searching for feasible patterns we may learn that (PA) is infeasible for some user types (v, i). When that happens, we re-solve (RA-δ) with a lower δvi, and iterate.

Consequently, the full Pattern-HCG algorithm has two phases: (1) a feasibility phase in which the focus is on aggregate quality and δvi values are iteratively tuned to ensure that

the solution to (RA-δ) can be translated into a pattern assignment by (PA) for every user type, and (2) a pattern improvement phase which focuses exclusively on optimizing the secondary, disaggregate quality objective without sacricing the value we obtained for the primary, aggregate quality objective at the end of the feasibility phase.

The feasibility phase begins by initializing the impression utilization factors δvi to 1 for all user types (v, i). We construct a reach allocation by solving (RA-δ), and then we solve a modied version of the pattern assignment problem (PA) for each user type (v, i):

(PA-F): Ψ^{(F )}_vi := Minimize X

Since we ignore disaggregate pattern quality in the feasibility phase, the pattern costs πvip

of (PA) do not factor into the objective. Instead, we relax the supply constraint (1.4c) and minimize its left-hand side, i.e., the number of users allocated by this pattern assignment, P

p∈Pviy_vip. Unlike (PA) which has a supply constraint, (PA-F) is always feasible, as we now show. For each campaign k ∈ Γ(v, i) we can create a pattern p(k) containing exactly f_k impressions of campaign k and no other campaigns; that is, bk,p(k) = 1 and bk⁰,p(k) = 0 for all k⁰ 6= k. Using only such single-campaign patterns, (PA-F) has a trivial solution y^∗_v,i,p(k)= svix^∗_vik with dual values ¯α^{∗(F )}_vik = 1, ∀k ∈ Γ(v, i). We initialize the pattern pool Pvi

with only these single-campaign patterns, and from this initial solution, continue to solve (PA-F) using column generation. The corresponding pattern generating problem is the following binary knapsack problem with |Γ(v, i)| items, which can be solved very quickly and eciently via dynamic programming:

If (PG-F) concludes with ψ^{∗(F )}_vi < 0, the resulting pattern improves (PA-F); we add it to Pvi

and re-solve (PA-F). Otherwise, we found the optimal solution to (PA-F), and have two cases to consider.

If (PA-F) converges to optimality with Ψ^{∗(F )}_vi > svi, we know the corresponding pattern assignment problem (PA) is infeasible; i.e., it is impossible to implement the solution x^∗_vikfrom (RA-δ) using svi users. In this case, δvi over-estimates the attainable impression utilization, i.e., 1−δviunder-estimates the fraction of impressions that must remain as excess. In this case, we decrease δvi, re-solve (RA-δ) to produce a new reach allocation x^∗_vik, and resume solving (PA-F) and (PG-F). To derive a good updating rule for δvi, note that the total number of impressions used (i.e., assigned to R&F ads) in pattern p is given by P_kf_kb_kp. Therefore, the total number of impressions used in (PA-F) at optimality is given by:

X

Not surprisingly, this impression count is closely tied to the solution from (RA-δ) and is known before solving (PA-F). Given that each of the Ψ^{∗(F )}_vi users provides Lvimpressions, the eective impression utilization rate at the optimal solution to (PA-F) is given by P_k∈Γ(v,i)^fksvix∗vik

LvΨ^{∗(F )}_vi . Based on this analysis, we suggest the following update rule:

δ_vi ← sviX_vi^∗/Ψ^{∗(F )}_vi − , (1.7)

where X_vi^∗ =P

k∈Γ(v,i) fk

Lvx^∗_vikis the left-hand side of constraint (1.3c) at optimality, and  > 0 is used to accelerate convergence.

On the other hand, if for all user types (v, i), (PA-F) converges to optimality with Ψ^{∗(F )}_vi ≤ svi, we have a feasible solution to all corresponding (PA) problems, and we switch to the pattern improvement phase. In this phase, for each user type (v, i), we solve (PA) and collect the optimal dual values ¯α^∗_vikand ¯β_vi^∗. Then we solve (PG) to construct a pattern with minimal reduced cost. If ψ_vi^∗ + ¯β_vi^∗ < 0, the resulting pattern is benecial; we add it to Pvi (with parameters bkp = b^∗_kand πvip= π(b^∗)) and re-solve (PA). On the other hand, if ψ_vi^∗ + ¯β_vi^∗ ≥ 0, the current solution to (PA) is optimal and we stop. Note that solving (PG) is harder than

(PG-F) if π(·) is not linear, however, in the pattern improvement phase we no longer solve the large-scale math program (RA-δ). Again, we remind the reader that iterations between (PA-F) and (PG-F), or (PA) and (PG), can be conducted in parallel across user types (v, i).

Finally, at ad serving time, when a type-(v, i) user arrives for the rst time, s/he is assigned pattern p ∈ Pvi with probability y_vip^∗ /svi. Subsequent visits of the same user are served the sequence of ads in his/her assigned pattern. If an unexpected user type (v⁰, i⁰) arrives, a near-optimal reach allocation xv⁰i⁰k is computed using Corollary 1, and a pattern is generated using the online part of Pattern-G algorithm. The full Pattern-HCG method is presented in Algorithm 1.4.

Remark 1: The value of δvi always decreases following update rule (1.7). This follows since X_vi^∗ ≤ δ_vi due to constraint (1.3c), and Ψ^{∗(F )}_vi > svi whenever δvi is updated. Further, note that a decrease in impression supply at some supply node can only increase the demand burden of other supply nodes. As a result, we may need to solve (RA-δ) and update the δvi

values several times before we converge.

Remark 2: A decrease in δvi implies forcing additional excess in supply node (v, i). If additional supply is not available in other supply nodes or using supply from other nodes would have a signicant impact on representativeness, a δ update may cause under-delivery to increase for some campaigns. In this case, the total volume of the publisher's trac left as excess (i.e., left for non-R&F ads) increases. However, it is also possible that after re-solving (RA-δ) with a lower δ, total under-delivery is maintained by shifting excess supply from one node to another.

Remark 3: (PA-F), which minimizes the number of users, also minimizes total excess, and thus attains the maximum impression utilization rate possible. Therefore, our update rule is conservative. See Appendix 1.K for a proof of this behavior in the more general case of the cutting stock problem.

Algorithm 1.4 Hierarchical Column Generation (Pattern-HCG)

• OFFLINE:

FEASIBILITY PHASE:

 Initialize: δvi← 1for all user types (v, i).

 [1]: Solve the Reach Allocation problem (RA-δ) using Modied SHALE (Algorithm 1.3)

 Parallelize: For each user type (v, i):

∗ [2F]: Solve the Pattern Assignment problem (PA-F) and obtain the optimal dual values

¯ α^{∗(F )}_vik .

∗ [3F]: Solve the Pattern Generation problem (PG-F). If ψvi^{∗(F )}< 0, add the generated pattern to Pviand go to [2F]. Otherwise, continue.

∗ If Ψ^{∗(F )}vi > svi, decrease δviaccording to update rule (1.7).

 If δviwas decreased for any user type (v, i), go to [1]. Otherwise, continue.

PATTERN IMPROVEMENT PHASE:

 Parallelize: For each user type (v, i):

∗ [2]: Solve the Pattern Assignment problem (PA) and obtain the optimal dual values ¯α^∗_vik, β¯^∗_vi.

∗ [3]: Solve the Pattern Generation problem (PG). If ψvi^∗ + ¯β_vi^∗ < 0, add the generated pattern to Pviand go to [2]. Otherwise, stop.

• ONLINE: Upon a visit from user j of type (v, i):

 If it is the rst visit from user j in the planning period:

∗ Set the number of arrivals qj ← 1.

∗ If user type (v, i) was explicitly considered as a supply node in the oine phase: Randomly draw a pattern p from the pattern pool Pviwith probability y^∗vip/svi, and denote the chosen pattern as pj. Otherwise, construct a generalized solution xvik using Corollary 1, and use the online portion of Pattern-G (Algorithm 1.2) to generate a corresponding pattern pj.

 Display the qj'th ad in pattern pj to user j. Set qj← q_j+ 1.

Remark 4: Re-solving (RA-δ) after a δ-update is quite fast, since we can warm-start SHALE using the solution from the last time we solved (RA-δ). See Theorem 2 for details.

Remark 5: We can construct bounds for the impression utilization factors δvi. Let δ^minvi = min_k∈Γ(v,i){f_k}/L_v, which is derived from the pattern consisting of only the campaign with the smallest fk, and let δ^maxvi = max_bk∈{0,1}{P

k∈Γ(v,i) f_k

Lvb_k : P

k∈Γ(v,i)f_kb_k ≤ L_v}, which can be computed by solving a binary knapsack problem with |Γ(v, i)| variables. A geometric illustration of the range [δ^min_vi , δ^max_vi ] and how the δvi values aect the feasibility of (PA) is

provided in Appendix 1.J.

Remark 6: We expect over time, a publisher may learn appropriate δvivalues, and initialize with δvi < 1to speed up convergence. Nevertheless, among our numerous synthetic test cases and real industry data, we never encountered a case where it takes beyond 10 (mostly 4-6) rounds of adjustments before the reach allocation from (RA-δ) is attainable at 95% of the supply nodes.