• No results found

The Project Portfolio Management Problem

N/A
N/A
Protected

Academic year: 2021

Share "The Project Portfolio Management Problem"

Copied!
50
0
0

Loading.... (view fulltext now)

Full text

(1)

The Project Portfolio Management Problem

Souvik Banerjee

Wallace J. Hopp

June 21, 2001

Abstract

We consider the Project Portfolio Management Problem (PPMP) in which a limited resource must be allocated among a set of candidate projects over time so as to maximize expected net present value. We formulate this problem as a dynamic program but conclude that this approach is too computationally complex to be of value in supporting real-world project management. So, we investigate the structural properties of the optimal solution to the PPMP and demonstrate that the solution reduces to a simple form under certain environmental conditions. This simplified policy, which we term the index policy, sequences projects according to a simple ratio and then allocates resource up to each project’s practical limit in the order given by this sequence. Through numerical tests we demonstrate that this policy performs robustly well on the general PPMP. Hence, we conclude that the index policy is a practical way to incorporate economic and timing issues into a multi-dimensional scoring model for addressing real-world project portfolio management situations.

1

Introduction

One of the most critical problems facing most product-oriented firms is management of the research and new product development processes. A steady stream of new products is essential to the long term health of a company. Hence, the decision of how to allocate resources (money, people, space) to research, development and commercialization projects is of enormous strategic importance.

The overall problem of investing in new product innovation is far too complex and multi-dimensional to be reduced to a single model. In real-world situations, managers must consider

(2)

economic, technical feasibility, marketing, competitive positioning, regulatory and many other issues. Because it is not possible to incorporate all of these issues in a detailed op-timization model, such models are rarely used in practice. Instead, simple scoring models that rate projects according to many criteria are often used.

In this paper we examine the Project Portfolio Management Problem (PPMP), which considers how to allocate a limited budget to a set of candidate projects over time with the objective of maximizing expected net present value. However, because we recognize that the many “intangibles” associated with product innovation are paramount to decisions in practice, we do not feel that a complex algorithmic solution to this piece of the problem would provide practical value. So instead we develop a simple procedure for ranking projects according to their economic attractiveness, which considers timing and resource interactions, but is transparent enough to incorporate into the scoring methods used by practitioners. By doing this, our work partially spans the gap between research and practice in the project management field.

The remainder of the paper is organized as follows: In Section 2 we review the literature on the PPMP. In Section 3 we formulate a DP to support the resource allocation decision, but note that solution of this DP is too cumbersome to be of much use in practice. In Section 4 we characterize the structure of the optimal policy and show that it is still complex in general. In Section 5 we show that the optimal policy is much simpler for some special cases of the PPMP. In Section 6 we consider one of these simplified policies, the index policy, as a heuristic for the general PPMP and show that it performs robustly well. We conclude the paper in Section 7.

(3)

2

Literature Survey

A vast amount of literature exists on the PPMP, most of which can be classified into: (a) scoring models, where each project is assigned an index based on various criteria (e.g., by a set of experts); (b) mathematical programming models, where an objective is maximized over a set of constraints using linear programming, integer programming, dynamic program-ming, goal programming or other optimization techniques; (c) economics or financial models, relying on cost/benefit analysis, payback period, NPV/IRR or other portfolio methods; d) decision analysis techniques such as decision trees, PERT/CPM and Monte Carlo simulation. Most early literature concentrated on analytical techniques for selecting and scheduling projects. However, it was soon pointed out, (for example in [1], [2], [3]and [23]), that these methods were virtually ignored by industry. Baker [2] reported that decision theory models were seldom used, scoring methods were somewhat more popular, but the most prevalent method actually used by managers was traditional capital budgeting. He conjectured that “the trend in application appears to be away from decision models and toward ‘decision information systems’ ”. A decade later, Liberatore [16] reported that even though managers were familiar with mathematical programming models they almost completely avoided them. Financial techniques and decision analysis techniques were more prevalent. Another decade later, Schmidt et al. [22], pointed out again that “there has been a overall mismatch between modeling efforts and modeling needs”. They argued that too much attention has been devoted to modeling the problem focusing on outcomes as opposed to developing schemes that help the decision makers gain insight into the decision process. The authors claimed that the failure to use mathematical models is not merely the lack of training or resistance to change on behalf of the managers, but rather the lack demonstrable proof that these models are indeed more effective in solving the problem. Gupta et al. [8] echoed most of the above conclusions.

(4)

A key shortcoming in early PPMP models which may have affected their usefulness, was that they failed to account for uncertainty. Many recent models have tried to correct this by using some sort of stochastic analysis. For instance, Heidenberger [9] used a network model to represent a project and considered different types of probabilistic nodes controlling the progress of the project. Some nodes were critical, ‘go-no-go’ type, (that is, a failure at these nodes terminates the project), while other nodes directed the outcome of the project based on a probability governed by the amount of resource allocated to the activity. Others, for example Tavares [24], considered uncertainty in the task duration, where the distribution is based on the level of resource allocation.

A second challenge in developing models that approximate reality is representing the dependencies of tasks on resource allocation. Most literature uses a static approach. That is, it is assumed that each task has a constant requirement for each resource, which has to be satisfied fully in order for the task to proceed. This assumption frequently leads to 0-1 integer programs. While it may be valid with respect to some discrete resources (e.g., allocation of a machine to a job), there are many practical resources (e.g., capital or manpower) where partial allocation is possible. Gerchak [6] considered the effect of partial funding where the level of funding affects the ‘achievement level’ of the task and developed a mathematical model which required numerical solution to find the optimal allocation of resources. Ulvila et al [25] also mentioned the effect of partial funding on the level of benefit obtained from a project and postulated that the benefit increase rapidly and monotonically with funding level in the initial stages, but above a certain efficient level of funding it displays diminishing marginal returns and eventually shows no further increase in benefit at all. Madey [19] similarly considered the success probability of a task being a monotonic nondecreasing function of the funding level. More recent approaches, e.g. [4], have modeled resource allocation dynamically, where at each decision current resources can be shifted among the ongoing projects.

(5)

A third area where PPMP models often fall short of reality is with respect to the fact that most project portfolio decisions involve more than a single criterion. Frequently, these criteria are non-quantifiable and highly subjective, for example long term strategic fit and company image. Notable work in this area includes [7], [11], [13], [17] and [19]. Related research has addressed the organizational structure and how to integrate multiple criteria via a Decision Support System (for example, [12], [20] and [26]).

In this paper, we address the PPMP and explicitly model task uncertainty and dynamic resource allocation. To enable our model to be used in multi-criteria contexts (e.g., as part of a larger scoring model), we seek a simple, intuitive solution, rather than a detailed algorithmic approach. Our work is most closely related to that of Kavadias et. al [10]. However, while they consider a similar setting and provide related structural insights, they do not provide a way of finding an optimal policy or a method for adapting their results to a decision making framework.

3

Problem Formulation

The PPMP involves allocating a budget over time to a portfolio ofM ={1, . . . , m}candidate projects. (We consider the case where all candidate projects are available at the start of the problem, but discuss how our results can be adapted to situation with new projects arrival later on.) Each project i consists of a sequence of n tasks. Task j of project i has a work requirement wi,j expressed in terms of total resource-hours required to complete the task. Hence, the total work requirement of projectiis given by wi =nj=1wi,j. The time required to complete a task is a function of the resource rate allocated to it. The total resource rate available at any point of time, or the budget is given by some constantB. Taskj of project

i is assumed to succeed with a predetermined probability pi,j, which is independent of all other problem parameters including the resource rate, and success or failure becomes known

(6)

only when the task is completed. If a task of a project fails then the entire project fails. Hence, in order for projects to be successful all tasks must be completed successfully, which implies that the probability of project success is given by pi =nj=1pi,j. Upon completion, each project iyields an expected profit of αi.

The objective is to find a policy that allocates the available budget so as to maximize the expected net present value. Such a policy must specify an allocation for any reachable state, where the state is defined by the work requirements and probabilities for all tasks (and partial tasks) of unfailed projects. Let the random variableti(γ) be the time when projecti completes all its tasks successfully under policy γ. Then the net present value for the policy can be expressed as Π = m i=1 piαiE[e−βti(γ)] (1)

where β is the continuous discount rate and the expectation is taken over all possible com-binations of task completion events. Because the objective function is monotonically non-increasing in the completion times it is easy to verify that without loss of optimality we can restrict attention to efficient policies, which are policies that always utilize as much of the available budget as possible while there are unfinished tasks. Note, that the implicit assumption is that all projects will be eventually undertaken, even though at any point in time the available budget might restrict work to only a few of them.

3.1

Modeling Resource Allocation

In general, we expect the completion time of a task to be inversely proportional to the resource rate allocated to it. The simplest possibility is a linear relationship. That is, if we allocate resource at ratex to a task that has a work content ofw, the time to complete the task will be w/x. However, a linear decrease in completion time is not sustainable forever since eventually there will be diminishing returns to additional resource allocation.

(7)

limit

Resource

Linear

Compl

eti

on rat

e

Approximation

Expected

Figure 1: Variation of Task Completion Time with Funding Level

Hence, we expect the behavior to be as shown by the dashed line in Figure 1. To approximate such behavior we model the resource-rate profile as shown by the solid line in Figure 1. We assume a linear relationship up to some efficient resource allocation limit li,j

for task j of projecti. Beyond this efficient limit, no further increase in rate is observed. In general, we expect the efficient limit of a project will be less than the budget available at all times, although this is not required. Also, we expect the sum of the efficient limits for the projects at any point to be greater than the budget. That is,

m

i=1

li,j > B, (2)

If this is not true, we can simply allocate each project resource up to its efficient level and no further optimization is necessary.

(8)

3.2

Constant Rate Equivalence

Our problem is to find a policy that allocates resource rate to each project in the portfolio in a manner that maximizes the expected NPV. The resource allocated to projectiat timet is denoted by xi(t) and represents a valid resource allocation as long as ni=1xi(t)B and 0xi(t)li(t), for alli, whereli(t) is the efficient limit of the task of projectiundertaken at timet. Any such resource allocation policy induces a sequence of task completion times. By computing the probabilities associated with the task completion times, we can use Equation 1 to compute the expected profit for a given policy.

We can simplify the required calculations by using an equivalent constant resource ratexi (constant w.r.t time) for each of the projects between any two task completion times. To see that this can be done without loss of optimality, let t1 and t2 be two task completion points that result from a given policy along a given sample path. Suppose during the interval [t1, t2] work wi is done on project i with a possibly variable rate, so that tt2

1 xi(t)dt = w1. Using a constant resource allocation xi such that xi(t2 t1) = wi will result in exactly the same completion times and therefore the same expected profit. Hence, we can restrict attention to policies that allocate constant resource amounts to tasks in between task completion times. The resulting problem will be equivalent in terms of both completion times and expected profit. Our use of constant rates is for algebraic convenience only; it does not imply that one is restricted to use them in real life. Note, however, that this restriction is possible only because the budget is constant.

3.3

Dynamic Programming Formulation

We now develop a DP for the general PPMP. The state space for the DP is given by the work content of the portfolio, s={n,w}, where nNm is the number of tasks left for each

(9)

The control variable is the vector x Rm, where xi denotes the allocation to project i, at the first decision epoch. The allocation vector x must satisfy the following constraints:

xi ˆli, i∈M (3)

where ˆli is the resource limit for the current task of project i,

m

i=1

xi ≤B (4)

where B is the total budget, and

xi 0, i∈M (5)

Constraints 3, 4 and 5 define the action space A for the DP.

Any feasible allocation causes the next completion event to consist of completion by one or more project tasks. Let C[s] denote the set of all possible combinations of project tasks that can complete (possibly simultaneously) due to a feasible allocation starting at state s. Let χc[x] be the indicator that completion c C[s] occurs due to allocation x. For any

c C[s], let τc(x) denote the completion time of a task (or tasks) in c under allocation

x. Let Y[s,c] denote the set of states reachable from state s under task completion c

and let ζ(s,y), y Y[s,c] be the corresponding probability. Finally, let r[s,y] denote the expected revenue from reaching state yin Y[s,c] from state safter task completion c. This expectation is positive only if the completion c corresponds to a successful completion of a project or projects. The value function V[s] for the PPMP is obtained by summing over all possible completion eventsc inC[s] and computing the expected value function from states

yY[s,c] reachable from states under completion event c. This is written as:

V[s] = max x∈A c∈C[s] χc[x]e−βτc(x) y∈Y[s,c] (ζ(y)V[y] +r[s,y]) (6)

(10)

The DP recursion (6) is terminated when we reach a state with only one task left. For such a state, the expected NPV is computed by allocating to the remaining task the minimum of the budget and its efficient limit.

3.4

Dynamic Programming Solution

The state-space of the DP given by Equation (6) is continuous. In general, such problems are difficult to solve. However, since the number of possible sample paths for the DP is finite, we can devise an algorithm to solve the DP in finite time. This is done as follows.

Any policy for the PPMP induces a task completion sequence for every possible sample path. Let Σ denote the set of all possible task completion sequences for a PPMP that occurs under the condition that all projects are successful. Let R denote the set of sample paths possible for the PPMP. Note that both |Σ| and |R| are finite and known in advance. For every sequence σ in Σ, each sample path r in R generates a subsequence. For each such subsequence (σ, r), let Kr(σ) be the number of task completion epochs that are realized.

Observe that these subsequences are mutually exclusive and collectively exhaustive. A policy for a PPMP should specify the allocation for each reachable subsequence of it (by ruling out unreachable subsequences, we can improve computational efficiency). Let xr

i,k(σ) be the

allocation to project i at the kth epoch under sample path r for sequence σ. Let τr k(σ)

denote the corresponding length of the task completion interval. Notice, that τr

k(σ) can be

computed as a function of the decision variablexat time 0. The completion time of projecti under (σ, r) can therefore be computed at time 0 as a function ofτkr(σ). We definetri(σ) = if project i fails under subsequence (σ, r). Let lr

i,k(σ) denote the resource limit for the task

of project ithat requires funding at that epoch. Further, if project i has already completed by this epoch under (σ, r), then we define lri,k(σ) = 0. Observe that at any epochk, the full task completion sequence and the sample path is not necessarily revealed. However, we are specifying a decision variable for all sequences and sample paths that are possible at epoch

(11)

k. Therefore, we need not know either the full sequence or the sample path in advance. Using the above notation we write the following optimization problem which is equivalent to DP (6): Z = max∆ σ∈Σ{Zσ} (7) where = max x∈C[σ] r∈R Pr[r] m i=1 αie−t r i(σ), σ Σ (8)

and C[σ] consists of the following set of constraints defined for each σΣ:

(i) the budget constraints,

m

i=1

xri,k(σ)B, k = 1, . . . , Kr(σ), rR (9)

(ii) the resource limit constraints,

0xri,k(σ)lri,k(σ), i= 1, . . . , m, k= 1, . . . , Kr(σ), rR (10)

(iii) the sequence determining constraints,

τkr(σ)>0, k= 1, . . . , Kr(σ), r R (11)

Each problem Zσ can be solved by standard NLP solution techniques. For example, since the objective and the constraints are differentiable, we can construct the Lagrangian and solve for the corresponding KKT conditions.

The above algorithm is useful, for example, to verify our numerical tests that follows. However, such an approach is not practical for large problems and is not consistent with our goal of a simple method that can be combined with other factors in a decision support

(12)

system.

4

Concurrent-Priority Policy

To develop a simple but effective heuristic for the PPMP, we now examine the optimal policy more clearly. Since we can restrict attention to policies with constant resource allocations between task completions, a policy need only specify the allocation at the beginning of each decision epoch. This allocation, in turn, determines the time of the next task completion (decision epoch). This process is repeated as long as there are projects to complete. Hence, policyγ induces a task completion sequence for each possible sample path. Therefore, in the same manner as Section 3.4, let tr

i[γ] denote the completion time of project iunder policy γ

and sample path r. Then, the expected NPV for policy γ can be computed as

E[Π(γ)] = r∈R Pr{r} m i=1 αie−βt r i[γ] (12)

We begin by showing that we can restrict attention to a specific class of policies, which we call concurrent-priority policies. Under a concurrent-priority policy we either follow a

priority policy or a concurrent policy, defined as follows:

Priority Policy: Under a priority policy, projects are ordered in a list. At each decision epoch, the project at the top of the list is allocated its efficient limit or the available budget, whichever is less. Next, the remaining budget, if any, is allocated in a similar fashion to the second project on the priority list. This process is continued until the entire budget has been allocated. Note that, under a priority policy, at each decision epoch, we fully fund a set of projects and possibly fund one additional project at a partial level. Projects funded in this manner are said to be prioritized. However, the priority order of the projects need not be the same at all decision epochs. As each task

(13)

is completed, the information on its success or failure can be used to recompute project priorities.

Concurrent Policy: Under a concurrent policy, two or more tasks finish simultaneously at some point in time for at least one sample path. Whether or not two tasks actually complete simultaneously depends on the task outcomes (success of failure). But in any concurrent policy there is a nonzero probability of simultaneous task completions. We denote the class of concurrent policies as ΩC and the class of priority policies as ΩP. The class of concurrent-priority policies is the union of these two sets and we denote it by Ω = ΩC P. To illustrate what is ruled out, observe that for a policy γ not in Ω, the following holds true. No two tasks complete simultaneously under any sample path with nonzero probability. Further, at some decision epoch there exist two or more projects that receive non-zero funding below their efficient limits.

We now prove that for any policy γ /Ω there exists a policy in Ω whose expected profit is at least as large. That is, we prove

Theorem 4.1 For the PPMP there exists an optimal policy that belongs to class.

Proof outline: (Full proofs appear in the Appendix.) To prove the theorem, we start with any policy not in Ω. For this policy it must be true that there exists an epoch where at least two projects, project i and j are funded at an intermediate level. For this epoch it is feasible to reallocate some resource from project i toj (or vice versa) without changing the task completion sequence if the change is sufficiently small. We show that such a perturbed policy does not decrease the expected NPV, which proves the result. Further, for a policy in Ω such a perturbation is not possible since any such change in the allocation will alter the task completion sequence.

The PPMP would be much simpler to solve if we could restrict attention to priority policies in ΩP. Unfortunately, this is not optimal in general. For example, consider the

(14)

Project α wi,1 li,1 pi,1 wi,2 li,2 pi,2 wi,3 li,3 pi,3

Project 1 20 0.1 0.4 1.0 0.3 0.4 1.0 2.0 0.7 1.0

Project 2 1 0.675 0.75 1.0 – – – – – –

Table 1: Example Portfolio

following two project portfolio whose optimal solution is a concurrent policy in ΩC. Project 1 has three tasks and project 2 has one task with parameters given in Table 1. The budget is normalized to 1 unit and the discount factor is β = 1.0. The optimal policy for this PPMP is obtained by solving DP (6) which yields the following nonunique solution: In the first decision epoch we fund project 1 at any resource level in the range [0.37,0.4]. This ensures that task 1 of project 1 finishes first. In the second epoch, we fund such that the second task of project 1 and the only task of project 2 finish together. At the third epoch, only the third task of project 1 is left and we fund it at its efficient limit until completion. For any policy in this set the optimal expected NPV is $0.73. Notice that, in the first two epochs, both projects may be funded in an intermediate level, so the policy is not in ΩP. Figure 2 (not to scale on the time axis.) illustrates the allocations for two cases: (i) allocate 0.37 at the first epoch and (ii) allocate 0.4 at the first epoch. Note that both projects complete at the same time under (i) and (ii) and therefore these policies yield the same NPV.

The restriction to concurrent-priority policies reduces the solution space for the PPMP. For example it reduces the number of (σ, r) pairs to search over in the solution procedure described in Section 3.4. However, the problem of finding an optimal policy is still complex.

5

Priority and Index Policies

While it is possible to find an optimal concurrent-priroity policy via numerical search, the solution from such a procedure may not be intuitive and therefore difficult to combine with

(15)

other criteria. A priority policy is fairly intuitive and easy to implement; a concurrent policy is not. We now show that for two simpler versions of the PPMP, we can restrict ourselves to priority policies.

5.1

Single Task PPMP

First, we consider a PPMP where each project has a single task, which we call the Single

Task PPMP, or STPPMP. We first show

Lemma 5.1 For the STPPMP, we can restrict attention to policies under which the alloca-tions to any project are non-decreasing in time until the completion of the project.

Proof Outline: Whenever the condition of the result is violated there exists a project i whose funding decreases across epochs. For the policy to be efficient there must also exist a project j whose funding increases across the same epochs. An alternate policy is created by reallocating resource from project i to project j in the first epoch and doing the reverse in the next. The expected NPV for this policy is shown to be at least as high as the original policy, which proves the result.

The intuition is that it is sub-optimal to postpone a project once resource has been invested in it unless additional information is obtained. In the multi-task PPMP, successful completion of a low probability task may warrant additional funding for a project. Such situations do not arise in the STPPMP. Hence, once a project is prioritized, it receives the same (or higher) priority until its completion. We use this result to prove the following property for the STPPMP:

Theorem 5.1 For the STPPMP, there exists an optimal policy inP.

Proof Outline: We show that project completion times are convex in the work content of the projects. This allows construction of an improving sub-gradient for every point corresponding

(16)

to ΩC. Hence, points in ΩC cannot be optimal, which from Theorem 4.1 implies that the optimal policy lies in ΩP.

In the next section we show that for a different simplification of the PPMP we can restrict attention to an even smaller, and more practical, action space.

5.2

Restricted Budget PPMP

The main reason the optimal policy for the PPMP is complicated is that the contribution of a project depends not only on its own parameters but also on those of the other projects. It would be vastly simpler, and hence more feasible to combine economic analysis with other considerations, if we could evaluate each project independently. In this section we show that this is optimal for a certain class of PPMP.

Specifically, we consider PPMPs where the efficient limit of all tasks of all projects is at least equal to the budget, that is,li,j B for alli, j. We call the PPMP with this property,

the Restricted Budget PPMP or RBPPMP. Although there are practical situations where

such conditions exists—for example, in smaller organizations that have only enough resources to fund a single project at a time—the main goal of this simplification is to gain insight and identify factors that will help us develop a useful and practical heuristic for the general PPMP.

First, we show that for the RBPPMP we can restrict attention to priority policies. In what follows, we normalize the constant budget B to 1 for notational convenience.

Theorem 5.2 For the RBPPMP, there exists an optimal policy inP.

Proof Outline: We show that whenever multiple tasks are funded in any decision epoch we can construct an alternate policy where we fund only one of these tasks, say of projecti, by reallocating resource from the other funded projects. As a result of this reallocation project

(17)

i’s task finish earlier, thereby improving the expected NPV for the project. We then show that such reallocation does not delay any of the other projects hence the expected NPV of the other projects are unaltered.

A priority policy is much easier to implement and understand than a concurrent-priority policy. Since a priority policy can be described as a sequence (that is, of projects listed in order of priorities), for a m-project PPMP there are at most m! possible priority policies. We now show that, for the PPMP, we can identify the optimal priority policy; in polynomial time.

For the general PPMP, the optimal allocation must be computed at each task completion interval to incorporate the information obtained from the current task completion events. However, we can show that for the RBPPMP such computation is unnecessary. Observe, that for the RBPPMP, a prioritized project absorbs the entire budget and hence only one project is funded at any given time. Therefore, if a funded project completes successfully, its work content decreases and probability of successful completion is increased, and hence it becomes even more attractive to fund. The projects that are not funded, do not undergo any change of state and hence their relative attractiveness is unchanged. Therefore, if the task of the funded project succeeds, we continue with it. If it fails, then we switch to the project that had the next highest priority at the beginning of the previous epoch. Therefore, the priority policy has to be computed only once, at the beginning of the decision process. We will prove this formally in Theorem 5.3.

This structure of the optimal policy for the RBPPMP allows us to identify the optimal sequence very easily. To do this, we introduce the following notation. Denote the expected NPV of prioritizing project i at time t = 0 as θi =αipie−βwi, for all i, and let Ai

j represent

the event that exactly j tasks of project icomplete. The probability of the eventAi

(18)

by qi,j = Pr[Aij] =        (1pi,j)kj−=11pi,k, j = 1, . . . , ni1 ni1 k=1 pi,k, j =ni (13)

Notice, that under a priority policy, whenever a project icompletes j tasks it introduces an additional delay of jk=1wi,k to all the projects that have a lower priority than project i. Therefore, the extra discount factor introduced by projecti to the expected NPV of all the lower priority projects is

γi,j =e−β j

k=1wi,k, i= 1, . . . , m, j = 1, . . . , n

i (14)

Using this notation we prove

Theorem 5.3 For the RBPPMP, an optimal policy is obtained by computing the optimal priority sequence at the beginning of the decision process, and then fully funding the highest priority project until it completes successfully or fails. Upon completion, we prioritize the next project in the original sequence and continue in this manner until all projects complete. Further, the optimal priority sequence is computed according to the non-increasing order of the quantity

Ii = θi

1 nj=1i qi,jγi,j

, i= 1, . . . , m (15)

Proof Outline: First we show that for an RBPPMP after a successful completion of a project we do not switch. This is done by showing that if for two projects i and j, it is optimal to switch to project j even after a successful task completion of project i, then it must be optimal to fund project j in the previous epoch, hence contradicting the optimality of funding project i in that epoch. The optimality of prioritizing according to Ii is proven as follows. Theorem 5.2 is used to write the expected NPV for a priority policy in closed form. The optimality condition for priority policies can then be written in terms of a set of

(19)

inequalities. These inequalities ultimately reduce to a condition that implies that project i is prioritized over project j iff quantity Ii > Ij.

Observe thatindex Ii given by (15) is completely determined by the parameters of project

i and hence can be computed independently of the other projects in the portfolio. Theo-rem 5.3 states that in order to obtain the optimal policy for the RBPPMP, we can compute and sort the indices for all projects in nonincreasing order and then execute the projects in this order. We term this simplified version of a priority policy an index policy.

We can interpret the index in Equation (15) more intuitively by defining a random variable Γifor the discount factor induced by projection all the projects with lower priorities, as measured from the time project i starts. The term ni

j=1qi,jγi,j

in the denominator of the index can be interpreted as E[Γi]. This allows us to write the index as

Ii = θi

1E[Γi], i= 1, . . . , m (16) Thus, the index for a project is proportional to the best expected NPV that can be obtained from a project and is inversely proportional to the delay it causes to other projects in the portfolio. It indicates, that shorter projects should be prioritized over longer projects if they have comparable expected NPV, since they introduce less delay on the projects subsequently undertaken.

An index policy provides a simple solution to the RBPPMP. The index is computationally inexpensive and offers insight into the relative worth of the projects. Moreover, it provides a cardinal ranking. As such, it is well-suited to a multi-objective framework, in which this economic index is combined with other criteria.

Further, since the index of a project is computed from parameters of that project only, it is easy to incorporate new project arrivals. Since the indices of the existing projects are unaffected by such arrivals, new projects can be inserted into the sequence according to

(20)

their indices. If a new project is attractive enough it may even replace the currently funded project.

Since the RBPPMP is only valid for environments where the entire budget can be devoted to one project at a time, it is not directly applicable to most realistic situations. However, we can use the insights from the index policy to develop a heuristic for the general PPMP.

6

Index Policy for the General PPMP

The solution to the RBPPMP suggests that the primary drivers of profitability of a project are its revenue and the delay it causes to other projects in the portfolio when it is undertaken. Since this is likely the case for the general PPMP as well, we now consider using a modified version of the index in Equation (15) for the general PPMP.

First we modify the index to allow for the delay caused by tasks with arbitrary resource limits. We modify γi,j as derived in Equation (14) as follows:

γi,j =e−β

j

k=1wi,kli,j , i= 1, . . . , m, j = 1, . . . , n

i (17)

Note, that with this modification it can no longer be interpreted as the exact discount factor introduced for the lower priority projects, since other projects can receive funding while project i does. However, it does gives a lower bound on it. Using thisγi,j , the index for the general PPMP becomes

Ii =

Πi 1 nj=1i qi,jγi,j

, i= 1, . . . , m (18)

(21)

6.1

Average Performance of Index Policy

First we show that the worst case error from using Equation (18) can be large.

Theorem 6.1 The worst case percentage error from using Index (18) for the PPMP is 100%.

Proof Outline: We first argue that the maximum percentage error occurs when the indices of the projects are equal. We then maximize the error from a two project portfolio and show that 100% error can be achieved asymptotically. This is obtained when the work content of one project reaches infinity while that of the other reaches zero (while their index remains the same). Since 100% is the maximum percentage error possible and the addition of further projects does not reduce the maximum error the result is proved.

Even though the worst case error for the index policy is not encouraging, the proof of Theorem 6.1 shows that the maximum error occurs when allocating vastly dissimilar projects (with respect to work content). In practice this implies that the project with small work content could be something like a one day improvement effort, while the larger project is a long-term exploratory research project. It is highly unlikely that two such dissimilar projects would be funded from the same resource pool. So, the cases where the index policy performs poorly are unlikely to be of practical interest. Therefore it is worthwhile to explore the average error that results from using an index policy in a general PPMP. We do this numerically below.

6.1.1 Simulation of Index Policy Errors

To evaluate the performance of the index policy for the general PPMP over the range of practical interest we consider the following ranges for the problem parameters. We feel that these reasonably represent most project portfolio management situations. From this range,

(22)

we select points representing individual cases of the PPMP. All points are considered equally likely, so we select the parameters according to the uniform distribution.

αi In a typical portfolio management decision we expect the ratio of the project revenues to

rarely exceed a factor of a thousand. That is, we expect for any two projects i and j,

αi/αj [0.001,1000]. Hence, we let αi ∼U[1,1000] fori= 1, . . . , m.

wi,j With similar reasoning as above, we expect the ratio of the work content of the projects

to rarely exceed a factor of a hundred. Hence we let wi,j U[1,100] for i = 1, . . . , m and j = 1, . . . , ni.

Further we restrict the efficient limits and probabilities as follows:

li,j We allow the efficient limits for the tasks to vary from 5% of the available funding to

100% of the available funding, orli,j U[0.05,1.0], where the budget is fixed at 1.

pi,j We assume that each task has at least a 5% chance of success in order for a project to

be considered for funding. Hence, we letpi,j U[0.05,1] for all i and j.

We fix the discount rate β at 50%. In most practical situations the discount rate is far less than this value. Therefore, we expect to get a conservative estimate of the error from this choice since the errors increases monotonically with the discount rate.

The methodology for analyzing the error is as follows. We sample the project parameters from the respective distribution to generate the project portfolio. We then find the optimal expected NPV for the portfolio. Next, we compute the index policy and the expected NPV for that portfolio. Finally, we tabulate the percent difference between the two. Note, that the sampling procedure may generate projects which are not resource constrained, that is, the sum of the efficient limits may be less than the available budget. In such cases, Theorem 5.3 guarantees that the index policy will be optimal. In order not to bias our results, we ignore such portfolios. Thus, we tabulate errors only when there is a chance of making an error.

(23)

2 Projects 3 Projects 4 Projects 5 Projects

%Error CDF Std. Dev. CDF Std. Dev. CDF Std. Dev. CDF Std. Dev.

0 0.7729 2.09E-05 0.585 5.74E-05 0.486 3.79E-04 0.47 2.07E-03 1 0.9037 1.36E-05 0.825 6.92E-05 0.784 3.98E-04 0.77 2.13E-03 2 0.9442 1.04E-05 0.898 9.98E-05 0.875 5.82E-04 0.87 3.00E-03 3 0.9657 8.13E-06 0.936 1.12E-04 0.921 6.62E-04 0.92 3.52E-03 4 0.9785 6.44E-06 0.958 1.20E-04 0.949 7.00E-04 0.95 3.78E-03 5 0.9863 5.15E-06 0.972 1.24E-04 0.966 7.27E-04 0.97 3.89E-03 7.5 0.9954 2.98E-06 0.990 6.84E-05 0.987 4.04E-04 0.99 2.13E-03 10 0.9985 1.67E-06 0.996 6.75E-05 0.995 4.03E-04 1.00 2.11E-03

15 0.9999 4.10E-07 1.00 2.91E-05 0.999 1.94E-04 1.00 1.06E-03

20 1.0000 0.0 1.00 2.11E-05 1.00 2.10E-04 1.00 8.66E-04

25 1.0000 0.0 1.00 0.0 1.00 1.64E-04 1.00 4.30E-09

. . . . . . . . . . . . . . . . . . . . . . . . . . .

100 1.0000 0.0 1.00 0.0 1.00 0.0 1.00 0.0

Sample Size 5.19 E08 1.26 E08 3.53 E06 1.25 E05

Mean 0.003 0.006 0.008 0.008

Variance 0.0001 0.0002 0.0003 0.0003

Std. Dev. 0.011 0.015 0.017 0.018

Table 2: Percentage Errors from using Index Policy for Projects with Single Task

So, in reality, the average errors we report below are higher than the true average across the entire sample space.1

The computation time required to find the optimal policy becomes prohibitively expensive if the number of projects or the number of tasks per project is increased. Therefore, we first study the effect of increasing the number of projects in the portfolio. Then we look at the effect of increasing the number of tasks per project.

First we show results for portfolios where projects have only one task. Table 2 shows the result of the computations for up to a 5 project portfolio. As can be seen, the average error

1A technical note on the sampling procedure: It is known that sampling from the Uniform distribution

in higher dimensions results in “banding”, that is, “thek-tuples (Ui, . . . , Ui+k−1) will always lie on a finite

number of hyper-planes in [0,1]k” (for example see [21] pp. 22). To alleviate such problems, we use a fifth

order multiple recursive random number generator with two components as provided by P. L’Ecuyer in [15].

(24)

from using the index policy is under 1% for all cases. Further, the distribution of the error is also shown, with the associated standard deviation for each of the probability estimates. Note, that while the standard deviation of the estimates are quite low, they become higher as number of projects grow. This is because less samples were generated for these projects since the procedure gets computationally intensive. The distribution of the errors are plotted in Figure 3. We see that a large percentage of the portfolios give no error at all and virtually no error above 20% was observed. We also note that while errors increase in the number of projects, the distribution of the 4 project portfolio and the 5 project portfolio are almost identical. Therefore, we expect that portfolios with a higher number of projects will also “converge” to this distribution of errors for this parameter range. However, due to the impractical computation time required to obtain the results for higher number of projects we are unable to verify this statement.

We now look at the effect of increasing the number of tasks per projects. Figure 4 shows the distribution of error for a 2 project portfolio with between 1 and 5 tasks. Again, while errors increase in the number of tasks, they are small and appear to be converging to the distribution for the 5 task case. Again, errors above 10% are very rare and no error above 20% was observed.

These analysis indicate that the index policy should perform extremely well in practical settings. Given that the input data is very difficult to estimate to an accuracy greater than 10%, the results in practice should be virtually indistinguishable from an optimal policy.

7

Conclusion and Future Research

We have shown that the index policy is a simple and effective way to evaluate the payoff and timing effects of projects in a limited-resource portfolio. Because it allows projects to be rated according to an index that can be computed for each project independently, it is

(25)

simple to use and well-suited to scoring models that assign ratings for projects along various dimensions. It is also suited to environments where the set of candidate projects evolves over time, since new projects can be inserted into an existing sequence according to their index. But while the index policy approach captures a useful piece of the innovation management process in a practical way, it leaves out a number of important issues that might be able to be incorporated into a mathematical model. In particular, further work is needed to address the following:

1. Risk: The treatment in this paper only considers expected values but not the variance of return or the likelihood of cash flow problems. Some authors, for example Dixit et. al [5] and Luehrman [18], have suggested a real options approach for incorporating risk into the financial analysis of projects. It remains to be seen whether such an approach could be usefully combined with the optimization framework of this paper.

2. Project Interactions: In many environments the payoffs from projects are dependent on one another. For instance, a pharmaceutical company would not want to fund only projects aimed at developing anti-depressants because the resulting products would be competitors in the marketplace. Further work is needed to extend the approach of this paper to situations where such interactions are important.

3. Information Feedback: In many environments success or failure of one project has an impact on other projects. For instance, a breakthrough on a fuel-cell automotive power plant might substantially reduce the potential returns on an all-electric car. How to represent information accumulation in a framework more sophisticated than the simple 0-1 model of success and failure used in this paper is an interesting and challenging topic.

Acknowledgement: This work supported in part by the National Science Foundation under grant DMI-9732868.

(26)

<1,1>

prioritized

<2,1>

<1,2>

<2,1>

<1,3>

prioritized

<1,1>

<2,1>

<1,2>

prioritized

<2,1>

<1,3>

prioritized 0.37 0.4

Simultaneous task completion

Simultaneous task completion

Allocate 0.4 to task 1 of project 1 at first epo

Allocate 0.37 to task 1 of project 1 at first ep

(27)

0.0 0.2 0.4 0.6 0.8 1.0 1.2 0 2 4 6 8 10 12 14 16 Percent Error Probability 2 Projects 3 Projects 4 Projects 5 Projects

(28)

0.0 0.2 0.4 0.6 0.8 1.0 1.2 0 5 10 15 20 Percentage Error Probability 1 Task 2 Tasks 3 Tasks 4 Tasks 5 Tasks

(29)

References

[1] Augood, D. R. (1973) “A review of R&D evaluation methods,” IEEE Transactions on

Engineering Management Vol. EM-20(4), pp. 114-20.

[2] Baker, Norman R. (1974) “R&D project selection models: an assessment,”IEEE Trans-actions on Engineering Management, Vol. EM-21(4), pp. 165-171.

[3] Baker, J. R. and J. R. Freeland, (1975) “Recent advances in R&D benefit measurement and project selection methods,” Management Science, Vol. 21(6) 1164-1175.

[4] Chung, A., W. D. Cook and M. Kress (1998) “Resource allocation in R&D depart-ments,” INFOR Vol. 36(1/2), pp. 41-56.

[5] Dixit, A. K. and R. S. Pindyck, (1995) “The options approach to capital investment,”

Harvard Business Review, Vol. 73(3), pp. 105-115.

[6] Gerchak, Y. (1998) “On allocating R&D budgets among and within projects” R&D Management Vol. 28(4), pp. 305-309.

[7] Gandforoush, P., P. Y. Huang and L. J. Moore (1992) “Multi-project, multi-criteria evaluation and selection model for R&D management,” Management of R&D and En-gineering, D. F. Kocaoglu (Ed.) Elsevier Science Publishers, pp 89-100.

[8] Gupta, D. K., T. Madakovic (1992) “Contemporary approaches to R&D project selec-tion, a literature survey”, Management of R&D and Engineering, D. F. Kocaoglu (Ed.) Elsevier Science Publishers, pp. 67-86.

[9] Heidenberger, K. (1996) “Dynamic project selection and funding under risk: A decision tree based MILP approach,” European Journal of Operations Research, Vol. 95, pp. 284-298.

[10] Kavadias and Loch, “Dynamic portfolio selection of NPD programs using marginal returns”, Working Paper. INSEAD, France.

[11] Krawiec, F. (1984) “Evaluating and selecting research projects by scoring,” Research Management, Vol. 27(2), pp. 21-25.

[12] Kocaoglu, D. F., M. G. Iyigun (1994) “Strategic R&D program selection and resource allocation with a decision support system application,” Proceedings of the IEEE Inter-national Engineering Management Conference 1994, Dayton Ohio, pp. 225-232.

[13] Lai, K. K. and L. Li (1999) “A dynamic approach to multiple-objective resource allo-cation problem,” European Journal of Operations Research Vol. 117, pp. 293-309. [14] Law, A. M and W. D. Kelton (1991) Simulation modeling & analysis, Second Edition,

McGraw Hills.

[15] L’Ecuyer, P. (1999) “Good parameters and implementations for combined multiple re-cursive random number generators,” Operations Research, Vol. 47(1), pp. 159-164. [16] Liberatore, M. J. and G. J. Titus (1983) “The practice of management science in R&D

(30)

[17] Lootsma, F. A., J. Meisner and F. Schellemans (1986) “Multi-criteria decision analysis as an aid to the strategic planning of energy R&D,” European Journal of Operations Research, Vol. 25, pp. 216-234. Management Science Vol. 29(8), pp. 962-974.

[18] Luehrman, T. A. (1998) “Investment opportunities as real options: getting started on the numbers,” (1998) Harvard Business Review, Vol. 76(4), pp. 51-67. option

[19] Madey, G. R. and B. V. Dean (1992) “An R&D project selection and budgeting model using decision analysis and mathematical programming,” (1992) Management of R&D Technology, D. F. Kocaoglu (Ed.) Elsevier Science Publishers. .

[20] Oral, M, O. Kettani and P. Lang (1991) “A methodology for collective evaluation and selection of industrial R&D projects,” Management Science Vol. 37(7), pp. 971-885. [21] Ripley, B. D. (1987) Stochastic Simulation, John Wiley & Sons.

[22] Schmidt, R. L. and J. Freeland (1992) “Recent progress in modeling R&D project-selection processes,” IEEE Transactions on Engineering Management, Vol. 39(2), pp 189-201.

[23] Souder, W. E. (1973) “Analytical effectiveness of mathematical models for R&D project selection”, Management Science Vol. 19(8), pp. 907-923.

[24] Tavares, L. V. (1990) “A multi-stage non-deterministic model for project scheduling under resource constraints,”European Journal of Operations Research, Vol. 64, pp 312-325.

[25] Ulvila, J. W. and J. O. Chinnis, Jr. (1992) “Decision analysis for R&D resource manage-ment,” Management of R&D and Engineering, D. F. Kocaoglu (Ed.) Elsevier Science Publishers.

[26] Winkofsky, E. P., N. R. Baker and D. J. Sweeney (1981) “A decision process model of R&D resource allocation in hierarchical organizations,” Management Science, Vol. 27(3), pp. 268-283. European Journal of Operations Research, Vol. 49, pp. 93-101.

(31)

A

Appendix

We first introduce the following notation, which is used throughout this Appendix.

We denote the allocation to projectiduring thekth task completion interval under policy γ and sample path r by xr

i,k(γ). The duration of the kth task completion interval under a

policy γ and sample path r is denoted by τkr(γ). The completion time of project i under policy γ and sample pathr is denoted astr

i(γ).

Theorem 4.1 For any PPMP there exists an optimal policy that belongs to class.

Proof: We need to show, that for any policyγ /Ω we can find a policy λ Ω which is as good or better than policy γ.

For any policy γ the expected NPV is given by expression (12). Since γ is not in Ω, there must exist some epoch where two (or more) projects are funded at positive levels below their efficient limits. We show that the expected NPV from such a policy can be improved. We show this for policies where an intermediate allocation occurs at the first decision interval. This is sufficient to prove the result since such a an allocation is embedded in the DP given by Equation 6 for the general PPMP.

Let projects 1 and 2 be two of the projects funded at an intermediate level at the first decision epoch. Therefore 0 < xri,1(γ) < li,1 for i = 1,2 and all r (because the first task completion interval is common to all possible sample paths). Now, consider an alternate policyγdefined as follows. In the first decision epoch, we allocatexr

1,1(γ)+to

project 1 andxr2,1(γ)to project 2, for some non-zero, such that, the task completion sequence for every possible sample path remains unchanged from those under γ. All other allocations are kept exactly the same as in policyγ. Notice, that such a policy is guaranteed to be feasible for a sufficiently small because γ /Ω. For any policy in Ω, if projects are prioritized at decision epoch 1, then any extra allocation will result in violation of the efficient limit constraint. Alternately, if the policy induces simultaneous task completions for some sample path, then perturbation by a non-zero may change the completion sequence for at least one sample path. Since,γ /Ω, these cases cannot occur.

We denote the change in the completion time due to this perturbation by δr

i(γ, ) = tr

i(γ)−tri(γ) for all iand r. In Lemma A.1, we prove thatδrk(γ, ) = drk,jδrj(γ, ), where drk,j is some constant independent of , that is, δkr(γ, ) is a linear in δrj(γ, ) for any

(32)

k, j ∈M. Using this we can calculate the expected NPV for this policy to be: E[Π(γ)] = r Pr{r} m i=1 αie−βt r i(γ) = r Pr[r] m i=1 αie−β[t r i(γ)+δir()] = r Pr[r] m i=1 αie−β[t r i(γ)+dri,1δ1r()] (19)

where the last equality follows from Lemma A.1. Substitutingδr

1() from Equation (31), we get E[Π(γ)] = r Pr{r} m i=1 αie −β tri(γ)+drixr 1,1(γ)+ (20) for somedri independent of .

We first consider the case when δr

i() is non-zero for at least one (i, r). For this case

we show that E[Π(γ)] is maximized only at the extreme points of . Differentiating

E[Π(γ)] once with respect to we get,

dE[Π(γ)] d = r Pr{r} x r 1,1(γ)β xr 1,1(γ) + 2 m i=1 αidrie −β tri(γ)+dirxr1,1(γ)+ (21)

Let this derivative vanish at = . Then, observing that in Equation (21) the terms outside the parenthesis are positive, we can write

m i=1 αidrie −β tri(γ)+drixr 1,1(γ)+ = 0, for all r (22)

Now consider the second derivative with respect to ,

d2E[Π(γ)] d 2 = r Pr{r} x r 1,1(γ)β xr 1,1(γ) + 2 m i=1 αidri 2(+xr1,1(γ)) +driβxr1,1e−β tri(γ)+drixr 1,1(γ)+ (23)

(33)

which after rearranging can be written as: d2E[Π(γ)] d 2 = r Pr{r} x r 1,1(γ)β xr 1,1(γ) + 4 2(+xr1,1(γ)) m i=1 αidrie −β tri(γ)+dirxr1,1(γ)+ +βxr1,1(γ) m i=1 αi(dri)2e −β tri(γ)+dirxr1,1(γ)+ (24)

At = , we can use Equation (22) to write the expression for the second derivative as, d2E[Π(γ)] d 2 = r Pr{r} (x r 1,1(γ)β)2 xr 1,1(γ) + 4 m i=1 αi(dri)2e −β tri(γ)+drixr 1,1(γ)+ (25)

which is clearly positive. Therefore, any interior point is a minimum rather than a maximum. Hence, the maximum must occur at an extreme point of . An extreme point of in the neighborhood of tr

1(γ) is reached for an such that either the efficient

limit constraint is tight or at the point where two (or more) tasks along some sample path (or paths) complete simultaneously. The latter case gives an extreme point since when more than a single task complete together for some sample path r due to the above allocation, the task completion sequence Sr(γ

)=Sr(γ) and equation (19) is no

longer valid. Finally, if all δr

i() = 0, then changing does not have any effect on the completion

times of the projects. Hence, we can still change such that we reach a policy in Ω. This proves the result.

Lemma A.1 δr

k(γ, ) = drk,jδjr(γ, ), where drk,j is some constant independent of , that is, δr

k(γ, ) is a linear in δjr(γ, ) for any k, j ∈M.

Proof: To prove the result we need to find the dependence of the first period’s allocation on the completion times of the projects. This is done as follows.

Suppose policyγ /Ω under sample pathr induces a task completion sequenceSr(γ) = {(ik, jk)}|k=1,...,Kr(γ). Further, if s = Skr(γ), then s[1] and s[2] denote the project and

task that finish at the kth epoch under policy γ respectively. Also, Kr(γ) denotes the total number of task completion epochs under sample path r. Let θr

i,j(γ) denote the

position of the jth task completion of project i in Sr(γ); where we define θr

i,0(γ) = 0.

Finally, letνir(γ) denotes the number of tasks of projecticompleted under sample path

(34)

We illustrate this notation via the following example. Suppose policy γ, under some sample pathr induces the task completion sequence

Sr(γ) = {(1,1),(1,2),(2,1),(3,1),(3,2),(2,2),(1,3),(2,3)}

This means, for example, the first task to complete is task 1 of project 1, followed by task 2 of project 1, followed by task 1 project 2 and so on. Then we for project 1 we have θr

1,1(γ) = 1, θr1,2(γ) = 2, θr1,3(γ) = 7, for project 2 we get θr2,1(γ) = 3, θr2,2(γ) = 6,

and so on. Also, since 3 tasks of project 1 complete, we have ν1r(γ) = 3. Similarly

νr

2(γ) = 3 and ν3r(γ) = 2.

We now derive an expression for the completion times tri(γ), for a given sample path. Using the above notation, at the kth task completion interval, task j of project i corresponding to sr

k(γ) gets completed. The resource allocated to this task at this

epoch is xs[1],s[2] for s = Skr(γ). The original amount of work for this task is ws[1],s[2] for s = Sr

k(γ), but at period k some work may have already been completed. This

is calculated as follows. Due to the precedence constraint, the first time any resource may be allocated to this task is after the completion of task j1. Task j1 for this project completes at epoch θr

i,j−1(γ). Therefore, this task may receive allocation from

epoch θr

i,j−1(γ) + 1 onward. Note, for task j = 1, θri,0(γ) is defined to be 0. Therefore,

the amount of work already done for task j of project i at the beginning of the epoch where it completes iskl=θ1r

i,j−1(γ)+1τ

r

l(γ)xri,l(γ). This quantity must be subtracted from

the original amount of work for the task in order to compute the length of this epoch. This analysis allows us to write the expression for τr

k(γ) as, τkr(γ) = ws[1],s[2] k1 l=θi,j−r 1(γ)+1τ r l(γ)xrs[1],l(γ) xs[1],s[2] , s =Skr(γ), k= 1, . . . , Kr(γ) (26) Therefore, the project completion times are obtained as,

tri(γ) =

θi,νrir (γ)(γ)

k=1

τkr(γ), i= 1, . . . , m (27) We are interested in the effect of changing the allocation during the first period. Let the first task to complete under γ be project i, that is, cr

1(γ) = (i,1). We first show

that τr

k(γ) can be written as, τkr(γ) = brk,0+ br k,i+ l∈M\ixrl,1(γ)brk,l xr i,1(γ) , k = 1, . . . , Kr(γ) (28) for someb

References

Related documents

In this PhD thesis new organic NIR materials (both π-conjugated polymers and small molecules) based on α,β-unsubstituted meso-positioning thienyl BODIPY have been

• Follow up with your employer each reporting period to ensure your hours are reported on a regular basis?. • Discuss your progress with

4.1 The Select Committee is asked to consider the proposed development of the Customer Service Function, the recommended service delivery option and the investment required8. It

How Many Breeding Females are Needed to Produce 40 Male Homozygotes per Week Using a Heterozygous Female x Heterozygous Male Breeding Scheme With 15% Non-Productive Breeders.

 Human elements involved in the problem  Support systems surrounding the problem  Tracking systems related to the problem..  Institutional process for managing

Minors who do not have a valid driver’s license which allows them to operate a motorized vehicle in the state in which they reside will not be permitted to operate a motorized

more than four additional runs were required, they were needed for the 2 7-3 design, which is intuitive as this design has one more factor than the 2 6-2 design

The threshold into the stadium is through a series of layers which delaminate from the geometry of the field to the geometry of the city and creates zones of separation,