Termination check:

If_兩ƒ_⬘(x)_{兩 ⭐ ␧1}and _兩(x_⫺ x₁) /x_{兩 ⭐ ␧2}, stop. Otherwise, set (i) x_{2 ⫽} x₁and x_{1 ⫽} x if ƒ_⬘(x)ƒ_⬘(x₁)_⬍ 0

(ii) x_{1 ⫽} x ifƒ_⬘(x)ƒ_⬘(x₂)_⬍ 0 In either case, continue with step 3.

Note that steps 1 and 2 constitute a bounding search proceeding along an expanding pattern in which the sign of the derivative is used to determine when the optimum has been overshot. Step 3 represents the calculation of the optimum of the approximating cubic. Step 4 is merely a safety check to ensure that the generated estimate is in fact an improvement. Provided that direct values of the derivative are available, this search is unquestionably the most efficient search method currently available. However, if derivatives must be obtained by differences, then the quadratic-based search of Powell is pre-ferred.

Example 2.12 Cubic Search with Derivatives

Consider again,

2 16 Minimize ƒ(x) _⫽ 2x _⫹

x

with the initial point x_0⫽1 and step size_{⌬ ⫽}1. For convergence parameters use

⫺2 ⫺2

␧ ⫽₁ 10 _{␧ ⫽2} 3 _⫻10

dƒ 16

ƒ_⬘(x)_{⫽ ⫽}4x _{⫺ 2}

dx x

Iteration 1

Step 1. ƒ_⬘(1)_{⫽ ⫺}12 _⬍ 0. Hence, x_{1 ⫽} 1 _⫹ 1_⫽ 2.

Step 2. ƒ_⬘(2)_⫽ 4

Since ƒ_⬘(1)ƒ_⬘(2)_{⫽ ⫺}48_⬍0, a stationary point has been bracketed between 1 and 2. Set x_{1 ⫽} 1, x_{2 ⫽} 2. Then, ƒ_{1 ⫽} 18, ƒ_2⫽ 16, ƒ_{⬘1 ⫽ ⫺}12, ƒ_{⬘2 ⫽} 4.

–3

ƒ_⬘(1.5657)_{⫽ ⫺}0.2640 clearly not terminated Since search of Example 2.9 returned an estimate x _⫽ 1.714, whereas the cubic search produced 1.5657. The exact minimum is 1.5874, indicating the clear superiority of the higher order polynomial fit.

(k _⫹ 1)st estimate from the true minimum is proportional to the deviation of the kth estimate raised to the _␣ power, where _{␣ ⬎} 1 (_{␣ ⫽} 1.3 has been claimed [6]). By contrast, if in the case of the golden section search the kth estimate of the true minimum is taken to be the midpoint of the interval remaining after k evaluations, then the deviation of this estimate from the actual minimum decreases linearly from the kth to the (k _⫹ 1)st iterations.

Assuming comparable regions of convergence for both methods, this indicates that the quadratic search method will converge more rapidly than any region elimination scheme. Of course, this comparison is based on assuming com-parable regions of convergence and on well-behaved unimodal functions.

The limited numerical comparisons available in the literature do not indi-cate any overwhelming advantage of either the derivative-based or the quad-ratic or region elimination search method over the others. If functional evaluations are lengthy and costly in terms of computer time, Box et al. [7]

claim superiority of a search strategy based on a modification of Powell’s method. This claim seems to be supported by the limited computational ex-periments presented by Himmelblau [8], who compares this strategy with one based on the golden section search and shows that a modified Powell’s search requires fewer functional evaluations to attain a specified accuracy in the estimate of the minimizing point. Certainly, if very high accuracy is desired, the polynomial approximation methods are quite superior. On the other hand, for strongly skewed or possibly multimodal functions, Powell’s search method has been known to converge at a much slower rate than region elimination schemes. Hence, if reliability is of prime importance, then golden section is the ideal choice. Because of these considerations, it is recommended that Powell-type search methods generally be used along with a golden section search to which the program can default if it encounters difficulties in the course of iterations. Readers may also refer to Brent [9] for a good discussion of the relative efficiencies of different search methods.

In a small study conducted as a class project for the optimization course at Purdue University, the following point estimation methods were compared on the function ƒ(x)_⫽ sin^kx for different values of k:

Bisection method Powell’s method Cubic search

In addition, the best region elimination method, namely the golden section search, was also included for comparison.

Three criteria—solution time, solution accuracy, and sensitivity to conver-gence parameter—were used to evaluate these methods. The first two criteria were examined by varying the value of the power k. Odd powers between 1 and 79 were chosen for k. Note that the minimum point is always x* _⫽ 4.71239 and ƒ(x*) _{⫽ ⫺}1.0 for all values of k. However, as k increases, the function becomes less smooth and has narrow valleys near the minimum. This would make the point estimation methods slow and less accurate.

The sensitivity criterion was tested by varying the convergence parameter.

As expected, results showed an increase in solution times as k increased. The bisection method showed the greatest increase in solution times with respect to k, due to rapid increase in the values of the gradient close to the minimum.

Golden section search was unaffected by the increase in power k. Similarly, solution accuracy, measured in terms of percentage error to the true minimum, became worse as k increased for all three methods—bisection, Powell, and cubic. However, golden section search was unaffected by changes in the steep-ness of the function. Sensitivity to change in the convergence criterion was minimal for all four methods tested.

2.7 SUMMARY

In this chapter, we discussed necessary and sufficient optimality conditions for functions of a single variable. We showed that a necessary condition for the optimum is that the point be a stationary point, that is, that the first derivative be zero. Second-order and higher order derivatives were then used to test whether a point corresponds to a minimum, a maximum, or an inflec-tion point. Next we addressed the quesinflec-tion of identifying candidate optima.

We developed a number of search methods called region elimination methods for locating an optimal point in a given interval. We showed that the golden section search is generally the preferred algorithm due to its computational efficiency and ease of implementation.

The region elimination method required a simple comparison of function values at two trial points and hence took into account the ordering of the function values only. To involve the magnitude of the difference between the function values as well, we developed point estimation methods. These in-volved a quadratic or cubic approximation of the function to determine the optimum. We pointed out that, assuming comparable regions of convergence, point estimation methods converge more rapidly than region elimination methods as a class for well-behaved unimodal functions. However, for strongly skewed or multimodal functions, the golden section search is more reliable. We concluded with a recommendation that Powell-type successive quadratic estimation search be generally used along with a golden section

ization,’’ ICI Ltd., Central Instr. Res. Lab, Res. Note, 64 / 3, London, 1964.

2. Kiefer, J., ‘‘Optimum Sequential Search and Approximation Methods under Min-imum Regularity Assumptions,’’ J. Soc. Ind. Appl. Math., 5(3), 105–125 (1957).

3. Bartle, R., Elements of Real Analysis, Wiley, New York, 1976.

4. Powell, M. J. D., ‘‘An Efficient Method for Finding the Minimum of a Function of Several Variables without Calculating Derivatives,’’ Computer J., 7, 155–162 (1964).

5. Raphson, J., History of Fluxion, London, 1725.

6. Kowalik, J., and M. R. Osborne, Methods for Unconstrained Optimization Prob-lems, American Elsevier, New York, 1968.

7. Box, M. J., D. Davies, and W. H. Swann, Nonlinear Optimization Techniques, ICI Monograph, Oliver and Boyd, Edinburgh, 1969.

8. Himmelblau, D. M., Applied Nonlinear Programming, McGraw-Hill, New York, 1972.

9. Brent, R. P., Algorithms for Minimization without Derivatives, Prentice-Hall, En-glewood Cliffs, NJ, 1973.

10. Phillips, D. T., A. Ravindran, and J. J. Solberg, Operations Research: Principles and Practice, Wiley, New York, 1976.

PROBLEMS

2.1. What is an inflection point and how do you identify it?

2.2. How do you test a function to be convex or concave?

2.3. What is the unimodal property and what is its significance in single-variable optimization?

2.4. Suppose a point satisfies sufficiency conditions for a local minimum.

How do you establish that it is a global minimum?

2.5. Cite a condition under which a search method based on polynomial interpolation may fail.

2.6. Are region elimination methods as a class more efficient than point estimation methods? Why or why not?

2.7. In terminating search methods, it is recommended that both the differ-ence in variable values and the differdiffer-ence in the function values be

Figure 2.17. Problem 2.9 schematic.

tested. Is it possible for one test alone to indicate convergence to a minimum while the point reached is really not a minimum? Illustrate graphically.

2.8. Given the following functions of one variable:

(a) ƒ(x)_⫽ x⁵ _⫹ x⁴_{⫺ ⫹} 2 x3

3 (b) ƒ(x)_⫽ (2x _⫹ 1)²(x_⫺ 4)

Determine, for each of the above functions, the following:

(i) Region(s) where the function is increasing; decreasing (ii) Inflexion points, if any

(iii) Region(s) where the function is concave; convex (iv) Local and global maxima, if any

(v) Local and global minima, if any

2.9. A cross-channel ferry is constructed to transport a fixed number of tons (L) across each way per day (see Figure 2.17). If the cost of construc-tion of the ferry without the engines varies as the load (l) and the cost of the engines varies as the product of the load and the cube of the speed (v), show that the total cost of construction is least when twice as much money is spent on the ferry as on the engine. (You may neglect the time of loading and unloading and assume the ferry runs continu-ously.)

2.10. A forest fire is burning down a narrow valley of width 2 miles at a velocity of 32 fpm (Figure 2.18). A fire can be contained by cutting a fire break through the forest across the width of the valley. A man can clear 2 ft of the fire break in a minute. It costs $20 to transport each man to the scene of fire and back, and each man is paid $6 per hour while there. The value of timber is $2000 per square mile. How many men should be sent to fight the fire so as to minimize the total costs?

2.11. Consider the unconstrained optimization of a single-variable function ƒ(x). Given the following information about the derivatives of orders 1, 2, 3, 4 at the point xi(i_⫽1, 2, . . . , 10), identify the nature of the test point xi(i.e., whether it is maximum, minimum, inflection point, non-optimal, no conclusion possible, etc.):

x₁ 0 _⫹ Anything Anything

x₂ 0 0 _⫹ Anything

x₃ 0 _⫺ Anything Anything

x_{4 ⫺ ⫺} Anything Anything

x₅ 0 0 _⫺ Anything

x₆ 0 0 0 _⫺

x₇ 0 0 0 0

x₈ 0 0 0 _⫹

x_{9 ⫹ ⫹} Anything Anything

x₁₀ 0 _{⫺ ⫹ ⫺}

2.12. State whether each of the following functions is convex, concave, or neither.

(a) ƒ(x)_⫽ e^x (b) ƒ(x) _⫽ e^⫺x (c) ƒ(x)_⫽ 1

x2

(d) ƒ(x) _⫽ x _⫹ log x for x _⬎ 0 (e) ƒ(x)_{⫽ 兩}x_兩

(f) ƒ(x)_⫽ x log x for x _⬎ 0

(g) ƒ(x)_⫽ x^2k where k is an integer (h) ƒ(x) _⫽ x^2k⫹1 where k is an integer 2.13. Consider the function

ƒ(x)_⫽ x3 _⫺ 12x_⫹ 3 over the region_⫺4_⭐ x_⭐ 4

Determine the local minima, local maxima, global minimum, and global maximum of ƒ over the given region.

2.14. Identify the regions over which the following function is convex and where it is concave:

⫺x2

ƒ(x)_⫽ e

Determine its global maximum and global minimum.

2.15. Consider a single-period inventory model for perishable goods as fol-lows:

Demand is a random variable with density ƒ; i.e., P(demand _⭐ x) _⫽ ƒ(x) dx.

兰x0

All stockouts are lost sales.

C is the unit cost of each item.

p is the loss due to each stockout (includes loss of revenue and good-will).

r is the selling price per unit.

l is the salvage value of each unsold item at the end of the period.

The problem is to determine the optimal order quantity Q that will maximize the expected net revenue for the season. The expected net revenue, denoted by_⌸(Q), is given by

Q

⌸(q)_⫽ r_{␮ ⫹} l

冕

0 ^{(Q⫺ x)ƒ(x) dx}

⫺ (p_⫹ r)

冕

Q⬁^{(x⫺ Q)ƒ(x) dx⫺ CQ}

where_{␮ ⫽}expected demand _{⫽ 兰}^⬁₀xƒ(x) dx.

(a) Show that_⌸(Q) is a concave function in Q(_⭓ 0).

(b) Explain how you will get an optimal ordering policy using (a).

(c) Compute the optimal order policy for the example given below:

C_⫽ $2.50 r_⫽ $5.00 l _⫽ 0 p_⫽ $2.50 1 for 100_⭐ x_⭐ 500

ƒ(x)_⫽

冦

400^{0 otherwise}

Hint: Use the Leibniz rule for differentiation under the integral sign.

2.16. Suppose we conduct a single-variable search using (a) the golden sec-tion method and (b) the bisecsec-tion method with derivatives estimated numerically by differences. Which is likely to be more efficient? Why?

2.17. Carry out a single-variable search to minimize the function

2 12 –1 –5

ƒ(x)_⫽ 3x _{⫹ 3 ⫺} 5 on the interval _{2 ⭐}x _{⭐ 2} x

3 2 2

ƒ(x)_⫽ (10x _⫹ 3x _⫹ x_⫹ 5) starting at x_⫽ 2 and using a step size_{⌬ ⫽} 0.5.

(a) Using region elimination: expanding pattern bounding plus six steps of golden section.

(b) Using quadratic point estimation: three iterations of Powell’s method.

2.19. Determine the real roots of the equation (within one decimal point accuracy)

2 5 6

ƒ(x)_⫽ 3000_⫺ 100x _⫺4x _⫺6x _⫽ 0

using (a) the Newton–Raphson method, (b) the bisection method, and (c) the secant method. You may find it helpful to write a small computer program to conduct these search methods.

2.20. (a) Explain how problem 2.19 can be converted to an unconstrained nonlinear optimization problem in one variable.

(b) Write a computer program for the golden section search and solve the optimization problem formulated in part (a). Compare the so-lution with those obtained in problem 2.19.

2.21. An experimenter has obtained the following equation to describe the trajectory of a space capsule (Phillips, Ravindran, and Solberg [10]):

3 2 x / 2

ƒ(x)_⫽4x _⫹ 2x _⫺3x _⫹ e

Determine a root of the above equation using any of the derivative-based methods.

2.22. Minimize the following functions using any univariate search method up to one-decimal-point accuracy (Phillips, Ravindran, and Solberg [10]).

(a) Minimize ƒ(x) _⫽ 3x⁴_⫹ (x _⫺ 1)² over the range [0, 4].

(b) Minimize ƒ(x) _⫽ (4x)(sin x) over the range [0,_⌸].

(c) Minimize ƒ(x) _⫽ 2(x_⫺ 3)² _⫹ e^0.5x2over the range (0, 100).

2.23. In a chemical plant, the costs of pipes, their fittings, and pumping are important investment costs. Consider the design of a pipeline L feet long that should carry fluid at the rate of Q gpm. The selection of economic pipe diameter D (in.) is based on minimizing the annual cost of pipe, pump, and pumping. Suppose the annual cost of a pipeline with a standard carbon steel pipe and a motor-driven centrifugal pump can be expressed as

Formulate the appropriate single-variable optimization problem for de-signing a pipe of length 1000 ft with a flow rate of 20 gpm. The di-ameter of the pipe should be between 0.25 and 6 in. Solve using the golden section search.

2.24. Solve the following problem by any univariate search method:

6_⫺ x

1 2

Minimize ƒ(x)_{⫽ 兹}–₃ x _⫹25 _⫹ 5 Subject to 0_⭐x _⭐ 6

2.25. You are interested in finding the positive root of the following equa-tion:

2 3

ƒ(x)_⫽ 3000_⫺ 100x_⫺ 50x _⫺ 4x

Given the initial bound for the root as (0, 10), apply one step of the secant method to find the first approximation to the root. What will be the new bound for the root?

2.26. John is planning to retire in 30 years with a nest egg of $500,000. He can save $2500 each year toward retirement. What minimum annual return should he get from his savings to reach his retirement goal?

Hint: Annual savings of $1 for N years will accumulate to [(1 _⫹ i)^N _⫺ 1] / i, where i is the compounded annual rate of return.

2.27. The XYZ Company plans to invest $10 million in a new manufacturing process that will result in an estimated after-tax benefit of $2 million over the 8-year service life of the process. Compute the annual rate of return for the investment.

2.28. (a) Using golden section search and 10 functional evaluations, what is the interval reduction we can achieve?

(b) To achieve the same interval reduction, how many function evalu-ations would be needed in the interval-halving method?

78

3

FUNCTIONS OF

In document Engineering Optimization (Page 82-93)