Case Study: Computing the Square Root

Suppose that we want to compute the square root of a given value. While themathlibrary already includes asqrtfunction for this purpose, it is instructive to see that we can perform the calculation ourselves.

We will use an approach known as Newton’s method. The technique is based on developing successive approximations to the answer. That is, we will start with a guess and repeatedly improve our guess. When computing a square root, ifguess == number/guess, we have the right answer (by rearranging, we seeguess*guess == number). More impor-tantly, if the guess is wrong, the true square root lies betweenguessandnumber/guess. By taking their average, we get a new guess that is better than the previous one. By repeating this process, we get closer to the correct answer.

As a first attempt we suggest the following implementation, which we namesqrtAto distinguish it from thesqrtfunction of themathlibrary:

def sqrtA(number):

guess = 1.0

while guess != number / guess:

guess = (guess + number / guess) / 2.0 # use average return guess

Interestingly, this function works in some cases but not others. For instance, a call to sqrtA(4.0)will probably return2.0, and a call tosqrtA(100.0)will return10.0. Yet some-thing goes wrong in other cases. As an experiment, try to compute a handful of other values: sqrtA(2.0), sqrtA(3.0), sqrtA(5.0), sqrtA(6.0). You will find that many of these calls never return; the culprit is an infinite loop.

In theory, Newton’s method leads to better and better approximations to the true square root. But there is no guarantee that it will ever reach the exact square root. This fact is compounded by an inherent limit in precision when using floating-point values.

Even though floating-point values can represent a wide range of numbers, they cannot perfectly represent every possible number. As a result, some of the arithmetic operations may have a small but noticeable error. Worse yet, the exact behavior may depend on your computer system, as Python typically relies upon the underlying computer hardware to perform floating-point computations. Different computers may use different floating-point implementations.

For thesqrtAimplementation, the problem is that this while loop terminates only when we reach values that precisely satisfy the equation guess == number/guess. For some numbers, this is impossible when using floating-point values. Although we have to live with the fact that we cannot compute square roots exactly, it does not seem reasonable to live with a function that never returns. Our challenge is to determine a better way to halt the loop. We look at several possible approaches.

Fixed number of iterations

A sure way to avoid an infinite loop is to perform only a fixed number of iterations of Newton’s method using a for loop. The question then becomes, how many iterations? For example, we could perform 100 iterations using the following code:

def sqrtB(number):

guess = 1.0

for trial in range(100):

guess = (guess + number / guess) / 2.0 # use average return guess

But it is not clear why to pick 100. There is a clear trade-off. The more iterations we do, presumably the better answer we reach. We could go further and do one million iterations while still ensuring that we avoid an infinite loop. But the more iterations we do, the more time consuming the process will become. In fact in some cases, we reach our final answer

If you want to dig deeper into the oddities that stem from the limited precision of floating-point values, try the following experiment on your machine. Different machines may have different behaviors, so don’t be surprised if the experiment does not turn out the same way for you. Feel free to try similar experiments with other values.

When using one of the standard floating-point implementations, a call to sqrtA(5.0)actually returns an answer (approximately 2.2360679774997898). Yet in reality, we know that√

5 is an irrational number and so there is no finite repre-sentation for the true value. We might wonder why our implementation of New-ton’s method ever completes.

The explanation of such behavior is to recognize that we did not need to locate the true square root in order for our while loop to end. We simply needed to reach a value satisfying the equalityguess == number/guess. This happens to be satisfied by our “solution” tosqrtA(5.0). We have a value that is not quite the true square root, but which matches the result of a (slightly inaccurate) division.

The following interpreter session demonstrates the experiment:

>>> s = sqrtA(5.0)

>>> print s == 5.0/s True

Perhaps even stranger yet, floating-point numbers can defy the basic rules of algebra. Ifsand5.0/sare the same then algebraically_s*sshould equal5.0. Yet in our experiment, the following test fails:

>>> print s*s == 5.0 False

The moral of this story is to be very cautious when using the precision of floating-point numbers. You should not rely on precise equality tests (i.e.,==) when using floating-point numbers, and similar oddities can be observed with many other arithmetic operations.

much earlier. For example, on our machine the computation of sqrtB(4.0)reaches the answer2.0after the fifth iteration. From that point on, we spend iteration after iteration revising our guess to be the average of2.0and2.0. In cases where the process converges on an answer, it typically does so in 10 to 20 iterations, though this is not guaranteed. By picking a fixed number of iterations in advance, we may get a good answer but perhaps not the best result that would be achieved if we had been willing to do more iterations.

Iterate until gap is small

Intuitively, another approach is to use a while loop that dictates that we continue comput-ing successive approximations until we get “close enough.” Of course, we need to define what constitutes close enough. One way to do this is to ask the user for an acceptable error bound; another way is for us to choose the error bounds. The following implementation accepts the allowable error as a parameter, yet with a default value of one millionth:

def sqrtC(number, allowableError=.000001):

guess = 1.0

while abs(guess−number / guess) > allowableError:

guess = (guess + number / guess) / 2.0 # use average return guess

Notice that the while loop formulation is quite similar to that ofsqrtA, but instead of requiring thatguessprecisely equalnumber/guess, we now continue looping so long as the difference between the two (in absolute value) is greater than the acceptable level of error. Our default error bound seems to be reasonable, and indeed it ensures that the function completes for many inputs. But it may cause the calculations to stop sooner than necessary, when more iterations could continue to improve the accuracy.

Even worse, the use of an error bound does not guarantee that the loop always ends.

For example, if the user were to specify an allowable error of0.0, we are in a situation similar tosqrtA. Even with our default error bound we can end up in an infinite loop. The precision of floating-point numbers decreases when the numbers are large, and so there may not be a solution that satisfies the error bound. Admittedly, we only seem to run into this trouble when taking the square root of a sextillion or more (that is, over 21 digits long).

Iterate until no more improvement

Our final approach uses a while loop to iterate so long as our estimate continues to improve.

In this way, we can get as good a solution as possible with floating-point precision, while ensuring that we do not enter an infinite loop nor waste time doing extra iterations that do not improve the result.

Our way to measure improvement is based on the following. If we had infinite pre-cision the two key values in Newton’s method, namelyguessandnumber/guess, serve as bookends. If they are not precisely equal to each other then one will be less than the true answer and the other greater. Each successive approximation causes those bookends to move closer to each other. With the limited precision of floating-point numbers, what happens is that the numbers reach a point where no further improvement can be made or, worse yet, take a step farther apart due to error introduced into the calculation.

To implement this approach, we need to alter our code slightly from previous ver-sions. Because we want to know whether we have improved from one iteration to the next, we need to keep track of not just the current guess, but also the previous guess.

Our goal will be to measure whether the lower of the two bookends continues to increase (with previous implementations, our guess oscillates between too small and too big). Our implementation is as follows.

def sqrtD(number): guess = min(avg, number / avg) return guess

The strict inequality in our while loop conditionprevGuess < guess ensures our notion of continued improvement. As a technicality, we had to make sure to declare an initial

“previous” guess of0.0to ensure that the loop was entered.

This final implementation is superior to any of our earlier versions. If the calculations converge to an answer after only a few iterations, our loop stops and returns the result. If it can keep making progress it does so. More importantly, calling this function will never result in an infinite loop. If we had infinite precision, Newton’s method would go on forever in certain cases, such as calculating approximations to the irrational number√

2.

But since a floating-point number has finite granularity, we cannot continue finding better floating-point values indefinitely.²Eventually the guess fails to improve, ending our loop.

Taking square roots of negative numbers

None of the implementations we have suggested work properly if the user sends a negative number as the parameter. Of course there is no real number representing the square root of a negative number, so Newton’s method is doomed to fail. The successive approximations fluctuate wildly with no possibility of convergence. Our various implementations fail in subtly different ways. The while loops ofsqrtAandsqrtCwill typically be infinite. The sqrtBfunction is based on a for loop, so it will return a result, although it has little rele-vance. Technically, the while loop ofsqrtDis never entered because the original guess is less than zero, so the original negative number is always returned as the answer.

Clearly, none of these responses seems appropriate. In these cases, we must decide whether the blame lies with us as the programmer or with the caller (or both). As a pro-grammer, we might wish to blame the caller, yet it seems unhelpful for an invalid input to lead to a misleading answer or a never-ending function call (imagine if this were the case for a handheld calculator). A better design would check for such a situation and respond in a more useful fashion. We address just this issue in the next section.