Measuring effectiveness through code coverage

* Produces a valid social security string (with dashes)

* @param param1 Area Number -> JSC.integer(100, 999)

* @param param2 Group Number -> JSC.integer(10, 99)

* @param param3 Serial Number -> JSC.integer(1000,9999)

* @returns {Function} Specifier function

*/

JSC.SSN = function (param1, param2, param3) { return function generator() {

const part1 = typeof param1 === 'function'

? param1(): param1;

const part2 = typeof param2 === 'function'

? param2(): param2;

const part3 = typeof param3 === 'function'

? param3(): param3;

return [part1 , part2, part3].join('-');

};

};

JSCheck works only with pure programs, which means you can’t test the showStudent program entirely, but you can use it to test each component in isolation. I leave that to you as an exercise. Property-based testing is compelling because it exercises functions to the limit. Its best quality, in my opinion, is that it can be used to verify whether code is indeed referentially transparent, because it’s expected to work consistently against the same contract and verdict. But why submit your code to such a heavy procedure?

The answer is simple: to make your tests effective.

6.5 Measuring effectiveness through code coverage

Measuring a unit test’s effectiveness is an arduous task if not done with the proper tools in place, because it involves studying the test’s code coverage through the func-tions under test. Getting coverage information involves traversing all unique paths belonging to a program’s control flow; one way to achieve this is by studying the flow of code against a function’s boundary conditions.

Certainly, code coverage alone isn’t an indicator of quality, but it does describe the degree to which your functions are tested, which correlates to better quality. Would you want code that’s never seen the light of day deployed to production? I didn’t think so.

Code-coverage analysis can find areas in your code that haven’t been tested, allow-ing you to create additional tests to uncover them. Normally, this includes code for error handling that you let slip through the cracks and forget to come back to. You can use code coverage to measure the percentage of lines of code that are executed when invoking a program via unit tests. To compute this information, you can use a library called Blanket.js, which is a code-coverage tool for JavaScript. It’s designed to

Listing 6.6 Custom JSC.SSN specifier

Added as part of the JSC object so the code looks consistent

Each part of the SSN number is made up of either a constant or a function that JSCheck uses to inject random inputs.

All three data points are combined into a valid SSN syntax.

complement your existing JavaScript unit tests with code-coverage statistics. It works in three phases:

1 Load source files

2 Instrument the code by adding tracker lines

3 Connect the hooks in the test runner to output coverage details

Blanket collects coverage information with the help of an instrumentation phase during which it captures meta-information regarding statement execution, which you can display nicely in a QUnit report. Details for setting up Blanket can be found in the appendix. You can instrument any JavaScript module or program via the cus-tom data-covered attribute in the script include line. By analyzing the statement-coverage percentage, you can see that functional code is much more testable than imperative code.

6.5.1 Measuring the effectiveness of testing functional code

Throughout this chapter, you’ve seen that functional programs are more testable due to the ease with which tasks can be broken apart to become atomic, verifiable units.

But don’t take my word for it; you can measure it empirically by performing a state-ment-by-statement percentage-coverage analysis on the showStudent program. First, let’s look at the simplest test case: a positive test.

MEASURINGEFFECTIVENESSOFIMPERATIVEANDFUNCTIONALCODEWITHVALIDINPUTS

First let’s look at code-coverage statistics against a successful run of the imperative ver-sion of showStudent, shown in listing 6.2. Using Blanket with QUnit, mark this pro-gram to be instrumented:

<script src="imperative-show-student-program.js" data-cover></script>

Now, running the following test

QUnit.test('Imperative showStudent with valid user', function (assert) { const result = showStudent('444-44-4444');

assert.equal(result, '444-44-4444, Alonzo, Church');

});

produces an 80% total statement-coverage percentage, as shown in the QU nit/Blan-ket output in figure 6.10.

This shouldn’t surprise you, because the error-handling code was all skipped. For imperative programs, 75–80% code coverage is considered to be very good. What you can take from this run is that 80% is the best coverage you can get with a single unit test execution. On the other hand, let’s instrument and run a positive test against the functional version:

<script src="functional-show-student-program.js" data-cover></script>

Again, running the “happy path” test runs the program with a valid SSN, but this time producing a whopping figure of 100% coverage (see figure 6.11)!

But wait: if the input was valid, why didn’t it skip the error-handling logic? This is the work of monads in the code, which can propagate the concept of an empty value, or nothingness (in the form of an Either.Left or a Maybe.Nothing) seamlessly through-out the entire program; thus, every function is run, yet logic encapsulated in mapping functions is skipped.

It’s remarkable how functional code is so robust and flexible. Now, let’s run a neg-ative test with invalid input.

Figure 6.10 QUnit/Blanket output running the imperative showStudent with valid input. The highlighted lines represent statements that never ran. Because 12 of 15 lines ran, this registers only 80% of total coverage information on this function.

Figure 6.11 A positive unit test against the functional showStudent generates a 100% line-percentage coverage. Every line of the testable business logic is executed!

MEASURINGEFFECTIVENESSOFIMPERATIVEANDFUNCTIONALCODEWITHINVALIDINPUTS

Let’s measure the effectiveness of both programs when run with invalid conditions, such as when the input is null. As you can see from figure 6.12, the imperative code reports (not surprisingly) a mediocre coverage value:

QUnit.test('Imperative Show Student with null', function (assert) { const result = showStudent(null);

assert.equal(result, null);

});

This result is due to the presence of if-else blocks that create divergent control flow that branches in different directions. As you’ll see shortly, this also leads to complex functions.

In contrast, the functional program handles the null case much more gracefully, because it only skips logic that would manipulate the invalid input (now null) directly. But the entire structure of the program (the interaction among functions) stays put and is successfully invoked and tested from start to finish. Recall that because there’s an error, the output of the functional code is a Nothing. You don’t have to check for a null output—the following test case is sufficient:

QUnit.test('Functional Show Student with null', function (assert) { const result = showStudent(null).run();

assert.ok(result.isNothing);

});

Figure 6.12 The imperative version of showStudent skips the positive path of execution, which translates to only a few lines being executed and a low 40% coverage.

Figure 6.13 shows the areas that were left untouched due to the skipped logic.

Even in the presence of invalid data, the functional program doesn’t just skip execution of entire sections of code. It gracefully and safely propagates the invalid condition in monads, outputting a decent 80% (twice as much as the imperative counterpart); see figure 6.14.

Because it’s a lot more testable, the functional code should give you a sense of security and comfort to deploy it to your production systems—in case immutability and elimi-nation of side effects hasn’t done the trick. As mentioned earlier, the presence of con-ditional and loop blocks in imperative code not only makes it hard to test and hard to

Figure 6.13 The functional version of showStudent skips lines related only to manipulating the data that would have originated from an otherwise valid input.

Figure 6.14 The functional showStudent continues to yield great coverage results even against invalid inputs.

reason about, but also further increases the complexity of the function in question.

How can you measure complexity?

6.5.2 Measuring the complexity of functional code

You can measure a program’s complexity by closely examining its control flow. At a glance, you determine that a block of code is complex when it’s visually difficult to fol-low. Functional programming presents a nice declarative view of the code that makes it visually appealing. This equates to reduced complexity from the developer’s point of view. In this section, you’ll see that functional code is also less complex from an algorithmic point of view.

Many factors can contribute to complex code, including conditional blocks and loops, which can also be nested in other structures. Branching logic, for instance, is mutually exclusive and splits the control-flow logic into two independent branches according to a Boolean condition. Multiple if-else blocks in your code can be hard to trace; the process is even harder when their conditions are based on external fac-tors—side effects dictating the path the code should follow. The higher the number of conditional blocks and nested conditional blocks, the harder functions are to test, which is why it’s important to keep your functions as simple as possible. This is deeply rooted in FP’s philosophy of reducing all functions to simple lambda expressions whenever possible and combining them using composition and monads.

Cyclomatic complexity (CC) is a quantitative software metric used to measure the number of linearly independent paths that functions take. From this concept comes the idea of verifying a function’s boundary conditions, to ensure that all possible paths through the functions are tested. This is accomplished with some simple graph theory of nodes and edges (as shown in figure 6.15):

■ Nodes correspond to indivisible blocks of code.

■ Directed edges connect two blocks of code if the second block can be possibly executed after the first.

B

C A

if-else

for

A

B C

filter ^reduce map

if-else for

Nodes Edges

Imperative Functional

Becomes

Figure 6.15 Imperative if-else blocks and for loops in imperative code are translated into the use of map, filter, and reduce in functional programs.