BLACK BOX TESTING

- White box testing - Grey box testing Black Box Testing

BLACK BOX TESTING

Imagine a soda vending machine. If you want a can of Coke, you insert money into the machine and get your soda. Nothing fancy. Now here’s another perspective:

- The coins that you insert into the machine are INPUT.

- The soda is an OUTPUT.

- The buttons to select type of soda and the slot where you insert your coins are the USER INTERFACE (UI).

- The internals of the vending machine is like a black box, because you have a vague or no idea about the concrete mechanism that exchanges coins for soda.

In regard to software, the black box (or area of unknown) is simply a software back end. From a black box tester perspective, the back end can be thought of as a virtual bridge that connects INPUT and OUTPUT.

There are two main things about black box testing:

1. The tester usually has no idea about the internals of the back end.

2. Ideas for testing come from expected patterns of user behavior.

Let's look into the details.

1. The tester usually has no idea about the internals of the back end.

On the one hand, the tester has an advantage over the programmer, i.e., the author of the back end code.

Why? It's human nature to see desired things as reality. The programmer wants to see his or her code working. Every parent thinks that his or her child is the smartest and the most talented. The code is the child of a programmer, and in his or her reality the programmer often perceives his or her code as a problem-free creature.

BTW

Here is my favorite legend about the power of perception:

When the ships of Columbus’ expedition stopped in front of one of the Caribbean islands, the natives didn't see those ships because their perception did not include images that were completely different from what they and their ancestors had been observing for thousands of years. Only their priest felt that something was wrong, and he kept scanning the horizon until he could distinguish the silhouettes of the Spanish frigates from the sea and sky. Then he said, "Check it out, dudes: Columbus with his expedition," and only after that could those natives see reality as it was.

The black box tester has no "bonds" with the code, and a tester's perception is very simple: a code MUST have bugs. Using the principle, "Ask and you shall receive," black box testers find bugs where programmers don't.

BUT, on the other hand, black box testing is like a walk in a dark labyrinth without a flashlight, because the tester doesn't know how the back end was actually constructed. That's why there are situations when

- A black box tester writes many test cases to check something that can be tested by only one test case.

- Some parts of the back end are not tested at all

Therefore, black box testing has the advantage of an unaffiliated opinion on the one hand and the disadvantage of blind exploring on the other.

2. Ideas for testing come from expected patterns of user behavior.

Scenarios And Patterns Of User Behavior

What we've been calling "steps" is a combination of 3 things:

- Actions - Data and - Conditions

combined together to achieve an actual result.

Example

SL

1. Go to www.sharelane.com.

2. Type word "expectations" into the "Search" text field.

3. Click "Search" button.

Steps 1-3 inclusively are actions. The text "expectations" is the data. If scenario assumes that book is in DB, then the condition is: DB has data about book with word "expectations" in its title.

Scenario is a combination of actions and data applied to software under certain conditions The purpose of a scenario is to bring test execution to the point where an actual result can be retrieved and compared with an expected result.

Expected patterns of user behavior are simply scenarios that we expect will be (OR are already) taking place as users use our software.

Thus, we can expect patterns of user behavior regarding software that - will be released in the future

OR

- has already been released to our users.

How can we figure out expected patterns of user behavior?

a. We can take them from the spec – e.g., the PM can assume the some users will enter an email address with two "@" characters.

b. We can figure them out by exploring. When you browse your Web site, you can imagine what a regular user would do.

c. We can develop scenarios by using a black box methodology. We'll cover this extensively.

d. Scenarios can be a gift from our intuition. You can just wake up in the middle of the night and think: "What if a user would do that?" IMHO, intuition is one of the greatest assets that software tester might have.

e. The PM or programmer might give you some valuable ideas.

f. There are many other sources. For example, you can read an article about how users interact with similar Web sites.

BTW

In some cases, after we have released the software we might discover some actual patterns of user behavior. For example:

- The developer who works on reporting can share with you the results of live data about how users are using the site.

- The customer support folks can give you that information.

- You can read customer feedback on Internet forums, blogs, etc.

We can use knowledge about the actual patterns of user behavior to:

- Change our existing test cases (that we created based on expectations about patterns of user behavior) and/or

- Add new test cases

We would use these changes and/or additions to increase the probability of bug finding.

Let's do a quick recap about conceptual issues regarding black box testing.

The black box approach assumes that the tester usually doesn't know how the back end was written, so ideas for testing come from expected patterns of user behavior. Expected patterns of user behavior are scenarios that we expect will be (OR are already) taking place as users use our software.

Here’s another illustration of black box approach.

Example

Spec #2233: "5.1. If user adds from 20 to 49 books (inclusively) to the shopping cart, we will offer a 2% discount."

SL

1. Create a new account on ShareLane and login (you can use Test Portal>Helpers>Account Creator to create the new account and login automatically.)

2. Add any book to the shopping cart.

3. Click the "Shopping Cart" link.

4. Change the number of books to 20.

5. Click the "Update" button.

6. Check the value under the "discount" column.

Expected result: 2

We just performed black box testing by executing a scenario when a user adds 20 books to the shopping cart. The expected result was 2, but is the code really bug-free? Read on!