Once the code is in production, then the corresponding release branch is LOCKED

BTW

When a bug is found in production, the following situation can occur:

The developer

- checks in his or her fixed code into the branch with the patch release and - forgets to checkin the fix into the trunk.

The consequence of that forgetfulness is this: When the next branch is cut from the trunk, that new branch will have the same bug. That’s why we can have a situation when a bug that has been fixed in a previous production version (e.g., 2.1) reappears in production after a major release (e.g., 3.0).

So the golden rule is to create a test case for each bug found in production. Add this test case to a special test suite called “Test Cases for Production Bugs.” I recommend keeping that suite dynamic:

you add new test cases as production bugs are found (and fixed) and retire test cases from it once you execute them for the coming release and the next release after the coming release.

Bug Postmortems BTW,

When we encounter a bug in production, it's a good idea to have a postmortem. This term is borrowed from the medical field where it refers to a "medical procedure that consists of a thorough examination of a corpse to determine the cause and manner of death and to evaluate any disease or injury that may be present" (Source: Wikipedia).

By way of analogy, during a bug postmortem at a software company, we:

- do a thorough examination of why that bug was missed;

- try to the identify weak points in our Cycle.

Depending on the severity of the situation, a postmortem can be held as a separate meeting or just as an email thread.

Postmortems should not be witch hunts. On the contrary, they should be constructive, positive measures targeted at improvements.

Sometimes Internet companies make a beta release prior to a major release. The idea behind a beta release it is this: Before we make an official major release (in other words, a major release available to ALL possible users), we make the code of that major release available to a limited group of people (beta testers) who represent our target users.

Beta Releases BTW

A target user is basically a person who we expect to use our Web site. Target users can be identified by different sets of criteria: age, gender, occupation, interests, country, etc.

Beta testers are not test professionals. They are just regular folks that can be useful to us; e.g., we at sharelane.com can invite our most active users to be our beta testers if we decide to do a beta release.

We can just send them an email with a secret URL; e.g., http://beta.sharelane.com. You can tempt users to become beta testers by offering them free items like t-shirts with your company logo.

We need beta releases, because beta testers will use our application in the same fashion as target users, and during beta testing:

1. Beta testers will report bugs to us.

2. We’ll monitor our system and see how it works under real life usage. For example, if the DB crashes during beta testing, we can assume that it will also crash after a major release when many more users are going to use that code.

As beta testing goes on, we fix bugs and deal with other discovered problems (e.g., we might decide to add more servers to improve Web site performance). An example of a beta release is the email service Gmail: until Feb. 2007 new accounts could've been created by invitation only.

BTW

Please note that in some cases, a company will push out a major release with a label of “Beta.” So, if you see “Beta” on some Web site that’s available to EVERYONE, you can translate the word “Beta”

as “This software is freshly baked and probably buggy. So don’t blame us if something wrong happens. Just send us an email with a description of the problem.”

As a rule, companies use beta releases in two cases:

1. The very first release (1.0) of the software.

2. The release of a large, important project; e.g., Gmail by Google.

The logical question is: “If we have beta testing, then we must have done alpha testing, right?” Yes, alpha testing is the testing done BEFORE releasing the software to beta or regular users; e.g., the testing done during the stage “Testing and bug fixes” is alpha testing. Please note that alpha testing is performed by anyone inside the company who tries the new code before it’s released. For instance, the PM can ask the developer to play with the fresh code on the developer’s playground to see how the ideas from the spec are implemented in the software.

Let's proceed.

Testers in Internet companies are in a privileged position compared to testers from other industries. If we at sharelane.com accidentally release a bug on production, we can do a patch release and remove the production problem within minutes. In many cases, that patch release will have a very low cost, and users will have no idea that a bug ever existed. But what if a P1 bug is found in the braking mechanism of an automobile?

- It will cost the auto company millions of dollars to make a recall.

- It will require active user participation to drive to the dealership to fix the problem.

Some last thoughts:

- A release that doesn’t have a critical urgency must be pushed to production while the majority of users are nonactive; i.e., during the night. You can define a “night” for your releases using some interval of time (for example, from 00:00 to 6:00) in the time zone where most of your target users live. This can be very difficult for Web sites with a big international exposure, like

www.google.com. As a rule, U.S. companies make releases between 11:00 p.m. Pacific Standard Time (2:00 a.m. Eastern Standard Time) and 3:00 a.m. PST (6:00 a.m. EST), so they have a four-hour window when the majority of people who live in the continental U.S. are asleep.

Nice Message;Releasing To 1 Machine;Release Moratorium

- Right before and during the time the release to production is under way, put a polite message like this on the production homepage: “Sorry for any inconvenience. This site is under maintenance. We’ll be up at 3:00 PST.” It’s not a big deal from a technical point of view, but your users will really appreciate your consideration.

- In many cases, a coming release is not pushed to all the servers in the production pool, but rather to just one or a few of them. The logic behind it is that we don't want to expose ALL of our users to the new code until we verify that this code works in real world conditions. So, random users hit our new code on one or several production machines and we monitor the quality in production by looking at the DB and log files. This approach is especially good for architectural releases when the front end is absolutely the same, but the back end is different.

- Depending on the specifics of the business, Internet companies usually can predict the times when users are going to be more active than usual. For companies that sell consumer goods, like Amazon, the period between December 1st and December 24th (the Christmas season) is the hottest time of year when a great deal of sales are made. If we know about that period of time beforehand, we must introduce a moratorium for any release to production, except EFB and EFR releases. The reason is simple:

- Our users need uninterruptible service.

- We don’t want to jeopardize our major revenues.

Maintenance

Maintenance is about customer support, minor releases and other measures to keep our production environment and business in general up-and-running. Maintenance is everlasting activity. That's why we don't put it as a separate block of the Cycle flowchart.

Now, take a deep breath. We are about to look at the Big Picture.

The Big Picture Of The Cycle

Let’s use the The Simpsons family for our example. For our episode we need:

- Mother - Marge Simpson (PM);

- Son - Bart Simpson (programmer, tester, and release engineer);

- Father - Homer Simpson (user);

Idea: Marge says to Bart, “Your father will be happy if you’ll build him a house just like this one in the picture as a gift for Father’s day.”

Product Design: Then Marge shows Bart an old photograph of Homer when he was five years old standing in front of his modest yet warm and loving house in Springfield.

Coding: After his usual procrastination, Bart looks at the photo and starts to assemble Lego pieces to build a similar looking house.

Testing and bug fixes: When the house is ready, Bart starts to kick it, throw it against the walls of the Flanders house, insert/retrieve food into/from inside the house, and do other things that Homer would probably do. Each problem is fixed as soon as it’s discovered.

Release: When everything looks fine with the house, Bart gives it to Homer.

Maintenance: Homer immediately tries to open a bottle of beer, and not being able to do this essential task, he asks Bart to fix the problem (call to the customer support team), which Bart does by gluing a bottle opener to the roof of the house (patch release for EFR).

Now, back to sharelane.com. First, let’s recall our knowledge about players, their roles, and the stages in which they participate (note that the activities for Maintenance can be performed at any stage).

Player Role Stage

Marketing dude Generate ideas and create MRD Idea

Product Manager

(PM) Create spec Product Design

Developer

Translate instructions from spec

into software code Coding

Fix bugs Coding

Testing and bug fixes

Tester Prepare test cases Coding

Execute test cases Testing and bug fixes

Release Engineer Push release Release

1. Let’s start at the historic meeting at the bar when the idea of v. 1.0 was conceived:

v. 1.0Idea

“Wow” at the bar

2. After we get an idea we have to develop a way to implement that idea; hence, we need a product design for v. 1.0. The product design is made by the PM.

At the same time,

- the marketing dude generates ideas for v. 2.0.

Product Design v. 1.0 v. 1.0Idea

“Wow” at the bar

January Idea

v. 2.0

3. After the product design for v. 1.0 is finished, we are at the next stage of the Cycle (“Coding”) where the developer writes code for v. 1.0.

At the same time,

- the tester writes test cases for v. 1.0 - the PM creates specs for v. 2.0 features - the marketing dude generates ideas for v. 3.0

Coding

4. After the developer is finished with his or her code for v. 1.0, it’s time for the testers to execute the test cases which have been written for v. 1.0 during the "Coding" stage. Now we are at the “Testing and bug fixes” stage of v. 1.0.

At the same time,

- the programmer writes code for v. 2.0 - the PM writes spec for v. 3.0

- the marketing dude generates ideas for v. 4.0

When the Testing and bug fixes stage is over, we have a major release of v. 1.0. Once v. 1.0 is pushed to production, the testers jump on the "Coding" stage of the Cycle for v. 2.0 and start writing test cases for v. 2.0 (see next page)

We have just looked at the Software Development Life Cycle for a major release version 1.0 of sharelane.com. After that, all is by analogy.

Now, let’s look at the big picture (see next page):

Testing and bug

The Big Picture is just a model; in real life (especially in the start-up environment) everything is not so smooth and structured. For example, during the idea stage of v. 2.0, the marketing dude can generate ideas for both short-term (v. 2.0) and long-term (v. 4.0) projects.

The main points to understand are:

- the concept of the Cycle

- who does what during each stage of the Cycle

- the concept that several Cycles can coexist. For example, in March we have activities for four Cycles where the coming major release is 1.0.

Lecture Recap

1. The software development life cycle is the process of taking a desired software product from the idea