Software: AltaVista Mail
The emergence of the Internet as a place where people can conduct business prompted DIGITAL to investigate the development of products specifically for use in this environment. Electronic messaging systems based on Internet technolo gies provide the communication medium for many businesses today. The development of AltaVista Mail illustrates many of the concerns facing engineers who are designing products for this new customer base. The results of our experience can be helpful in many ways and should be of interest to those involved in designing technologies for running Internet applications.
I
Nick Shipman
In late 1993, the Mail Interchange Group ( MI G ) within DIGITAL started the AltaVista Mail develop ment program. At that ti me, the members of MIG bad substantial experience in the development of elec tronic mail ( e - mail) technologies; however, the new products were being targeted �or use on the Internet in an environment that was qu ite different from the
one for their previous prod ucts. In an e ffort to satisf)r the new customer base, the members of MIG reexam i ned their design and development process.
The Alta Vista Mail prod uct emerged from efforts to improve M IG's support for I nternet- based e-mail technologies. Our previous products were electronic mail and directory servers for net:\vork backbone use,
based on the most recent X.400 and X . 500 standards.
Products suitable tor the Internet envi ron ment would clearly be quite different.
This paper begins by presenting our analysis of Internet services and support sofrware and descri bing the transmission of e - mail on the I nternet. The paper then discusses the implications of developing a prod uct for the I nternet environment and explains the impact of those implications on the design and imple mentation decisions that defined rhe AltaVista Mail product. The paper concludes with the engi neeri ng assu mptions and habits that had to be overturned to build the product set .
Internet Services and Software
Duri ng our i nitial anal ysis of the prod uct possibil i ties, w e made several interesting observations about I n ternet services and software, particul arly in compari son to the mission -critical prod ucts in MIG's existing portfolio. (Our observations could more accurately be called assertions-it was and is remarkably difficult to get hard information about I nternet use . )
l . The academic/research/technical community determi ned the nature of the Internet's service offerings. Most of the software defining the Internet's services was generated by and �or this community.
34
2. The real market opportu nity would not be among the academic/research/technical community but would be drawn from ordinary businesses.
3. Much of the service software on tJ1e I n ternet was unsatisf..1ctory for routine business usc, either because it was u nreliable or because it was difficult for end users to deaJ with it. Even tJ1ough !Tee software was abundant, much of it did not work. Support tor tJ1e !Tee software was a risk: some software had excellent peer support; Lmfortunately, not aJ I end users were aware of its existence or were able to access it. 4. We judged the operating system platforms com
monly used for service software to be unsuitable tor a large part of the business community. The various
UNIX platforms need skilled l ocal stan; corrupted or poorly configured Windows version 3 . 1 and Macintosh run - time environments would be diffi cult to diagnose and expensive to support.
Expanding i nto support of the Internet environ - ment would requ ire us to build native equivalents for some of our existing server software. We believed that the Windows NT platform offered a good framework for systems that would work well in any business envi ronment and be easy to support.
lni tiaJJy, we required the foiJowing products: a server that supported the Simple Mail Transfer Protocol (SMTP) and tJ1c Post Office Protocol version 3 ( POP3 ) ;
S M T P c o m m a n d / r e s p o n s e
a gateway to Lotus cc : Mail post offices; and a gateway to Microsoft Mail post offices.
The SMTP /POP3 server is the mail system compo
nent responsible tor accepting messages from mail client programs. I t transmits them toward the recipi ents' SMTP servers and performs local delivery using
the POP3 protocol. This process is described in more detail in the next section.
E-mail on the Internet
This section briefly describes how Internet e-mail is tr;mskrred from the originator to the reci pient.
The originator's mail client program constructs a message according to the rules described i n the Internet standard , R.FC 8 2 2 . 1 The R.FC 8 2 2 standard
defines a message as a seq uence of short l ines of7- bit
ASCII text, each terminated by a carriage-return and
line-ked sequence ( C RJ�F ) . The fi rst l i nes are header
fields; these are extensi ble but typical ly include the originator and recipienr e-mail addresses, the date, and the message subject. The header ends with a blank line, and the remaining l ines constitu te the body of the message. Figure 1 shows an SMTP d ialogue that
includes an .RFC 822 message.
Where appropriate, the mail program can also toiJow the Multipurpose I nternet MaiJ Extensions ( M I M E )
rules in R.FC 1 5 2 1 and R.FC l 522L'; these describe Comments
2 2 0 s e r v e r 1 . a l t a v i s t a . c o . u k A l t a V i s t a Ma i l
Caller opens connection Server's welcome message V 1 . 0 1 1 . 0 B L 2 2 S M TP r e a dy h e l o c l i e n t 1 . a l t a v i s t a . c o . u k 2 5 0 OK m a i L f r o m : < F r e d @ a l t a v i s t a . c o . u k > 2 5 0 O K r c p t t o : < B i l l @ a l t a v i s t a . c o . u k > 2 5 0 O K d a t a 3 5 4 S t a r t m a i l i n p u t ; e n d w i t h < C R L F> . < C R L F> D a t e : M o n , 7 J u l 1 9 9 7 0 8 : 3 0 : 1 3 + 0 1 0 0 F r o m : F r e d < F r e d @ a l t a v i s t a . c o . u k > T o : B i l l < B i l l @ a l t a v i s t a . c o . u k > S u b j e c t : E x a m p l e m e s s a g e H i B i l l , T h i s i s a t e s t m e s s a g e . I t ' s n o t v e r y L o n g . F r e d 2 5 0 O K q u i t 22 1 r e d s v r . a l t a v i s t a . c o . u k c l o s i n g c o n n e c t i o n Figure 1 Example SMTP Dialogue
Digital Technical journal Vol . 9 No. 2 1 997
Cliem gives irs own host name Host name was acceptable
Identi fies return path for nondelivcrv reports Return pad1 was acceptable
A recipient [()r this message Recipient was acceptable
Message tollows
OK to start message header Message's date
Originator field Recipient field
Subject field
Blank line ends message-header fields
Con rem li nes . .
. . . End of content
Message has been accepted
No more tnessagcs; signing off finished
how to construct a message body to transfer typed and structured data and how to pass non-ASCII characters in header fields. Figure 2 shows an example of a mes sage constructed according to the MIME standard .
The originator's client submits the message to a nearby SMTP server using the SMTP protocol:' This very simple protocol uses short, CRLF-terminated lines of 7 -bit ASCI I text to transfer its commands and responses. To submit a message, three commands are used : the MAIL, RCPT, and DATA commands i n tro duce the originator's e-mail address, the recipients' e-rn;lil addresses, and the RFC 8 2 2 message data, respectively.
The SMTP server examines each recipient's e-mail address to decide where the message should be sent. Recipients <lre routed by consul ting the Domain Name System ( DNS ), a distributed directory that asso ciates domain names with sets of typed resource records that denote the p ublished properties of each domain.' 7 For a recipient, [email protected], the
target domain .name is looked u p and the resource records of type MX (tor Mail eXchange) arc retrieved.' These records list the hosts that the domain nominates to receive its mail; each host has a numeric preference
M I M E d a t a D a t e : M a n , 7 J u l 1 9 9 7 0 8 : 3 0 : 1 3 + 0 1 0 0 F r o m : F r e d < F r e d @ a l t a v i s t a . c o . u k > T o : B i l l < B i l l @ a l t a v i s t a . c o . u k > S u b j e c t : B i n a r y a t t a c h m e n t M I M E - v e r s i o n : 1 . 0 C o n t e n t - t y p e : m u l t i p a r t / m i x e d ; b o u n d a r y = " z z z B o u n d a r y z z z " - - z z z B o u n d a r y z z z
value. Eventually, the mail must be del ivered to the most preferred host.
For each target domain , the S MTP server uses the SMTP protocol to transfer the message to the
domain's most preferred, reachable M.,'{ host. (The
most preferred host may be unreachable from the local server: it may be switched off for a while, or it may be behind a firewall , a machine that protects a private net work by limitjng access from the open I n ternet to the machines inside the protected network. ) The chosen host, if it is not the most preferred, forwards the message to a more preferred host and so on, until the message reaches the recipient domain's most preferred host. That host then del ivers the m essage to an area from which the recipient's mail client program can fetch it.
Fetching a message is often a platform-specific oper ation, but a standard protocol such as POP3 can also be used 8 This simple, text- based protocol allows the mail client to l ist the messages waiting to be fetched, to fetch i nd ividual messages, and to delete them from the server once they are satdy stored within the client.
Newer, more feature-rich protocols and i nterfaces, such as the Internet M essage Access Protocol version
Commems
Normal RFC 8 2 2 header fields
This is a M I M E message . . . . . consisting of a list of body parts Blank line ends message -header fields Start . .
. . . of first body part . . C o n t e n t - t y p e : t e x t / p l a i n ; c h a r s e t = " u s - a s c i i "
C o n t e n t - T r a n s f e r - E n c o d i n g : ? b i t
. . . in ASCI I plain text . . . . . . using 7 - bit encod ing H i B i l l , H e r e ' s a b i n a r y f i l e . I t ' s f o u r b y t e s o f a l l 1 ' s . F r e d - - z z z B o u n d a r y z z z C o n t e n t - t y p e : a p p l i c a t i o n / o c t e t - s t r e a m ; n a m e = " f o o . d a t " C o n t e n t - T r a n s f e r - E n c o d i n g : b a s e 6 4 / 1 / ! ! w = = - - z z z B o u n d a r y z z z - - Figure 2 Example M I ME lv!essage
Blank line ends bod)'-part-header fields Bodv-part contents
End . . .
. . . of first body part and start of second . . . . . . a stream of bytes cal led too.dat. . . . using base64 encoding
Blank line ends body-part-header fields Hex FFFFFFFF encoded in base64 End . .
. . . of message
3 6
4 (IMAP4 ) , do offer certain user advantages; however, they do not perform the basic job of delivering mes sages any better than POP 3 .'' Even though support for
IMAP4 vvas added to AltaVista Mail version 2.0, POP3
remains the method of choice for retch ing messages from a remote server: this protocol is so simple that i t is hard to implement incorrectly.
Product Design Decisions
The definition of the AltaVista Mail prod uct set did not start with technical issues. Instead, it started with an assum ption about the p urchase price of a product. Even though the price we chose was not used f(>r the released product, our assumption turned out to be, perhaps, the most usefu l and powerful design tool available during development.
Product Pricing and Organizational Concerns
We were interested in exploring the implications of offering a product at a very low price and sel ling in very large quantities to make the business worthwh ile.
MIG's previous products had been priced at the oppo site end of the price scale: they were expensive, b u t thev were valuable t o the relatively rew customers who needed their functions.
( Interestingly, as we explored the necessary organi zational and technical changes involved in moving to the low end, we realized that they would in no way jeopardize our ability to sell at the high end. The orga
nizational changes improve efficiency no matter what the product price, and the technical changes make for a better, more usable product regardless of the cus tomer pro fi le . )
Our starring point was t o investigate tl1e engineer ing implications of b u i lding a prod uct that would sell for $ 100. We concluded the following:
• To maximize the nu mber of products sol d , we
would have to satiS�' the largest imaginable cus tomer base and not exclude a potential customer for any reason.
• Customers attracted to a low purchase price wil l
also require low running costs. No h idden costs could be Jssociated with ru nning the product.
• Support costs wou l d have to be kept to a minimum.
If each customer needed telephone support several times over the life of the prod uct, the $ 100 price would not cover support expenses. We wou ld hJve to aim at receiving zero support calls.
• Vve would have to minimize the implementation
and maintenance cost and deliver products and updates as early as possible. The Internet market moves quickly, particularly at the low end, and a long development cycle loses sales.
Digiral Tcch niCl! Journal Vol. 9 No. 2 ! 997
Our extstmg approach to development i nvolved obtaining agreement from many groups within
DIGITAL concerning the nature of a problem area and the architecture of any sol utions, and then imple menting prod uct versions ag:�inst the architecture. This process is slow and expensive with considerable management overhead .
For the Al taVista Mail prod uct, we decided insteJd to direct a small team to generate a product-q uality prototype as quickly as possible and to ship that proto type as a product. In the interests of rapid develop ment, we would deliberate ly d iscard much of tbe traditional Ph ase Review Process but would use regu lar, informal monitoring to ensure that the prototype remained acceptable to our target customer.
Al l design and i m p l ementation decisions wou ld be j u dged by their eftect on this target customer,
not by their ad herence to an archi tecture. All future
development wou ld be guided by customer teed back. This method is far l ess expensive, delivers a prod uct far sooner, and is more Ji keJy to reflect cur rent customer needs.
Technological Concerns
Our ideas on product pricing and the design process led to three initial design decisions.
First, nonexpert users must be able to get the fu l l value from the product. Setting u p and configuring the prod uct must involve answering the minimum number of q u est.ions. Each question must relate to a topic on which the user can re:�sonably be expected to have an opinion. The user must not be asked questions about the internal operation of the product, only about topics with an external significance.
The product must offer the minimum number of operational cont rols. (Some high-end customers demand many controls. If necessary, these controls could be added in a later version; but the product must not depend on them , they shou ld not be presented to the average user, and those users who i nsist on seeing them shoul d be charged a premium to cover the addi tional support costs. We would expl icitly accept that there are certain customers we should not aim to sat is�' and certa.in features we should never offer. )
Second, the product must never go wrong. The