Provide Remote Console Access

Servers need to be maintained remotely. In the old days, every server in the machine room had its own console: a keyboard, video monitor or hardcopy console, and, possibly, a mouse. As SAs packed more into their machine rooms, eliminating these consoles saved considerable space.

AKVM switch is a device that lets many machines share a single key- board, video screen, and mouse (KVM). For example, you might be able to fit three servers and three consoles into a single rack. However, with a KVM switch, you need only a single keyboard, monitor, and mouse for the rack. Now more servers can fit there. You can save even more room by having one KVM switch per row of racks or one for the entire data center. However, bigger KVM switches are often prohibitively costly. You can save even more space by using IP-KVMs, KVMs that have no keyboard, monitor, or mouse. You simply connect to the KVM console server over the network from a soft- ware client on another machine. You can even do it from your laptop while connected by VPNed into your network from a coffee shop!

The predecessor to KVM switches were for serial port–based devices. Originally, servers had no video card but instead had a serial port to which one attached an terminal.6_{These terminals took up a lot of space in the computer}

room, which often had a long table with a dozen or more terminals, one for each server. It was considered quite a technological advancement when someone thought to buy a small server with a dozen or so serial ports and to connect each port to the console of a server. Now one could log in to the console server and then connect to a particular serial port. No more walking to the computer room to do something on the console.

Serial console concentrators now come in two forms: home brew or appliance. With the home-brew solution, you take a machine with a lot of serial ports and add software—free software, such as ConServer,7 _{or com-}

mercial equivalents—and build it yourself. Appliance solutions are prebuilt

6. Younger readers may think of aVT-100 terminalonly as a software package that interprets ASCII codes to display text, or a feature of aTELNETorSSHpackage. Those software packages are emulating actual devices that used to cost hundreds of dollars each and be part of every big server. In fact, before PCs, a server might have had dozens of these terminals, which comprised the only ways to access the machine.

4.1 The Basics 81

vendor systems that tend to be faster to set up and have all their software in firmware or solid-state flash storage so that there is no hard drive to break.

Serial consoles and KVM switches have the benefit of permitting you to operate a system’s console when the network is down or when the system is in a bad state. For example, certain things can be done only while a machine is booting, such as pressing a key sequence to activate a basic BIOS configura- tion menu. (Obviously, IP-KVMs require the network to be reliable between you and the IP-KVM console, but the remaining network can be down.)

Some vendors have hardware cards to allow remote control of the machine. This feature is often the differentiator between their server-class machines and others. Third-party products can add this functionality too.

Remote console systems also let you simulate the funny key sequences that have special significance when typed at the console: for example,CTRL- ALT-DELon PC hardware andL1-Aon Sun hardware.

Since a serial console is receiving a single stream of ASCII data, it is easy to record and store. Thus, one can view everything that has happened on a serial console, going back months. This can be useful for finding error messages that were emitted to a console.

Networking devices, such as routers and switches, have only serial con- soles. Therefore, it can be useful to have a serial console in addition to a KVM system.

It can be interesting to watch what is output to a serial port. Even when nobody is logged in to a Cisco router, error messages and warnings are sent out the console serial port. Sometimes, the results will surprise you.

Monitor All Serial Ports

Once, Tom noticed that an unlabeled and supposedly unused port on a device looked like a serial port. The device was from a new company, and Tom was one of its first beta cus- tomers. He connected the mystery serial port to his console and occasionally saw status messages being output. Months went by before the device started having a problem. He noticed that when the problem happened, a strange message appeared on the console. This was the company’s secret debugging system! When he reported the problem to the vendor, he included a cut-and-paste of the message he was receiving on the serial port. The company responded, “Hey! You aren’t supposed to connect to that port!” Later, the company admitted that the message had indeed helped them to debug the problem.

When purchasing server hardware, one of your major considerations should be what kind of remote access to the console is available and

determining which tasks require such access. In an emergency, it isn’t rea- sonable or timely to expect SAs to travel to the physical device to perform their work. In nonemergency situations, an SA should be able to fix at least minor problems from home or on the road and, optimally, be fully productive remotely when telecommuting.

Remote access has obvious limits, however, because certain tasks, such as toggling a power switch, inserting media, or replacing faulty hardware, require a person at the machine. An on-site operator or friendly volunteer can be the eyes and hands for the remote engineer. Some systems permit one to remotely switch on/off individual power ports so that hard reboots can be done remotely. However, replacing hardware should be left to trained professionals.

Remote access to consoles provides cost savings and improves safety fac- tors for SAs. Machine rooms are optimized for machines, not humans. These rooms are cold, cramped, and more expensive per square foot than office space. It is wasteful to fill expensive rack space with monitors and keyboards rather than additional hosts. It can be inconvenient, if not dangerous, to have a machine room full of chairs.

SAs should never be expected to spend their typical day working inside the machine room. Filling a machine room with SAs is bad for both. Rarely does working directly in the machine room meet ergonomic require- ments for keyboard and mouse positioning or environmental requirements, such as noise level. Working in a cold machine room is not healthy for people. SAs need to work in an environment that maximizes their produc- tivity, which can best be achieved in their offices. Unlike a machine room, an office can be easily stocked with important SA tools, such as ref- erence materials, ergonomic keyboards, telephones, refrigerators, and stereo equipment.

Having a lot of people in the machine room is not healthy for equipment, either. Having people in a machine room increases the load put on the heating, ventilation, and air conditioning (HVAC) systems. Each person generates about 600 BTU of heat. The additional power required to cool 600 BTU can be expensive.

Security implications must be considered when you have a remote con- sole. Often, host security strategies depend on the consoles being behind a locked door. Remote access breaks this strategy. Therefore, console systems should have properly considered authentication and privacy systems. For ex- ample, you might permit access to the console system only via an encrypted

4.1 The Basics 83

channel, such as SSH, and insist on authentication by a one-time password system, such as a handheld authenticator.

When purchasing a server, you should expect remote console access. If the vendor is not responsive to this need, you should look elsewhere for equipment. Remote console access is discussed further in Section 6.1.10.