Since 1994: The Original Magazine of the Linux Community

(1)

Since 1994: The Original Magazine of the Linux Community

AUGUST 2008 | ISSUE 172

Billix | Rails | Gumstix | Zenoss | Wiimote | BUG | Quantum GIS

™

0 09281 03102 4

0 8

$ 5 . 9 9 U S $ 5 . 9 9 C A N

w w w. l i n u x j o u r n a l . c o m

Learn to Fake a

UFO Landing Video

Wiimote Linux

Interface HOW-TO

+

AND OTHER COOL

PROJECTS TOO

REVIEWED:

Neuros OSD and Cradlepoint PHS300

WE’VE GOT

BUGs

Billix Kiss Install CDs Goodbye How To:

16 Terabytes in One Case E-Ink +

Gumstix

Perfect

Match?

(2)

(3)

(4)

CONTENTS AUGUST 2008 Issue 172

FEATURES

48 THE BUG: A LINUX-BASED HARDWARE MASHUP

With the BUG, you get a GPS, camera, motion detector and accelerometer all in one hand-sized unit, and it’s completely programmable.

Mike Diehl

52 BILLIX: A SYSADMIN’S SWISS ARMY KNIFE

Build a toolbox in your pocket by installing Billix on that spare USB key.

Bill Childers

56 FUN WITH E-INK, X AND GUMSTIX

Find out how to make standard X11 apps run on an E-Ink display using a Gumstix embedded device.

Jaya Kumar

62 ONE BOX. SIXTEEN TRILLION BYTES.

Build your own 16 Terabyte file server with hardware RAID.

Eric Pearce

ON THE COVER

•Neuros OSD, p. 44

•Cradlepoint PHS300, p. 42

•We've got BUGs, p. 48

•E-Ink + Gumstix—Perfect Match?, p. 56

•How To: 16 Terabytes in One Case, p. 62

•Billix—Kiss Install CDs Goodbye, p. 52

•Learn to Fake a UFO Landing Video, p. 80

•Wiimote Linux Interface How-To, p. 32

(5)

ABERDEEN

The Straight Talk People

S I N C E 1 9 9 1

SM

HOW MUCH STORAGE DO YOU NEED?

Intel, Intel Logo, Intel Inside, Intel Inside Logo, Pentium, Xeon, and Xeon Inside are trademarks or registered trademarks of Intel Corporation or its

subsidiaries in the United States and other countries. For terms and conditions, please see www.aberdeeninc.com/abpoly/abterms.htm. lj026

888-297-7409

www.aberdeeninc.com/lj026

• Aberdeen Stirling Storage Servers deliver vast and flexible expansion capabilities in a single storage server without additional controllers

• Expand up to 328TB with Aberdeen DAS and JBOD units without any performance degradation

• SAS and SATA support concurrently in same server (separate arrays)

• Four Gigabit Ethernet Ports allow network teaming

• Dual Quad-Core Intel^®Xeon^®Processors with up to 1600 MHz Front Side Bus

• iSCSI support with available NAS versions

• Multiple available expansion slots

JBOD STORAGE

Expandable Storage Solution

• Dual Quad-Core Intel^®Xeon^®processors with up to 1600MHz FSB

• Up to 64GB 800 ECC FBDIMM Memory

• Supports both SAS & SATA Storage Drives

• Internal OS Hard Drives Included

• RAID 0, 1, 5, 6, 10 Capable

• Redundant Power Supply

• SAS & iSCSI Expansion Ports

• Windows & Linux NAS Available

• 5-Year Warranty

3U 12TB Starting at ^$

7,995

9,995

12,995

17,995

20,995

STORAGE SERVER

+

Add up to 128TB

• Daisy-Chain JBOD Boxes

• Storage Server RAID Array

• SATA & SAS Drive Support

• 5-Year Warranty 16TB JBOD ^$

7,995

XDAS STORAGE

Add up to 160TB

• Daisy-Chain DAS Unit + JBOD Expansion Boxes

• 2U, 3U, 4U Units Available

• RAID 0, 1, 5, 6, 10 Capable

• SATA & SAS Drive Support

• 5-Year Warranty 16TB DAS^$

12,995

16TB JBOD ^$

9,495

+ = ^EXPAND ^{UP TO}

328TB

The Aberdeen line of expandable storage servers provide a full spectrum of scalable, maximum capacity storage solutions for data backup, media sharing storage, content creation, streaming media, nearline storage, and post-production needs. These servers feature high performance Quad-Core Intel Xeon processors and enterprise-level SATA drives providing over 800MB/s internal transfer rates, while being robust enough to provide up to 328TB of scalable storage without added controllers.

lj026:lj018.qxd 5/14/2008 4:00 PM Page 1

(6)

COLUMNS

8

SHAWN POWER’S CURRENT_ISSUE.TAR.GZ

Linux: the Root of All Coolness

22

REUVEN M. LERNER’S AT THE FORGE

Profiling Rails Applications

26

MARCEL GAGNÉ’S COOKING WITH LINUX

Cool as Ice!

30

DAVE TAYLOR’S WORK THE SHELL

Movie Trivia and Fun with Random Numbers

32

KYLE RANKIN’S HACK AND /

Wiimote Control

96

DOC SEARLS’

EOF

Mixing Up a Generative Mobile Feast

REVIEWS

42

HOT AND BOTHERED AT STARBUCKS

Dan Sawyer

44

THE NEUROS OSD CONNECTS YOUR TV TO THE INTERNET

Marco Fioretti

IN EVERY ISSUE

12

LETTERS

16

UPFRONT

36

NEW PRODUCTS

38

NEW PROJECTS

81

ADVERTISERS INDEX

INDEPTH

68

LINUX FOR THE LONG HAUL

Checking in with the Greater Houlton Christian Academy’s switch to Linux.

Michael Surran

72

ZENOSS AND THE ART OF ENTERPRISE MONITORING

Stay on top of your network with an enterprise-class monitoring tool.

Jeramiah Bowling

80

HOW TO FAKE A UFO LANDING

Use Voodoo to solve video match-moving problems.

Dan Sawyer

86

QUANTUM GIS: THE

OPEN-SOURCE GEOGRAPHIC INFORMATION SYSTEM

Hooked on Google Earth? Check out Quantum GIS to satisfy your geographic cravings.

James Gray

92

BUILD A MYTHTV BOX WITHOUT BREAKING THE BANK

A quick-and-easy guide to the world of MythTV.

P. Surdas Mohit

CONTENTS AUGUST 2008 Issue 172

USPS LINUX JOURNAL (ISSN 1075-3583) (USPS 12854) is published monthly by Belltown Media, Inc., 2211 Norfolk, Ste 514, Houston, TX 77098 USA. Periodicals postage paid at Houston, Texas and at additional mailing offices. Cover price is $5.99 US. Sub scrip tion rate is $29.50/year in the United States, $39.50 in Canada and Mexico, $69.50 elsewhere. POSTMASTER: Please send address changes to Linux Journal, PO Box 980985, Houston, TX 77098. Subscriptions start with the next issue. Canada Post: Publications Mail Agreement

#41549519. Canada Returns to be sent to Bleuchip International, P.O. Box 25542, London, ON N6C 6B2

THE UNDERDOG ISSUE

Everybody loves an underdog.

Some people even consider Linux to be an underdog (we prefer “Undiscovered Champion”). Next month, we focus on the little guy. Sure you know all about Ubuntu, but what about Gentoo? Granted, Apache is the king of the Web server world, but what about the alternatives? (No, not IIS.) Heck, even Inkscape has some less-popular alternatives like Xara Xtreme. So, whether you’re looking for a command- line e-mail client, or you want an alternative to BIND, next month will be an issue you don’t want to miss.

Next Month

56E-INK’S E-PAPER DISPLAY

(7)

(8)

Associate Editor Senior Editor Art Director Products Editor Editor Emeritus Technical Editor Senior Columnist Chef Français Security Editor

Proofreader

Publisher

General Manager

Director of Sales Regional Sales Manager Regional Sales Manager

Circulation Director

System Administrator Webmistress

Accountant

[email protected] Shawn Powers [email protected] Doc Searls

[email protected] Garrick Antikajian [email protected] James Gray

[email protected] Don Marti

[email protected] Michael Baxter [email protected] Reuven Lerner [email protected] Marcel Gagné [email protected] Mick Bauer [email protected]

Geri Gale

Carlie Fairchild [email protected]

Rebecca Cassity [email protected]

Laura Whiteman [email protected] Joseph Krack [email protected] Bruce Stevens [email protected]

Mark Irgang [email protected]

Mitch Frazier [email protected] Katherine Druckman [email protected]

Candy Beauchamp [email protected] Contributing Editors

David A. Bandel • Ibrahim Haddad • Robert Love • Zack Brown • Dave Phillips • Marco Fioretti Ludovic Marcotte • Paul Barry • Paul McKenney • Dave Taylor • Dirk Elmendorf

Linux Journal is published by, and is a registered trade name of, Belltown Media, Inc.

PO Box 980985, Houston, TX 77098 USA

Reader Advisory Panel

Brad Abram Baillio • Nick Baronian • Hari Boukis • Caleb S. Cullen • Steve Case Kalyana Krishna Chadalavada • Keir Davis • Adam M. Dutko • Michael Eager • Nick Faltys • Ken Firestone

Dennis Franklin Frey • Victor Gregorio • Kristian Erik • Hermansen • Philip Jacob • Jay Kruizenga David A. Lane • Steve Marquez • Dave McAllister • Craig Oda • Rob Orsini • Jeffrey D. Parent Wayne D. Powel • Shawn Powers • Mike Roberts • Draciron Smith • Chris D. Stark • Patrick Swartz

Editorial Advisory Board Daniel Frye, Director, IBM Linux Technology Center

Jon “maddog” Hall, President, Linux International Lawrence Lessig, Professor of Law, Stanford University

Ransom Love, Director of Strategic Relationships, Family and Church History Department, Church of Jesus Christ of Latter-day Saints

Sam Ockman Bruce Perens Bdale Garbee, Linux CTO, HP Danese Cooper, Open Source Diva, Intel Corporation

Advertising E-MAIL: [email protected] URL: www.linuxjournal.com/advertising

PHONE: +1 713-344-1956 ext. 2 Subscriptions E-MAIL: [email protected] URL: www.linuxjournal.com/subscribe

PHONE: +1 713-589-3503 FAX: +1 713-589-2677 TOLL-FREE: 1-888-66-LINUX MAIL: PO Box 980985, Houston, TX 77098 USA Please allow 4–6 weeks for processing address changes and orders

PRINTED IN USA

LINUX is a registered trademark of Linus Torvalds.

(9)

(10)

I

f you are a regular visitor to the LinuxJournal.com Web site, you might recognize me as the goofy video Gadget Guy, or possibly as the Web author with a penchant for controversy.

While the latter is largely coincidental, the former is just the way I am (my wife can grudgingly attest to that). This month marks the first issue that I’m the Associate Editor of the print magazine as well. Whether adding me to the staff will be beneficial, or more like the spreading of Windows spyware, is yet to be determined.

The Cool Projects issue is significant to me for another reason as well. A year ago, in the August 2007 issue, my “How to Build Your Own Arcade” article marked the first time I was pub- lished in Linux Journal (www.linuxjournal.com/

article/9732). It also appeals to my inner child that thinks life should revolve around stuff that’s

“cool”. The 2008 Cool Projects issue (the one you’re reading now) offers plenty of opportunity to have fun with our favorite operating system.

Whether you’re looking for a cool way to do your job, or whether you’re trying to avoid doing your job altogether, we’ve got you covered.

If you subscribe to Linux Journal at work, and you’re trying to justify the Cool Projects issue to your boss, fear not. We make it much easier than trying to explain the Sports Illustrated Swimsuit Edition to your significant other. Eric Pearce shows us how to make a 16-Terabyte file server out of bubble gum and popsicle sticks. Well, okay, maybe not with those ingredients, but he walks us through the process of creating a really big server.

Bill Childers shows us one of the coolest uses of a USB Flash drive I’ve ever seen. With an outdated 256MB drive, Bill shows us how to make a bootable device that will install many different Linux distributions and launch a handful of utilities too!

If you can’t find something this month that directly ties to your job, feel free to play the

“professional development” card, and have some fun while you’re furthering your techno- logical horizons. Michael Surran, for example, tells us all about his use of Linux in education.

As someone who has implemented Linux in schools before, I always find it cool when schools take the plunge.

Perhaps you are a programmer, and code all day, and code all night. Reuven M. Lerner shows us how to make sure our Rails are optimized, and Dave Taylor helps us extract really important data—from the Internet Movie Database. Along with some open-source mapping software, this issue is really full of easily justifiable diversions.

For me, however, the exciting thing about the Cool Projects issue is building cool stuff. Have you seen the Bug Labs’ BUGs? All you have to do to build a cool project with them is snap together the pieces you want. The BUGs are amazingly versatile and are being developed every day. We show you how to make the little buggers bend to your will. Or, maybe you want to learn to use E-Ink technology and handcraft your own tiny PC. Jaya Kumar shows us how.

What if you don’t subscribe to Linux Journal at work, and you’re just looking for some cool things to do with Linux in your spare time? Kyle Rankin and Marcel Gagné felt the same way.

Kyle shows us how to interface a Wii remote (Wiimote) to our Linux machines and use the controller as a joystick and mouse. Marcel, taking the word “cool” literally, shows us a handful of penguin and ice games bound to keep you busy for hours.

Finally, if reality isn’t cool enough, we’ve got Zenoss, and we’ve got “How to Fake a UFO Landing”. Granted, the two have nothing to do with each other, but if you name your network- monitoring system Zenoss (Zeen-ohss), you’re just asking for some taunting. So, sit back, prop up your feet, and enjoy this issue of Linux Journal.

If you get tired of reading, maybe catch a few flicks on TV with your Neuros OSD. We’ll tell you about that little beauty as well.I

Shawn Powers is the Associate Editor for Linux Journal. He’s also the Gadget Guy for LinuxJournal.com, and he has an interesting collection of vintage Garfield coffee mugs. Don’t let his silly hairdo fool you, he’s a pretty ordinary guy and can be reached via e-mail at [email protected]. Or, swing by the #linuxjournal IRC channel on Freenode.net.

Linux: the Root of All Coolness

SHAWN POWERS

Current_Issue.tar.gz

(11)

(12)

For powerful choose a web host!

World’s #1 Web Host

With a wide variety of products and hosting packages, superior data center technology, unparalleled reliability, special offers, great prices and a 90-day-money-back guarantee, it’s no wonder 1&1 is the world’s biggest and fastest growing web host!

Speedy Connection

1&1 servers benefit from 100 MBit connection

Great Prices

All dedicated servers are on sale!*

Cutting-edge Technology

Hardware to meet your strong demands

64 Opteron

(13)

US335_V5_406_5x276.indd 1 09.06.2008 12:44:14 Uhr

Call

1.877.go1and1

Visit us now

1and1.com

servers,

powerful

Best Value:

Compare for yourself.

B t V l Best Value:

Compare for yourself.

1&1 ^{The Planet}

BUSINESS I Intel Pentium 4 Series 2.4+

CPU AMD Athlon 64 3500+ Intel Pentium 4 2.4 GHz+

RAM 1 GB 512 MB included, add $15/month for 1 GB

Useable Disk Space 160 GB 80 GB

RAID RAID 1 Included (2 X 160 GB HD) Add $10/month for second 80 GB HD for RAID 1

BackUp Full 160 GB Backup Included Add $80/month for 80 GB Backup

Monthly Traffi c (GB/month) 2000 GB/month 750 GB included, add $175/month for 1750 GB

Total Monthly Fee as Confi gured

$ 389

ENTERPRISE I Conroe 3060 Series - SATA

CPU Dual Core AMD Opteron 1218-2.6 GHz Dual Core Intel Xeon 3060 Conroe Processor-2.4 GHz

RAM 4 GB 2 GB included, add $50/month for 4 GB

Useable Disk Space 400 GB 250 GB

RAID RAID 1 Included (2 X 400 GB HD) Add $40/month for RAID 1 + $20/month for 2nd 250 GB HD

BackUp Full 400 GB Backup Included Add $200/month for 200 GB Backup

Monthly Traffi c (GB/month) 4000 GB/month 2500 GB included, add $175/month for 3500 GB

Total Monthly Fee as Confi gured

1 ^$ 694

$

99

$

299

$

99

$

99 9 99 9 9 9 9 9

$

9 9 99

$

299

$

299 9 9 9 9 9 99 9

$

29 9

$

299

Visit 1and1.com for details. Prices based on comparable packages, effective 5/21/2008. *Offer valid for dedicated server packages only, with a 24 month minimum contract term required.

Prices shown reflect Linux (Root) and Managed server configurations.

**Price valid for first year of .us domain registration. After the first year, regular prices will apply. Product and program specifications, availability, and pricing subject to change without notice.

All other trademarks are the property of their respective owners.

ENTERPRISE I

$ 49 ⁵⁰

^*

1 1 1 1 1 1 1 1 1 1 1 1 1 1

$ 149 1 ⁵⁰

^*

V

Visit 1and1.com for details. Prices based on comparable packages, effec P

Prices shown reflect Linux (Root) and Managed server configurations.

*

**Price valid for first year of .us domain registration. After the first year, A

All other trademarks are the property of their respective owners.

Call

1..877

Call

1. Domain only .US

$2.99

fi rst year!

^**

For a limited time, America’s internet address is on sale.

Create your own .us domain by visiting www.1and1.com.

50% OFF 50% off ALL Dedicated Servers for the first 3 months!*

Visit our website for details.

US335_V5_406_5x276.indd 2 09.06.2008 12:44:34 Uhr

(14)

Try Puppy Linux

Wow, what a great article by Louis Iacona [LJ, April 2008, “Puppy Linux”].

I was pleasantly surprised to find it so in depth for a magazine article, which is usually no more than two pages. It definitely encouraged me to try Puppy Linux, which I will do. I hope to see more articles by this gentleman and hope he was well paid. Thank you for such a great magazine.

--

George Mulak On ISOs

I have noticed mention of ISOs now and then in LJ articles.

While working on our own applications, we’ve often gone searching on the Internet for tools and applications that might already be found in the form of a ready-to-run ISO.

Because ISOs are relatively new to the public, we concluded that currently, it is difficult to find such works or list them if you are an author.

Therefore, we have created a new site, www.isotogo.com, to help the public and authors in working with ISOs.

There is no cost to use it or to list your works, and because it is new, we are

working hard on moving up the search engine ladder.

--

Mike Paradis

Comments on the OLPC XO Dave Phillips is effusive in his praise of the OLPC XO [LJ, June 2008], and most of it is well deserved, indeed. But it must be conceded that the keyboard is a piece of absolute trash. After just a few hours of use, mine developed keys that stick or fail to actuate or actuate with Alt applied unpredictably. This got worse until it was completely unusable.

A Google search turned up many such complaints and detailed instructions for excising and replacing the keyboard (serious tinkerers only!). The USB keyboard is really a necessity, but that is not convenient in all circumstances.

The mouse does have two active but- tons, as you can verify by copying the binary of gpm from a compatible system (I use Fedora Core 6 on a Dell Latitude). Run it as:

gpm -m /dev/input/mouse0 -t ps2 -r 5 -a 3

and play with the -r and -a settings to get it the way you like.

The display is a little bit strange. Plotting a bunch of random pixels in white on a black screen makes red, green and blue dots. A white-on-black line may show bands of color, depending on the point density and inclination. So your favorite graphics apps may need some tweaking.

If you want to run something that uses SVGALIB, you need my framebuffer version of that. It’s not complete yet, but it does basic pixel, line and block functions.

I’ll put it on my SourceForge site soon, but meanwhile, if interested, send me a note at [email protected].

--

Bill McConnaughey Yet Another One-Liner

The June 2008 issue of LJ published a letter from David Newall, which responded to a letter from Joao

Macedo published in the February 2008 issue, which was, in turn, a response to Dave Taylor’s column in the December 2007 issue. Both Joao and David wrote in with one-liners using the echo and bc commands to do floating-point calculations in place of using Dave Taylor’s solve.sh script.

Joao’s example embedded an actual newline character, whereas David Newall’s used the escape code version of the same. There is yet another way to do this, and it is, in fact, my prefer- ence, as it is much more intuitive to folks accustomed to writing shell scripts. In bourne-shell scripts, a semi- colon can be used to place commands together on a single line. It can be used for the same purpose with bc.

Here is a third rewrite of the example to demonstrate:

echo 'scale=4;11/7' | bc --

James Williams Zavada Correction

First off, great magazine! While reading page 13, in the UpFront section [LJ, June 2008, “Eee PC Gets an Upgrade”], Doc talks about the upgrade that the Eee PC is getting, along the lines of the larger screen, larger SSD hard drive and more memory. He lists the new Eee PC 900 as having 1GB of RAM, and say that this is

“up from 512KB”. I dunno about you, but my Eee PC (from which I am sending this letter) has 512 megabytes, not kilobytes! If yours has 512KB of RAM, you should send it back! Great magazine, small typo, I forgive you!

--

Eric Jennings Thanks

I just wanted to send a quick note to thank all of the contributors to LJ. You have inspired me over the past couple of years to migrate over to Linux as my OS of choice and motivated me to learn new projects. I am in the middle of setting up an LTSP project for our home-schooling community and using info gained from various LJ articles and book recommendations.

letters

(15)

(16)

Keep up the good work. Hopefully I will send some converts your way soon!

--

Dean Anderson Sony

I’m happy to see the coverage of Sony’s use of Linux in the June 2008 issue. There is actually an even bigger list of Sony products running Linux at www.sony.net/Products/Linux.

Myself, I was surprised to find my new digital camera on that list. Now if only we could turn this into some sort of quality label instead of a hid- den feature. (Disclaimer: I am a UNIX sysadmin working for Sony.)

--

Nico De Ranter

More Must-Have Firefox Extensions

I’m surprised the article [Dan Sawyer’s

“Must-Have Firefox Extensions” in the June 2008 issue of LJ] didn’t even mention AdBlock Plus. It’s the first extension I put on any Firefox instal- lation I come across. After I installed it on my girlfriend’s laptop, she exclaimed, “now I understand why you actually like that one site!”

Her laptop’s running Kubuntu with VMware Workstation for those pesky Windows-only apps, by the way.

Another helpful extension is Cookie Button, which prevents all those cookie confirmation windows from popping up and still allows one to enable them easily for a specific site if required.

I also enjoyed last month’s article regarding using virtualization on Mac OS X, because that’s what I’ve been doing ever since I got my MacBook Pro [Dave Taylor’s “Running Ubuntu as a Virtual OS in Mac OS X” in the May 2008 issue of LJ]. It’s running Parallels with Kubuntu and Windows XP, and it allows me to develop and test software on all three operating systems with ease.

And, I can run Amarok to listen to music, because iTunes simply doesn’t measure up.

--

U. Hertlein

[

LETTERS

^]

At Your Service

MAGAZINE

PRINT SUBSCRIPTIONS:Renewing your subscription, changing your address, paying your invoice, viewing your account details or other subscription inquiries can instantly be done on-line, www.linuxjournal.com/subs. Alternatively, within the U.S. and Canada, you may call us toll-free 1-888-66-LINUX (54689), or internationally +1-713-589-3503. E-mail us at [email protected] or reach us via postal mail, Linux Journal, PO Box 980985, Houston, TX 77098-0985 USA. Please remember to include your complete name and address when contacting us.

DIGITAL SUBSCRIPTIONS:Digital subscriptions of Linux Journal are now available and delivered as PDFs anywhere in the world for one low cost.

Visit www.linuxjournal.com/digital for more information or use the contact information above for any digital magazine customer service inquiries.

LETTERS TO THE EDITOR:We welcome your letters and encourage you to submit them at www.linuxjournal.com/contact or mail them to Linux Journal, 1752 NW Market Street, #200, Seattle, WA 98107 USA. Letters may be edited for space and clarity.

WRITING FOR US:We always are looking for contributed articles, tutorials and real- world stories for the magazine. An author’s guide, a list of topics and due dates can be found on-line, www.linuxjournal.com/author.

ADVERTISING: Linux Journal is a great resource for readers and advertisers alike.

Request a media kit, view our current editorial calendar and advertising due dates, or learn more about other advertising and marketing opportunities by visiting us on-line, www.linuxjournal.com/advertising.

Contact us directly for further information, [email protected] or +1 713-344-1956 ext. 2.

ON-LINE

WEB SITE:Read exclusive on-line-only content on Linux Journal’s Web site, www.linuxjournal.com.

Also, select articles from the print magazine are available on-line. Magazine subscribers, digital or print, receive full access to issue archives; please contact Customer Service for further information, [email protected].

FREE e-NEWSLETTERS:Each week, Linux Journal editors will tell you what's hot in the world of Linux. Receive late-breaking news, technical tips and tricks, and links to in-depth stories featured on www.linuxjournal.com. Subscribe for free today, www.linuxjournal.com/enewsletters.

(17)

(18)

UP FRONT

N E W S + F U N

Having to reboot production systems to incorporate security patches is a pain. How much better would it be simply to graft the patch onto an already running kernel, and let it keep running? This is exactly what Jeff Arnold has been working on.

He calls it KSplice, and at least for the moment, it supports any kernel that can load a module. The kernel itself doesn’t have to support the feature explicitly.

The way it works is that the user sup- plies the source tree for the running kernel and the patch to graft on. Then, KSplice compiles the patch and loads modules that apply the patch internally.

Because of the interest it has generated, it’s looking very likely that KSplice will be accepted into the kernel tree, at which point it might stop supporting kernels that don’t know about it.

One obstacle standing in the way of Jeff’s work is Microsoft’s legal depart- ment. During the course of discussion, it came out that patent application 10/307,902 from 2002 seemed to cover Jeff’s idea. And, although a number of folks, including Gilles Espinasse and Willy Tarreau, said they’d been “hot- patching” operating systems since the 1990s and earlier, Bill Davidsen felt that trying to launch a challenge against a Microsoft patent would be prohibitive- ly expensive. However, according to Jeff, the patent application was rejected by the patent office. So, Microsoft may give up at this point, depending on its current internal weirdness level.

NFS is a lovely filesystem, but it has various problems that make people want to replace it. One of the latest attempts is POHMELFS, or Parallel Optimized Host Message Exchange Layered File System. POHMELFS is written by Evgeniy Polyakov, and it is a user- space layer that can be applied to any back-end filesystem, such as ReiserFS or XFS. It also seems to outperform NFS fairly significantly, at least according to the tests Evgeniy has performed so far.

But, it’s still only ready for playing around with. People shouldn’t use it to store data they want to keep. Evgeniy

says it’s too soon to talk about the filesystem being reliable for use. He’s been able to do regular user activity on it without a problem, but he expects to find bugs, POSIX conformance issues and other issues.

ReiserFS has migrated its develop- ment from the NameSys servers to kernel.org, where work is continuing.

Edward Shishkin and others continue to develop the filesystem in spite of Hans Reiser’s murder conviction.

There have been various other main- tainer updates. Greg Kroah-Hartman is no longer the PCI maintainer; he’s handed the whole kit and kaboodle off to Jesse Barnes. Timur Tabi has listed himself as the official maintainer of the Cirrus Logic CS4270 sound driver, the Freescale QUICC engine library, the QUICC engine UCC UART driver and the Freescale SOC sound drivers. And, Zhang Wei has abdicated maintainer- ship of the Freescale DMA driver and handed that project over to Li Yang.

The politics of competing filesystems is never pretty. LogFS wants to support Flash drives, but its development has been slower than some people would like. So, Artem Bityutskiy and Adrian Hunter recently announced their own alternative, UBIFS, that is apparently quite a bit further along than LogFS. It’s faster, more stable and more featureful, although it still has trouble with devices larger than 64G. LogFS, developed by Jorn Engel, also came out with a new release, perhaps partly in response to the UBIFS announcement. Of course, nothing says there can’t be two co- existing Flash filesystems, but apparently one of the motivations for UBIFS was

the relatively slow pace of LogFS development. Artem in particular seemed bitter about this, especially since it appeared that various folks were suggesting waiting for LogFS instead of developing UBIFS, while UBIFS was already superior to LogFS.

Matthew Wilcox and others are trying to eliminate as many semaphores as they can from the kernel, and replace them with simpler locking structures, such as mutexes, spinlocks and comple- tions. The problem is that semaphores provide additional features that are hard to mimic with those other types of locks.

For example, semaphores can manage access to a cluster of resources, while other locks basically are just on/off switches. The benefit of eliminating semaphores is that it’s possible to gain speed and, in the case of single-processor systems, to optimize the lock entirely out of the compiled binary, resulting in further speed gains. But to do this, key features of semaphores have to be mim- icked near any lock that replaces one.

Matthew, Arjan van de Ven and Ingo Molnar are addressing this by develop- ing kcounter, which will provide ways of managing access to clusters of resources, similar to how semaphores do it.

Unfortunately, kcounter takes a cookie- based approach, similar to other things that have been seen in the kernel before, which have resulted in what David Chinner characterized as a very ugly inter- face. Hopefully, kcounter will avoid that pitfall, although it does seem as though a significant speedup might justify a little cookie ugliness. That question undoubt- edly will spawn some lively debate.

— Z A C K B R O W N

WHAT’S NEW IN KERNEL DEVELOPMENT

diff -u

(19)

w w w. l i n u x j o u r n a l . c o m august 2008 | 17

[

UP FRONT

^]

LJ Index, August 2008

1. Number of new toys the average child gets per year: 70

2. Size of the “baby industry”, in trillions of dollars: 1.7

3. Number of computers donated to the World Computer Exchange: 26,695

4. Schools, orphanages and libraries served by the World Computer Exchange: 2,543

5. Youth connected per year by the World Computer Exchange: 1,079,110

6. Number of languages other than English among contributors to DistroWatch: 42

7. Number of countries covered by Linux-hosted Global Voices Online:192 8. Number of authors for Global Voices: 101

9. Number of languages into which Global Voices is being translated: 11

10. Number of computers in the Windsor Unified School District (California): 2,500

11. Percentage of computers at Windsor Unified to be replaced by Linux thin clients: 50

12. Estimated thousands of dollars in energy saved annually by Windsor Unified School District, thanks to Linux servers and thin clients: 25

13. Estimated thousands of dollars in energy saved annually by Windsor Unified School District by switching to free software: 50

14. Estimated thousands of dollars in equipment costs saved annually by Windsor Unified School District by switching to Linux gear: 280

15. Number of dollars in four spent on energy in schools that is unnecessary: 1

16. Months children at Villa Cardal, Uruguay, had spent with beta XO OLPCs: 6

17. Average number of files created per XO by Villa Cardal kids: 1,200

18. Number of MB, +/–10, produced per machine by Villa Cardal kids: 40

19. Thousands of XOs due for Uruguay in 2008: 90

20. Thousands of XOs ordered by Peru in December 2007: 260

Sources:1, 2: Pamela Paul in Parenting, Inc.

3–5: World Computer Exchange (May 10, 2008) 6: DistroWatch.com | 7–9: Global Voices Online (www.globalvoicesonline.org) 10–14: The Press-Democrat (pressdemocrat.com) 15: Ed Tech Magazine (edtechmag.com) 16–20: Ivan Krstic

One could play for hours with Google Trends (trends.google.com). Not only does it show the spikes and slopes of search volume across time since the beginning of 2004, but it also lists the current top ten regions, cities and languages for each search. You can search for multiple keywords, comma-separated, and see colored lines for each. The results are usually more interesting than revealing. Such as:

I Searches for Ronaldo and Beckham both spiked in 2005 at the last World Cup.

I Searches for John Paul and Ratzinger peaked one after the other in early 2005 when the former died and the latter succeeded him as pope.

I Searches for Linux and Microsoft have both gone down, the former slightly more than the latter.

I Searches for Red Hat, SUSE and Ubuntu show declines for the first two and a steady rise for the third. Add Linux and find that Ubuntu has almost overtaken Linux in search volume. Does the rise in Ubuntu account for the decline in Linux searches? They seem somewhat reciprocal, but who knows?

What’s more surprising are the top ten regions, cities and languages for each.

Some results for Linux:

I Russia is the top region, closely followed by India and the Czech Republic. The

US is not among the top ten.

I Beijing is the top city, followed by Tokyo and San Francisco, which is the only US city. The rest, in order, are Milan, Frankfurt, Augusta (Italy), Paris, Amsterdam, Madrid and London.

I Russian is the top language, followed by German, Japanese, Italian, Chinese, Polish, Finnish, Portuguese, English and Swedish.

Some results for Ubuntu:

I Norway is the top region, followed by Italy and Russia. The US is not on the list.

I Milan is the top city, followed by San Francisco and Augusta (Italy). The rest are Helsinki, Madrid, Paris, Santiago (Chile), Frankfurt, Zurich and Mexico City.

I Italian is the top language, followed by Finnish, Swedish, Russian, French, Spanish, German, Polish, English and Portuguese.

Google’s qualification: “Google Trends aims to provide insights into broad search patterns. Several approximations are used when computing your results. Please keep this in mind when using it.”

Also keep in mind that these were results on May 13, 2008. Try them when you read this to see how they change.

Having used Google Trends for a while now, I can assure you the answer is: a lot.

— D O C S E A R L S

Polynational Tux Curiosity

On the Web, Articles Talk!

Every couple weeks over at LinuxJournal.com, our Gadget Guy Shawn Powers posts a video. They are fun, silly, quirky and sometimes even useful. So, whether he's reviewing a new product or show- ing how to use some Linux software, be sure to swing over to the Web site and check out the latest video: www.linuxjournal.com/video.

We'll see you there, or more precisely, vice versa!

(20)

[

UP FRONT

^]

A few weeks ago, I was flying west past Chicago, watching the ground slide by below, when I spotted the signature figure eight of the Fermi National Accelerator Laboratory, better known as Fermilab. I shot some pictures, which I put up at the Linux Journal Flickr site (www.flickr.com/

groups/linuxjournal/pool, which runs on Linux too).

I figured Fermilab naturally would use Linux, and found that Fermilab has its own distro: Fermi Linux. Its public site provides a nice window into a highly professional and focused usage of Linux.

Within Fermi Linux, specific generations are known as Scientific Linux Fermi, each with version numbers and the code names Charm, Strange, Top, Bottom, Up, Feynmann, Wilson and Lederman.

Some also have LTS in their names.

LTS stands for Long Term Support. It has a FAQ. The first Q is, “What is Fermi Linux LTS?” The A goes:

Fermi Linux LTS (Long Term Support) is, in essence, Red Hat

Enterprise, recompiled.

What we have done is taken the source code from Red Hat Enterprise (in srpm form) and recompiled it. The resulting binaries (now in rpm form) are then ours to do with as we desire, as long as we follow the license from that original source code, which we are doing.

We are choosing to bundle all these binaries into a Linux distribution that is as close to Red Hat Enterprise as we can get it.

The goal is to ensure that if a program runs and is certified on Red Hat Enterprise, then it will run on the corresponding Fermi Linux LTS release.

A follow-up Q goes, “I really don’t

One Tale of Two Scientific Distros

The 1994–2007 Archive CD, back issues, and more!

www.LinuxJournal.com/ArchiveCD

Includes issues 1–164

of Linux Journal

(21)

[

UP FRONT

^]

want to get into legal trouble, please convince me that this is legal.” The A says:

What we are doing is getting the source rpm of each Red Hat Enterprise package from a publicly available area. Each of these packages, except for a few, have the GPL license. This license states that we can freely distribute that package.

We are recompiling those packages without any change.

Hence, we can freely distribute those rpms that were

built....And although these rpms are basically identical to Red Hat’s Enterprise Linux, they were built by us and are freely distributable. We can do with them what we want....

Although it is basically identical to Red Hat Enterprise Linux, it is, in essence, a completely

different release, just with the same programs, packaged the same way.

Fermilab supports its own users and directs others toward Scientific Linux, which was codeveloped by

Fermilab, CERN and other laborato- ries and universities. Troy Dawson is the primary contact for both Fermi Linux and Scientific Linux. On his own site, he explains, “Fermilab uses what is called Fermi Linux. It is now based on Scientific Linux. It is actually a site modification, so technically it is Scientific Linux Fermi. But we call all of the releases we have made Fermi Linux.”

While Fermi Linux’s version history

starts with 5.0x in 1998, Scientific Linux’s history starts with 3.0.1 in 2004.

Both sites’ current distribution version pages have near-identical tables of releases, dates and notes. The latest version for both is 5.x.

In a comment to an on-line Linux

Journal article (www.linuxjournal.com/

article/8253), William Roddy wrote,

“Scientific Linux will work in any environment Red Hat would, and even better. It’s a work of art and genius, and in the field of high-energy physics, if this Linux didn’t work, it wouldn’t be used. Yet, it is useful to anyone. If you demand stability and security, you will not do better.

It will always be there and it will always be free.”— D O C S E A R L S

(22)

[

UP FRONT

^]

By 1988, Larry Bird had already won the three-point shootout twice. In the locker room, while waiting to defend his title, he said to his opponents,

“Who’s finishing second?” Then he went out and won for a third time.

Ubuntu is in the same position. In DistroWatch.com’s Page Hit Ranking, Ubuntu has been #1 for three years running. Mandrake (now Mandriva) was

#1 in 2004. Ubuntu took over in 2005,

then repeated in 2006 and 2007.

Meanwhile, the “Who’s finishing second?” question is always up for grabs, changing almost as constantly as all the other positions on down the list. Table 1 shows the results for three measures in 2008, taken on May 11.

Meanwhile, it’s pretty clear that Ubuntu will hold its lead for at least the current year.

Ubuntu Stays atop a Volatile Distro Market

WiiLi (www.wiili.org) is a Linux port in the works for the Nintendo Wii.

Drivers are being developed for Wii features, including the SD card slot, wireless 802.11b/g and Bluetooth hardware, and for the remote.

A proof-of-concept Wii distribution was rolled out earlier this year by the GameCube Linux Project. It relies on a Twilight Hack developed by Team Twiizers. The hack leverages an exploit in a Wii game title to launch a Linux bootloader (see a video of this in action at www.linuxjournal.com/

content/meet-mii).

We’re eager to see Linux on the Wii. In the meanwhile, settle for see- ing Linux Journal on the Wii—we all have our own Miis here in the office.

Add Linux Journal as a Wii friend, and we’ll send you any Linux Journal staffers you’d like. Our console

number is: 2924 2525 5556 2687 (post yours in the comments at www.linuxjournal.com/content/

meet-mii, and we’ll add you too).

— K A T H E R I N E D R U C K M A N

Meet Mii at LinuxJournal.com

I predict an odd period in history, where famous quotes from techies will all be 140 characters or less.

—Keith Hopper, twitter.com/khopper/statuses/

801585685

Here’s to “Now” for as long as it lasts.

—Shelora Fitzgerald, quoted in an e-mail to Doc Searls

Productivity is up 99% because nothing is failing.

—Heather Carver, District Technology and Information Services Director for the Windsor Unified School District in California, which is migrating to a Linux-based system of servers and thin clients, www.edtechmag.com/k12/issues/may-june-2008/

save-green-by-going-green.html

For about a year, however, Microsoft has been working to get a slimmed- down version of Windows to run on XO laptops. As a result, Negroponte said Tuesday that he expects XOs to soon have a “dual-boot” option, meaning users would be able to run Windows or Sugar....Eventually, Negroponte added, Windows might be the sole operating system, and Sugar would be educational software running on top of it.

—Associated Press, April 22, 2008, ap.google.com/article/

ALeqM5hXa0O9XLMsWfaqt-sI9FqFy2IewgD9073PPG0

It’s easy to get caught up in the doom and gloom over OLPC’s future. But keep things in perspective: they aren’t as bad as they seem.

To the developers at OLPC and the tireless volunteer community contributors unsettled by Nicholas’

plans—remember that no matter what happens, your work has not been for naught. Far from it. You brought the smiles to children’s faces in Escuela No. 109 in Florida, Uruguay. Your work astounded me with the results, after little more than half a year, in the mountains of Arahuay, Peru. Bryan Berry’s team is kicking ass on establishing a pilot in Nepal because of your work. And if you haven’t read the linked articles yet, now’s the time.

Nothing can take away the real, palpable impact you’ve already had on children’s lives.

—Ivan Krstic, April 25, 2008

They Said It

2007 Last 3 months Last 30 days Last 7 days

1 Ubuntu Ubuntu Ubuntu Ubuntu

2 PCLinuxOS OpenSUSE OpenSUSE OpenSolaris

3 OpenSUSE Fedora Fedora Puppy

4 Fedora Mint Mint OpenSUSE

5 Sabayon PCLinuxOS Mandriva Fedora

6 Mint Mandriva PCLinuxOS PCLinuxOS

7 Debian Debian Debian Slackware

8 MEPIS Dreamlinux Kubuntu Mandriva

9 Mandriva Sabayon Slackware Mint

10 Damn Small Damn Small Damn Small Debian

Table 1. Results

(23)

Geoff Goodell has a PhD in Computer Science from Harvard and is currently researching Internet surveillance and network neutrality. Recently, I found myself sitting next to Geoff, who was typing casually at a furious speed on his ThinkPad, while looking at a black screen divided into four quadrants, each apparently operating in command-line mode with colored type far too small for my old eyes to read, but obviously quite legible to Geoff. Because it was too hard for me to eaveswatch, even at close range, I asked him if he was an Emacs or a vi guy. He smiled and replied, “vi”.

So, I asked him to let us know the rest of what he used, and here’s his reply:

I use a constellation of machines running Debian Linux. I store most of my working files (such as research, code, papers and personal Web pages) in AFS, which allows me to share data across machines seamlessly. For authentication, I use a combination of MIT Kerberos and OpenSSH public keys. I exchange multimedia files using rsync, and I use MPlayer for audio, video and streaming.

For hardware, I use mainboards by Gigabyte, monitors by NEC and Samsung, IBM M13 keyboards and multihead video cards by NVIDIA. My laptop is an IBM ThinkPad T41, which also runs Debian Linux.

For software, I try to keep a low profile. My window manager is ion2, which allows me to tile the screen with xterms and mostly avoid using the mouse. My favorite font is 6x10, and my color scheme is something that evolved over the years. For text editing, I use Vim. For e-mail, I use Mutt and GnuPG. For IRC, I use Irssi. For browsing, I use Firefox plus NoScript plus AdBlock Plus. For privacy, I also use Tor. I run Linuxnet IRC servers, Apache Web servers and an exim4 mail server. I coordinate non-IRC instant messaging through a set of gateways, so that my IRC client handles all the interaction. I use LaTeX to write papers, personal letters and presentations (occasionally I use OpenOffice.org for slides).

My ISP is Speakeasy. My domain registrar is GANDI (in France).

When not near a workstation or laptop, I have a Treo 750p (which runs a lovely SSH client called pssh) and a Motorola H12 noise-canceling Bluetooth headset.

What They’re Using

Geoffrey Goodell

1259 Reamwood Ave Sunnyvale, CA 94089 www.kingstarusa.com

Email: [email protected]

KING STAR COMPUTER

Customizable system solutions since 1989

Tel: 1-800-875-8590 Fax: 408-736-4151

2U Supermicro 6026-TR+B

2U Barebone Sytem w/ No RAM 2GB *-16GB

No CPU $1,345

Xeon Quad Core 1.6Ghz $1,548 $1,615*

Xeon Quad Core 2.0 Ghz $1,660 $1,727*

Xeon Quad Core 2.66Ghz $1,793 $1.860*

Intel®, Intel® Xeon^TM, Intel Inside®, Intel® Itanium® and the Intel Inside® logo are trademarks or registered trademarks of Intel Corporation of its subsidiaries in the United States and other countries.

Prices and availability subject to change without notice. Not responsible for typographical errors.

No CPU $943

Xeon Quad Core 2.0 Ghz $1,258 $1,325*

1U Barebone System w/ No RAM 2GB

technology services provider across the United States in industries including Fortune 500 companies, mid-size to small business, start-ups and govern- ment to educational organizations.

King Star Computer can help you config- ure your next high-performance server system - Contact us Today

To learn more, Visit www.kingstarusa.com

King Star Computer

is a leading

1U Supermicro 6015B-TV

Tel: 1-800-875-8590 Fax:408-736-4151

(24)

I am writingthis article in mid-May 2008, several weeks after Twitter was rumored to be moving to a platform other than Ruby on Rails. Twitter, of course, is an extremely popular service that allows users to write updates and notes about their current status—and allows readers to follow any number of people’s Twitter feeds. You can think of Twitter as a combination blogging and RSS platform, populated by people who express themselves with only 140 characters at a time.

Like many other runaway Internet successes, Twitter appears to have become too popular for its own good. This has led to some outages, most notably one at the beginning of 2008 that took more than a day to restore. Thus, it was seen as more than a mere coincidence when Twitter’s main architect left the company, and that within a few days, the TechCrunch blog was quoting anonymous officials within Twitter about how the service would be transitioning away from Ruby on Rails.

This was followed by a great deal of discussion over whether Rails is a “scalable” architecture.

Scalable used to mean that it was possible to scale up applications using a Web site, almost regardless of how many people are using it. But today, a scalable architecture is one that’s lean and mean, handling as many users as possible with as few servers as possible. PHP, Java and .NET are pretty universally considered to be scalable in this sense.

Although even the most efficient PHP application can handle only a finite number of simultaneous users, it’s undeniable that Ruby is a slower language than PHP, and that the Rails framework adds some more overhead.

Of course, it’s one thing to say that Rails doesn’t scale as quickly as PHP, and another to say that it doesn’t scale at all. And, there are other arguments to be made, including the fact that programmers cost more than servers, and that programmer productivity should be at least as significant a factor as scalability.

That said, it’s easy for a Rails application to become slow. So, it is nice to know that a variety of utilities can be used to profile Rails applications—

meaning, finding out exactly which portion of the program is taking a long time to execute. This

month, we look at some techniques for profiling Rails applications. Although such profiling doesn’t make the software run any faster, it can help identify the slowest parts of an application.

Profiling Pages

If you aren’t happy with the performance of your Web site—and quite frankly, you always should be concerned about the performance, trying to give it a boost wherever possible—the first question to ask is, “Where are people spending their time?” After all, if there are 100 different pages on your site, it doesn’t really matter whether page 35 is really slow if no one ever visits it.

The first tool to examine, the production log analyzer, is designed to look at the Rails production log and produce some basic statistics about it. The production log, as well as the development and test logs, are typically stored in the log directory under the Rails project root. Thus, on a production server, the log is in log/production.log. This logfile is not rotated or modified automatically; there are clearly a number of ways to do that using cron and other UNIX command-line tools.

The thing is, there already is a facility on UNIX (and Linux) systems for handling logfiles, including their periodic rotation and disposal. This facility is known as syslog, which makes it possible to send logging information to a variety of different files based on priorities and source materials. The /var/log directory on my Ubuntu server is full of different logfiles, and nearly all of them were created and written to by syslog.

It turns out that we can use syslog for our Rails production logs. Once we have done that—

and yes, we must use syslog for this to work—we then can analyze our production logs, learning exactly how much time people have spent in our Rails application.

To move your Rails production log to syslog, you need to do several things. First, you must install the Ruby gem that provides this behavior:

gem install --remote SyslogLogger

This installs the gem in the appropriate place

Profiling Rails Applications

Wondering if your Rails application is running at peak efficiency?

Before optimizing, profile your application to see which parts are slow.

AT THE FORGE

REUVEN M. LERNER

(25)

on your system; on mine, it was put into

/usr/lib/ruby/gems/1.8/gem. Next, you need to add the following to one or more of your environment configuration files (either environment.rb or one or more files in environments/*.rb) for your Rails system:

require 'syslog_logger'

RAILS_DEFAULT_LOGGER = SyslogLogger.new

This, of course, loads the syslog_logger gem and sets the default logger to a new instance of SyslogLogger.

Now that you have told Rails to use syslog, you must tell syslog what to do with the files that come from Rails. I opened /etc/syslog.conf and added the following lines to the bottom:

*.info /var/log/production.log

And yes, the documentation system says that you can use a !rails tag before this line, or one like it, to restrict logging to messages coming from Rails.

Unfortunately, this syntax does not appear to be supported by Linux. So, this means production.log will include messages from other programs and facilities, not just Rails. That shouldn’t concern us right now, although it might be an issue on a busy machine with many services in active use.

Once you have modified syslog.conf in this way, you can restart syslog.conf. Almost immedi- ately, your production log should be stored to /var/log/production.log. You can check this, of course, with the following:

tail -f /var/log/production.log

Now, this logfile is similar in many ways to the logfile you just eliminated from the log directory in your application root. However, it is formatted in such a way that the production log analyzer will be able to find and perform calculations based on its output. To analyze the logfile, type:

pl_analyze /var/log/production_log

(26)

If you prefer to have the results sent to you via e-mail, rather than stored to a disk file, use the -e option:

pl_analyze /var/log/production_log -e

¯[email protected]

This option is particularly useful when you invoke pl_analyze from a cron job, for example.

The output file from pl_analyze is divided into three parts:

I Time spent in each request.

I Time spent in the database for each request.

I Time spent rendering the output from each request.

For each controller action, pl_request lists how many times it was invoked, as well as the average time it took to execute. It also gives the min, max and standard deviation, allowing you to see how much the execution time varied over time.

Thus, the production log analyzer shows which actions take the greatest amount of time overall, which take the greatest amount of time in the database (or to render) and how many times each was invoked.

I have found pl_analyzer to be an indispensable tool when trying to determine whether a site is fast enough and where I should focus my attention to improve its speed.

Request Profiler

The production log profiler shows which actions require attention, but it doesn’t tell why a particular action might be giving you trouble. For that, you need to dive into the application a bit more, profiling not a set of actions, but one particular action.

This is possible thanks to a built-in script that comes with Rails, script/performance/request. This script follows a set of instructions written in a (pre- sumably short) Ruby program, using a similar set of commands and subroutines that are available for integration tests.

In other words, you use integration-test syntax to describe a short sequence of one or more actions and run this program via the request profiler. Then, the request profiler produces two output files that describe what was going on behind the scenes as those requests were serviced. This information can help you improve the performance of this particular action.

In order for this script to work, first install the ruby-prof gem:

gem install --remote ruby-prof

Once that is installed, you need to create a simple integration test script. This script doesn’t need to be wrapped in the same object that the integration tests themselves use. Instead, simply create a file named test.rb, and put it somewhere on the filesystem. I created a directory named test/performance and put it there, with the one-line contents as follows:

get('/')

Notice that I’m using URLs here, rather than names of controllers and actions. Finally, with this in place, invoke the profiler:

script/performance/request -n 10 test/performance/test.rb

Now you should see the program telling you that it’s warming up and then reporting as it goes through each of the iterations you specified. In the above example, the -n 10 option indicates the number of times the script should be invoked;

by default, it’s 100.

Note that the output files are put in the test directory (to which you might not have write access by default). And, indeed, the output files are quite useful, but they can be confusing the first time you look at them.

The first output file, profile-output.txt, is (as the suffix implies) a text file that shows how much time was spent in each method, both as a time measure and as a percentage of the total run time. Consider the following:

%self total self wait child calls name 13.74 58.35 38.13 0.00 20.22 608720 Buffer#read

AT THE FORGE

Resources

The Rails Way, by Obie Fernandez, has become my favorite, because it includes so much useful information, as well as code examples. It doesn’t try to teach you Rails, but it does provide a great deal of information that is useful for advanced users as well as newcomers.

Advanced Rails, by Brad Ediger, gives some greater depth to several topics, such as performance opti- mization, ActiveRecord features, RESTful sites and internationalization, among others.

Rails Analyzer Tools: this is a collection of tools that can help you better understand your Rails-based site.

The production log profiler is part of the Rails Analysis Tools set; see rails-analyzer.rubyforge.org for more information.

(27)

This means there were 608,720 calls to Buffer#read during the test, which took a total of 38.13 seconds, or 13.75% of the execution time. Because this is a built-in method, you can’t optimize it. However, you can try to reduce the number of times it is called, so that it will take even less time.

The question is, how do we know which functions are calling Buffer#read? Perhaps reading from buffers is an inevitable part of a Web application, and we just need to realize that?

If you look at the second file, profile-graph.html, you see a nicely linked description of which methods called which other methods, and how long it took. Each box represents the analysis of one method, and the method being analyzed is printed in bold.

All of the methods above the boldfaced method name are parent methods (that is, methods that called the one in question);

whereas, methods below the current one are child methods (that is, methods that are called by the method being analyzed). By looking at

who called Buffer#read, you can see which methods (if any) need optimizing or a smaller number of invocations. By going back and forth between methods, their parents and the source code, you can cut down on a great deal of waste, making your sites more efficient than before.

Conclusion

This month, we looked at two basic profiling tools that programmers can use to identify performance problems in Rails-based Web sites.

There are, of course, other tools we can use, but the fact that these are so nicely integrated into Rails makes us all the more likely to use them.

With constant monitoring and tweaking, we can make our sites run faster without having to resort to buying additional servers.I

Reuven M. Lerner, a longtime Web/database developer and consultant, is a PhD candidate in learning sciences at Northwestern University, studying on-line learning communities. He recently returned (with his wife and three children) to their home in Modi’in, Israel, after four years in the Chicago area.

Since 1994: The Original Magazine of the Linux Community