Active Directory

The largest impact of the upgrade came from the changes brought by Active Directory, a new technology component of Windows 2000 architecture. While it is possible to adopt

Windows 2000 without implementing Active Directory, many of the benefits anticipated in Windows 2000 would not be realized without this back-end component.

Active Directory was a leap in network technology integrating applications, users, and data into a centralized location. Serving as the “main switchboard of a network operating system,” Active Directory allowed system administrators to manage computing in distributed environments. Although the benefit of active directory was not obvious to standard users, IT personnel found the new back-end architecture to be very impressive. One task in particular was the capability to distribute software over the network.

At a large organization like CP, applying security patches or application upgrades to users was labor intensive and time consuming. Prior to the Windows 2000 project, the standard way of applying software patches or upgrades to users’ systems was by making personal visits to each desktop with a disk in hand and performing the upgrade manually. Thus, the new capability for IT personnel to automate the deployment of patches and upgrades was seen as, “one of the largest advantages,” according to the planning manager. “Some applications like SAP “would

come out with an upgrade once a quarter, where we would have to go out once a quarter and hit every machine,” he said. “We don’t have to do that anymore. And, that would always be like a week-long process,” recalled another operations manager.

Even though not all software had a client upgrade every quarter, many other software applications like Norton AntiVirus, MS Explorer also required frequent updates of files. Each round of software distribution could take from a few days to several weeks to complete. The senior manager of the Infrastructure Support Group explained the time was dramatically decreased after Active Directory was installed.

Over the last few months, there's been a whole bunch of security patches, for OS,

browsers, and Outlook…Through Windows 2000 Active Directory, we have been able to apply these patches automatically within, actually within half a day, we have applied the patches to 9200 desktops vs. weeks doing by hand… So it's been a huge impact there.

- Sonny, Senior Manager of ISD

The impact of Active Directory extended beyond the time saved on software deployment. Before Active Directory was put in place, software installation was conducted after regular office hours to minimize the interference of upgrade on users’ routine. Zack could not help displaying his joy as he recalled how that had changed, “So I can tell you myself I have not been here on weekends doing that sort of thing anymore, ha ha ha (laugh).” Even though no official report showed how much time CP saved, the amount of overtime that IT put in was dramatically decreased, “I know that for a fact,” said the manager.

Besides software distribution, Active Directory allowed system administrators to have a tighter control over users’ desktops. Through the use of group policy and other technical

features, a system administrator could exert control from the type of application that could run on the desktop to what a user can perform on the system. “There is a lot more control over user policy and stuff, keeping them from loading unauthorized software; a lot easier in 2000,”said the IT contractor, James. After the upgrade, many users found themselves stripped of all privilege to install any software. While some of the power users were not thrilled with the changes, system administrators have found the undertaking necessary to enable better security for the

organization.

What Active Directory brought to the table was not just the capability to remotely distribute software or tighten control on user’s desktop. The new technology created an architecture that integrated all network components together and revolutionized back end operation management.

Nonetheless, Windows 2000 implementation was not an overnight success. Besides the long technology governance process used to evaluate Windows 2000, all IT personnel at CP, depending on their job responsibility, attended between three days to one week of training. Like any IT project, Windows 2000 was carefully planned to make sure the deployment would run smoothly. Despite the best effort, the biggest challenge of Windows 2000 upgrade came a few months into the rollout process when Active Directory crashed.“We were about a third of the way rolled out, every single one of those sites that were rolled out in windows 2000, they were down. The whole [manufacturing facility] was down. They weren’t making [product]. It was bad, for about a week.” The project leader recalled that he and his team were in the field

deploying Windows 2000 when the disaster happened. “The rollout completely stopped for about a two, three-week period. We had to cancel trips, we had to come home from where we were at,

we had to pack up and come home. It got very ugly. We had some ugly conference calls. People saying ‘Somebody needs to get fired, we’re losing millions of dollars a day.’”

As it turned out, someone in the field location had placed a large file into a folder that automatically“gets replicated all over the entire enterprise, from Seattle, Washington to

Jacksonville, Florida and everywhere in between…ha ha ha,” one IT manager recalled, laughing while telling about the incident. Of course, it wasn’t funny when IT personnel were in the midst of it. “We had to do a lot of disaster recovery; we had to make a lot of phone calls, a lot of apologies. We had to get Microsoft in here to help us get it back up and functioning. It was just a mess for about three weeks,”said the project leader.

The Windows 2000 project was a partnership between NTPS and CP. While CP was responsible for bringing the upgrade to thousands of users across the nation,NTPS was in charge of the back end infrastructure. Even though the operations of Active Directory would eventually be passed to the corporate operations group when the environment was considered stable by NTPS, corporate IT essentially owned the technology at the initial stage.

We are accountable for that platform if you will. So if anything happens my group’s accountable for it during rollout… if they roll it out to the [manufacturing facilities] and there is a design issue or there’s a problem with the operating system that impacts their business they’re going to come to my team for an answer of why they’re having

problems.”

Although the division and corporate IT were polite to not point fingers, slightly different perspectives were given when they talked about the incident. The project leader at CP felt corporate IT was still treating the project like a test and didn’t give it the serious attention it deserved. “What it did to those guys is they realized they needed to get in gear and get this into production where they’re monitoring it, where they know when something goes wrong before it happens instead of somebody calling from across the country saying “I can’t log on, why not?” From corporate standpoint, one of the problems was having too many IT personnel who had “admin. capability to make changes in Active Directory.” Someone who had insufficient knowledge could easily make an error and create an outage. While acknowledging it was a learning curve, the project manager at corporate IT explained they had taken precautionary steps to prevent the incident from recurring.

First, they began to perform “frequent backups” on multiple domain controllers including the main database, several times a day. Second, they “greatly limited the people who have ‘that’ level of access within ‘that’ area of the active directory.” Restricting the access to Active

Directory to four people within each division allowed them to have better control over Active Directory. Third, they changed the approach used to manage the infrastructure, “Another action we’re taking is to centrally manage the domain controllers within one group, instead of having that across all divisions.” It took some adjustment to implement this new policy because CP put domain controller and applications on the same server, and its IT personnel would need access to the server. To solve the problem, corporate IT provided the division with a dedicated server to run domain controller so personnel from corporate IT could have full control over it.

CP ended up rebuilding all domain controllers for the field locations that were affected. Despite the mess that the incident caused, the project lead was glad that it happened early in the

upgrade process, because it could have shut down the entire division if it had occurred after all manufacturing sites had migrated to the new systems. It was a learning experience for all parties. After the dust settled, project lead reassured offices at remote locations to continue with the upgrade.

Finally, then I had to get everybody’s nerve back up; okay, who wants to do the next rollout? …because people were, like, I don’t want it now, I don’t want it. So we almost had to resell it again saying “look everything’s fine now, it’s not going to happen again” and we haven’t had anything happen since then. And they’re doing a great job of

monitoring the systems. There have been problems but they caught them real early and before it affects anybody.

- Luke, Project Leader