Introduction
This paper outlines the successful design and implementation of a 100% virtualized environment, capped with an implementation of VMware View at GreenPages Technology Solutions, a 190-employee, $115M national IT consulting and integration firm that helps midmarket and enterprise clients fully virtualize their environments and transform their datacenter and IT operations to strategically leverage the power of cloud computing. In addition, GreenPages has a growing Managed Services portfolio of offerings which provide next generation cloud management services as well as 24x7x365 monitoring and systems management.
GreenPages has offices in Kittery, ME; Boston, MA; New York, NY; and Atlanta, GA; as well as numerous employees who work full-time from their home and a large contingent of engineers who are full-time “road warriors.” The implementation of VMware View makes GreenPages’ production environment 100% virtualized, including the phone system.
The technology solutions in use at GreenPages are very similar to those that GreenPages provides to its customers. The company often deploys solutions that are very early in the technology lifecycle, providing real-world experience to GreenPages’ engineers that cannot be duplicated in a lab environment; this “use what you sell” approach is a value differentiator for GreenPages and allows the company to thoroughly understand and test best-of-breed solutions before deploying in customer environments.
The Vision
With a growing number of remote offices and employees working from home offices, GreenPages’ internal IT staff found it difficult to provide reliable, consistent access to corporate resources. Further adding to the complexity was the increased employee use of smartphones and other personal devices. GreenPages’ ultimate vision was to have systems that could be accessed:
From any device
From any location
At any time
These systems had to be uniform in nature, and easy for a small IT staff to manage.
GreenPages VMware View Design
Reference Implementation
The technology solutions in use
at GreenPages are very similar
to those that GreenPages’
provides to its customers.
Employees should be able to move from device to device, and location to location, seamlessly. Additionally, employees should be able to work at any time, and connect to corporate systems on a variety of personal devices with little to no assistance from internal IT. Most important, corporate data must remain secure, and access to corporate systems and data centrally controlled. For example, an employee should be able to access corporate systems from a home PC, disconnect from that session, and reconnect to that session from the office. When in the office, an employee should be able to jump from a PC to a smart device (such as an iPad), and back to the PC. Employee documents must be available on the network, and when an employee leaves the company, access to all systems must be immediately discontinued. GreenPages employees would need access to the complete Microsoft Office suite of products including Visio, FileMaker PRO, a variety of internal and external websites requiring multiple browsers, and various accounting and HR applications.
Background & Challenges
Based on Badger’s Island in Kittery, ME, GreenPages’ corporate office sits on the edge of the Piscataqua River, with impressive views of neighboring Portsmouth, New Hampshire. Ships taller than the 4-story headquarters routinely sail by; although the view is nice, life on an island can be challenging for a rapidly growing 24x7x365 company.
GreenPages’ Kittery headquarters was originally designed to handle the IT needs of a smaller organization, where all users worked in a single building, Monday – Friday, 8:00 – 5:00. A small server room was sufficient to handle the few servers necessary to provide services for the corporate employees, and keeping on top of desktop support was a relatively simple task for the IT staff. Data circuit requirements were generally low, with only email and general internet browsing required. A standard phone system was sufficient for the largely static employee base.
As GreenPages grew into a larger solutions provider, staff numbers increased rapidly, more servers were required to handle the load, lab environments had to be built, and more and more employees worked outside the corporate office on a daily basis, requiring remote access to systems.
The increased server load in the Kittery office also taxed both the power and air conditioning in the server room, and on several occasions, servers experienced thermal shutdowns when the air conditioning system failed. Ultimately, GreenPages’ rapid growth required more servers, but there was insufficient power available to add the servers, the air conditioning couldn’t support the additional heat generation, and physical space was running out in the small server room.
Employees should be able to
move from device to device, and
location to location, seamlessly.
As a solutions provider specializing in virtualization, GreenPages knew it had to virtualize its own infrastructure, not only to address the obvious physical reasons, but to use itself as a technology test bed—if GreenPages was going to sell the solutions, it needed to know how the solutions really ran in a production environment; testing in a lab just wouldn’t cut it.
GreenPages set out on a three-phase upgrade to become 100% virtualized, capping off with VMware View, as outlined below.
PHASE 1 - Hardware Upgrade and Server Virtualization
GreenPages started down the virtualization path with a total of 5 HP DL380 G5 and 2 DL360 G5 servers, running ESX 3.5, on a Dell EqualLogic iSCSI SAN. Legacy HP ProCurve core switching was retained. Physical servers were converted via P2V, starting with basic (file/print, web, etc.) servers, followed by more advanced client/servers. Critical client/server, SQL, and Exchange servers were rebuilt as new virtual machines and services migrated accordingly. Only one system, an internally developed sales application based on a FileMaker database, remained on a physical server. The FileMaker system was not converted due to its critical role in the organization, and (then) current lack of support for a virtualized environment.
At this point, GreenPages’ infrastructure went from more than 30 physical servers, each with Direct-Attached Storage (DAS), to 7 physical servers running more than 40 virtual machines and an iSCSI SAN. Power and cooling issues were eliminated, and space was available for a Solution Architect lab environment.
Over the next few months, GreenPages’ IT staff evaluated and optimized performance of the P2V machines, mostly creating new VMs and migrating services to them, and retiring the old VMs. Additional VMs were built for lab and experimental use, ultimately bringing the total number of VMs to just fewer than 100.
In addition to the growth of VMs, physical equipment added to the server
As a solutions provider
specializing in virtualization,
GreenPages knew it had
to virtualize its own
infrastructure.
room quickly took up the space vacated by the old servers, and the power/ space/air conditioning issues returned. There was limited room for VM ex-pansion on the existing ESX cluster due to memory and CPU utilization, and the core network switch was overutilized due to the increased load from the additional servers.
Phase 2 - Hardware Upgrade, IP Telephony, and 100%
Server Virtualization
GreenPages still needed to get to a 100% virtualized environment, but the new ESX cluster didn’t have sufficient capacity to get there. The server room was again overtaxed, so basic expansion by adding additional servers was not feasible. Additionally, GreenPages wanted to implement a virtual Cisco VoIP phone system, which is only supported on Cisco UCS hardware.
GreenPages went with the latest V + C + E (VMware + Cisco + EMC) solution, deploying 7 Cisco UCS blades spread across dual UCS chassis, a Cisco Nexus 7000 core switch, VMware vSphere 4, a Cisco Nexus 1000v virtual switch, and an EMC NS-120 Celerra SAN.
VMs on the existing ESX cluster were migrated to the new environment using SVMotion; the storage was then re-provisioned.
With the new equipment in place, scalability was no longer an issue (power and air conditioning still were) and the virtual Cisco VoIP system and FileMaker system went into place with no issues.
At this point, 100% of GreenPages’ server infrastructure was virtualized, leaving only the desktop environment and server room power and air conditioning issues that needed addressing.
PHASE 3 - Virtual Desktop Implementation
GreenPages looked at multiple options for its VDI environment, and ultimately decided on a VMware View 4 deployment, primarily due to the company’s familiarity with VMware and the built in management features available in the technology. When looking at the various solutions, the protocol being used was taken into consideration as well. GreenPages determined that the PCoIP solution offered by View would be a more viable long term protocol due to its ability to adjust to varying bandwidth situations and customization.
GreenPages added three Cisco B230 blades to the existing UCS chassis, and took one more from the existing production environment, dedicating them to VDI to ensure performance.
The View 4.6 environment GreenPages deployed consists of a locked-down Windows 7 base image with the FileMaker client installed. This image is deployed non-persistently to users, with additional applications such as
VMs on the existing ESX
cluster were migrated to the
new environment using
SVMotion; the storage was
then re-provisioned.
Microsoft Office 2010 and Microsoft Project layered on using ThinApp. User Profile information and individual settings are stored on the network and are managed using Profile Unity.
A single View security server provides remote PCoIP access to the View environment, allowing access to remote users on various platforms including PC, Mac, iPad, iPhone, and Android devices.
GreenPages’ IT staff took the older desktops in use at the Kittery office, migrated any local data to users’ individual home drives shared on the network, and deployed a locked-down Windows XP base image with the View client on the desktop. For hardware, the old desktops were re-provisioned as “thick” clients and any desktops needing replaced were done so with EVGA PD02 zero clients. The zero clients were chosen for their built in Teradici chips and ease of management and deployment.
GreenPages knew that a fully virtualized environment with all
systems—servers, desktops, and phones—entirely dependent on systems running in an outdated, underpowered, and undercooled server room in a building on an island was risky. Rather than invest in renovations to the server room, the ultimate decision was to move those systems to a colocation facility where redundant power, air conditioning, and data circuits are standard.
Network layout with remote offices connected via MPLS. Pre-move to the datacenter.
Presenting virtual desktops with the core infrastructure connected via LAN, then moving that infrastructure to a WAN connection creates a bandwidth challenge; the gigabit ethernet connectivity to desktops available via LAN is far more responsive than connectivity over WAN. Since GreenPages planned on moving the infrastructure, the IT staff did not want to roll out VDI on an “unlimited” local connection only to have the connection speed and user
The zero clients were chosen
for their built in Teradici chips
and ease of management and
deployment.
experience reduced after the move. Additionally, because GreenPages did not want to pay for more data bandwidth than absolutely necessary, the decision was made to artificially limit bandwidth per connection to accom-modate approximately 60 user sessions on a 20Mb data circuit. This decision was made to validate that there would be sufficient bandwidth for acceptable performance, and the GreenPages users would not see a performance difference after the infrastructure moved.
As a result of the bandwidth limitations, video performance in View was acceptable, but not as responsive as the local desktop had been. Because of the artificially limited bandwidth, PCoIP’s progressive build capability worked as advertised, presenting the desktop first as a lossy image, then progressively improving the image.
PHASE 4 - Data Center Relocation
After the successful Kittery office rollout, GreenPages focused on moving the server infrastructure to an 80 square-foot cage in a more stable colocation facility (colo) in Boston, Massachusetts. Internet connectivity at the colo consists of a 20Mb connection on the Kittery MPLS network, and a 30Mb direct internet data circuit. All intra-office communication traverses the MPLS network, and all internet traffic goes on the 30Mb circuit. Remote connections all connect to the 30Mb circuit.
View performance after the data center relocation was the same for Kittery users, and actually improved for users accessing View remotely, which was attributed to the increased bandwidth at the colocation facility. After the relocation, GreenPages lifted the artificial limitations on PCoIP to allow PCoIP to manage itself and further enhance performance.
Network layout with remote offices connected via MPLS. Post-move to the datacenter. Note that all users are now considered remote.
After the relocation,
GreenPages lifted the artificial
limitations on PCoIP to allow
PCoIP to manage itself and
further enhance performance.
PHASE 5 – View 5 Deployment
In January 2012, GreenPages upgraded its View infrastructure to View 5, running on vSphere 4.1, making no other changes apart from a standard upgrade. All users were upgraded to the latest View 5 client and the default View 5 PCoIP group policy settings were put into place.
After the upgrade, all users noted the performance increase, remarking that responsiveness under View 5 is as good, if not better, than it had been on local computers.
RESULTS TO DATE
GreenPages averages approximately 120 concurrent sessions on its View environment, and is very pleased with the results of the View deployment. Users frequently “hop” from machine to machine, working on their corporate machine, reconnecting on personal devices such as iPads for meetings, and working from their home computers. Performance, both internal and remote, from a user perspective is excellent using the default PCoIP group policy settings. GreenPages holds a quarterly VDI user group with its users to gauge feedback on this and other aspects of the View environment. With the current infrastructure the company expects to be able to double the amount of users while only having the increased bandwidth to accommodate both View sessions and other traffic such as VoIP that will increase with the user load. IT support requirements for end users have been greatly simplified. New applications and service patches are deployed quickly, either by updating the master image itself, or via ThinApp.
IT support requirements for
end users have been greatly
simplified.
Hardware Quantity Specs
Cisco B200 M1 blades 8 Dual quad Intel Xeon X5570 @2.93ghz 72GB RAM Cisco B230 M1 blades 3 Dual 8 core Intel Xeon
X6550 @2.00ghz 82GB RAM
Cisco 5108 Chassis 2
Cisco 6120 Fabric Interconnects 2
Cisco Nexus 7000 1
EMC Celerra NS120 1 59 x 450GB 15k FC drives; 19 x 1TB 7200 SATA II drives. Raw storage 45.5TB
EVGA 124-IP-PD02-KR Zero Client
Teradici PCoIP 3.5
Software Version
VMware vSphere ESXi 4.1 U1
VMware View 5
VMware vCenter server 4.1 U1
Microsoft SQL Server 2008
Microsoft Windows Server 2008 R2
Microsoft Windows 7 32bit SP1
UCS Manager 2.0(1q)
VMware ThinApp
Liquidware Labs ProfileUnity 4.8.0 Teradici PCoIP Management Console 1.7.0
Cisco Nexus 1000v 4.2(1)SV1(4a)
Rack layout of hardware in use, all UCS and Nexus hardware is contained in 1 rack. All internet circuits terminate in the Nexus 7000.
4 of the B200 blades are used solely for View 5.0, the other 9 blades are used for servers only.
Switching from the View desktops to the core switch is all 10GB/S.
2 View pools are run, 1 for production and 1 for testing. All changes are tested first before being published to the production pool. GreenPages keeps a minimum of 1 known good snapshot at all times in order to quickly roll back.
Average concurrent View connections are at 120.
Vsphere overview of hosts and machines running on the UCS cluster. On an average day the hosts run approximately 35% utilization on CPU and 46% on memory with spikes throughout the day ~10-15%.
About GreenPages Technology Solutions
GreenPages is a leading cloud management consulting and integration firm that helps clients fully virtualize their environments and transform their datacenter and IT operations to effectively leverage the power of cloud computing. The company’s deep technology expertise, broad engineering certifications, and vendor agnostic business model offer clients a strategic, cross-platform, proactive approach to designing, deploying, and supporting modern IT environments.
