July 20, 2013
Network Infrastructure Enhanced for Business Continuity
Executive Summary: A business continuity risk was identified that, if encountered, would have a high impact to CU members and operations. The risk was the potential failure of the physical network connectivity and/or communication provider networks to our Main Office and Data Center used for all voice and data communications directly impacting our 800#, call center, phone system, core system, website, online banking, email, and all other business systems. A cost-effective, flexible, and diverse network infrastructure using multiple communication providers, including a wireless broadband technology provider were implemented to mitigate this risk.
Strategic Challenge: A single telecom demarcation point has been identified as a medium risk, high impact business continuity disaster scenario that needed to be remediated. Adding a second demarcation point with geographical and provider diversity was seen as extremely cost-prohibitive. In addition, each remote branch location is connected via a single T1 communications line for both voice and data services.
A second challenge was to implement this enhanced network infrastructure within the existing IT budget for Disaster Recovery Services. Prior to this project MECU was spending $64,800/year with Sungard for very limited DR services and did not provide any network infrastructure redundancy.
Environment/Circumstances: The primary datacenter facility at our Main Office had only one last mile connectivity and network demarcation point into the facility from a local carrier. Three different network communications providers used the same last mile provider to deliver three different
communication services including: private MPLS network connectivity, public internet connectivity, and voice communications.
Previous Network Infrastructure Environment 1 of 1 Last mile Connectivity Single Point of Failure
1. Deliver reliable network connectivity services to support an SLA of at least 99.5% availability 2. Protect against a communications service provider network outage by distributing the
connectivity services across multiple networks and diverse media types. 3. Enable network connectivity with disaster recovery datacenter
4. Ensure network security is maintained.
5. Deliver flexible network infrastructure that can enable a physical relocation of the HQ office with a 48-hour RTO.
6. Cost-effective solution Solution:
• Fully utilize existing FatPipe ISP load-balancer, resilient connectivity management platform
• Contract with four different service providers for VoIP, public internet, and private MPLS network services.
• Include a wireless broadband service provider for both public internet and Ethernet private line connectivity services.
• Use wireless private line service as last mile for connectivity to MPLS provider to achieve geographic diversity for Main Office/Data Center and Disaster Recovery site connectivity
• Use cellular wireless data services as backup connectivity to each remote branch location, in addition to the T1 connectivity.
• The Wireless Broadband Services provides the ability to increase bandwidth to over 20MB with a phone call and 15 minutes of configuration time. In addition, they are able to setup an
alternate Wireless Broadband link to any facility we choose within 48-hours of a DR declaration.
• All wireless network segments utilize 256-bit data encryption to ensure security is maintained. In addition, the private network connectivity is implemented so that at no point does traffic on the private network ever touch the public internet.
Enhanced Network Infrastructure Environment Internet Sprint Global MPLS Network Lowell, MA Branch Lawrenceville, GA Branch Plantation, FL Branch Seguin, TX Branch Deer Park, IL Branch Libertyville, IL Branch Arlington Heights, IL Branch Schaumburg, IL Branch (IL02) Schaumburg, IL Branch (Corporate Tower) Schaumburg, IL Main
Office and Data Center
US Signal ISP Public Switched
SIP Trunk ISP
ISP Windstream Ethernet PrivateConnect Business Only Broadband AT&T Central Office 2 of 2 1 of 2
• Highly available, load balanced network connectivity services with increased and better-utilized bandwidth.
• The additional bandwidth implemented with our wireless broadband provider have been added to our network load balancer and is being used daily, as a result we have increased our useable network bandwidth to the Internet by 5MB and to our branches via the private MPLS network by 5MB. This provides us with the confidence that the enhanced network connectivity services are functioning and can be relied on in the event of a disaster.
• Disaster recovery location connectivity to main data center to allow for real-time replication of data and system configurations across the network.
• New disaster recovery cost model represents an annual cost savings with more redundancy and flexibility.
• Incremental Network Infrastructure Costs o Equipment
$17,000 - Fatpipe Network Load Balancer and Bandwidth Management o One-Time Non-Recurring Engineering Expense (NRE) - $1,000
o Service Providers
$5,868/year ($489 MRC) Wireless Broadband for Internet (5MB) $7,200/year ($600 MRC) Ethernet Private Line connectivity (5MB) $16,200/year ($1,350 MRC) Cross-Connect to Sprint (MPLS)
$0/year ($0 MRC) branch connectivity 3G wireless backup (included in existing branch connectivity costs)
o Annual Costs for Network Redundancy $20,868
• Former Disaster Recovery Services Provider Costs
o $64,800/annually for very limited DR services and no network DR services (Sungard)
• New Disaster Services Cost Model
o $29,268 ($2,439 MRC) for redundant network connectivity services o $31,824 ($2,652 MRC) for disaster recovery data center services o $3,708 Annual Savings
1. Network and Service Provider redundancy – a downed circuit does not cause an outage 2. Increased private and public network bandwidth
3. Support for disaster recovery datacenter and network infrastructure to allow much faster recovery in the event of a BC/DR scenario.
4. Reduced costs for more useable and flexible disaster recovery network infrastructure and services.