PACKET NETWORKING
SOLUTION HELPS MODERNIZE
MIAMI-DADE COUNTY
Executive Summary
In 2012, the Miami-Dade County IT Department (ITD) faced growing service demand from a widening set of applications, including video feeds for traffic and law enforcement, data center virtualization and backup supporting property tax and court records, and plain old voice service for county agencies. Supporting more than 25,000 employees and a population of 2.5 million people spread over roughly 2,000 square miles, the county faced a serious challenge to upgrade its communications infrastructure while supporting existing services within tight budget constraints. Their disparate user set included libraries, public safety agencies, courts, health care facilities, and every type of county-level citizen service agency. Each had its unique set of emerging applications and legacy service requirements. The county IT department needed to implement a converged solution that would continue to reliably deliver existing services, support new applications, and scale for the future at reasonable cost.
Ciena helped Miami-Dade design a network upgrade that allows simple transport of embedded TDM services while adding existing and emerging video and data services from various county agencies. The network topology makes heavy use of Ciena’s Packet Networking Portfolio, including the 3930 Service Delivery Switch and the 5150 Service Aggregation Switch. Combined, these two platforms create an optical-Ethernet service delivery mechanism operating over two separate protected 10 Gb/s optical rings touching essential locations around the county. This solution allows the county to assign each service a quality level based on application priority needs. Future applications, expected to be largely packet-based, may be added to the network via the 3930 or similar service delivery switches at strategic service points in the network and through utilization of excess capacity in the 5150 aggregation switches. If needed, ring capacity may later be increased to multiple 10G wavelengths as part of a larger network scaling upgrade through one of Ciena’s Optical Transport products.
This new network allows Miami-Dade to better support their public constituents in every aspect of government, from residents checking on recycling pickup or finding
Benefits
Benefits for government campus optical LAN or metro WAN deployments:
> Network flexibility – Deliver advanced services to any public service
department
> High reliability and resiliency – Operate with assurance that rare network outages will be quickly and automatically resolved
> Scalability – Quickly and efficiently adapt to growing network capacity and application needs
> Service security – Maintain separation of services through
department monitoring live traffic conditions to better manage incident response. All of these applications are powered through networking technology that allows the public to be better engaged in government operations, at a lower cost than ever thought possible.
The Old Miami-Dade County Government Network
Like most state and local municipal networks, Miami-Dade had evolved its networking infrastructure based largely on a voice circuit-based telecom infrastructure that supported various government departments. As applications such as fax, electronic record storage, and Internet access grew, the county had to accommodate demand by adapting existing TDM networks, often in a parallel, application-specific nature. In time, their applications began to include not only
traditional voice circuits but also packet-based data traffic provisioned onto TDM transport systems. Like many other large enterprise networks, they deployed SONET optical rings at OC-48 and, later, OC-192. The county operates most of this network over fiber leased from dark fiber service providers. By the time OC-192 was becoming the next logical upgrade step, the majority of their network traffic was packet-based bandwidth provisioned into a TDM
infrastructure. The design no longer made sense for future growth and proved to be expensive to provision, monitor, and maintain. A fundamental architectural change was needed to support immediate and future needs.
Public Access to Government Services
Much like state and local governments everywhere,
Miami-Dade faced a need to address a public thirst for rapid, effective, and transparent government service. Not long ago, services provided through in-person transactions, such as motor vehicle registration, were labor-intensive and time-consuming. State, county, and local governments needed physical offices and staff to provide access to many basic services. As online access became common and the ability to move services to Web-based applications emerged, most government entities began utilizing IT as a means to save money and improve service. By 2012, many services were migrated to Web portals where the public could complete transactions from their homes or on mobile devices. This trend is expected to continue as the public continues to become more familiar with and demanding of electronic service.
Miami-Dade County’s website lists their top ten online services as:
1. Property Search – searchable database of real property ownership and tax records
2. County Jobs – searchable database of open county jobs
3. Public Records Search – database of public court and other records
4. Property Taxes – portal for payment of real estate taxes
5. Water Bill – portal for payment of water and sewer bills
6. Inmate Search – searchable database of convicted criminals presently incarcerated
7. Vehicle Renew – portal for motor vehicle transactions, including drivers’ license and registration renewal
8. Services Near You – a wide range of general information ranging from census and county boards to trash pickup and zip codes
9. Transit Pass – portal for purchase of public transit passes
10. Bulky Waste Pickup – portal for entry of special waste pickup
Delivering all of these public services implies a need for connectivity among all branches and agencies, from the county to the state government. This connectivity must be scalable, as the demand from each is not static; seasonal and demographic trends alter short- and long-term demand. Also, the data generated by these varied applications must be protected and backed up among a reliable set of data centers.
Government Agencies and Operations
In addition to the public access aspect of Miami-Dade’s IT services, all county-level departments and agencies need to be connected to their respective branch offices and, in many cases, to the state or federal government. In addition to traditional voice service, these agencies increasingly look to IP-based voice, video, and data services as a means to improve their operational efficiency and effectiveness. Three specific examples include transportation, public safety, and county data centers.
Transportation – Working in conjunction with the state department of transportation, the county operates a network of video surveillance cameras to monitor live traffic conditions and respond to incidents as they occur.
Public Safety – Police, fire, and rescue departments are adopting tools for E911 call center and dispatch coordination and incident command.
Data Storage and Protection – As applications increase from public and government applications, data sets also increase. These large banks of information must be protected and stored in multiple physical locations as part of a
state governments increasingly rely on cloud applications and virtualization to accomplish this task.
Packet Networking Meets the Challenge
Carrier-grade Ethernet provided a perfect solution to Miami-Dade’s challenges because it leverages Layer 2 networking principles for standards-based transport for packet-based applications. Carrier-grade extensions to Ethernet increase overall network reliability, targeting five- or six-9s availability using classic telecommunications equipment supplier specifications to produce network elements with ultra-high reliability. It also increases the scalability by several orders of magnitude higher than traditional Local Area Network (LAN) designs.
Carrier-grade Ethernet technology has become widely adopted in enterprise and government environments because of its ability to offer differentiated service at a highly attractive cost. Standards-based carrier Ethernet is defined by the Metro Ethernet Forum (MEF) and the Optical Interworking Forum (OIF) as a connection-oriented Layer 2 service. Carrier-grade Ethernet enables the ability to offer bandwidth-assured, Layer 2 Virtual Private Networks (VPNs) with up to eight layers of Quality of Service (QoS), allowing differentiated service offerings for voice, video, and data IP networks.
The carrier-grade Ethernet standards and frameworks mentioned above are built on well established networking principles defined by IEEE 802.3. When applied to carrier-grade Ethernet, the benefit of this adherence is that the technologies still use standards-based Ethernet components to realize the lowest-cost solution per bit, delivered by leveraging the millions of ports of Ethernet deployed around the world.
Network Flexibility
Ethernet is inherently a multi-point technology routinely characterized as “plug and play” when adding devices to the network. With well-established mechanisms such as MAC flooding and learning, Ethernet can automatically discover paths through the network to enable communication. With this high level of built-in intelligence and adaptability, Ethernet can be deployed in any physical topology, including mesh, partial mesh, ring, and hub-and-spoke, or any mix of these architectures. Because Ethernet networks can adapt to growth and change, these topologies can expand easily or change over time.
All of these topologies have advantages, depending on their application. With the flexibility Ethernet provides, there is no restriction on how fiber can be rolled out to accommodate
the network. This increased flexibility reduces overall installation cost.
Additionally, Ethernet standards provide for the use of bidirectional optics, where both transmit and receive are performed on a single fiber, from 100 Mb/s to Gigabit Ethernet (GbE). This efficiency greatly maximizes the use of fiber, often cutting fiber costs in half.
Along with flexibility in physical topology, Ethernet provides flexibility in ‘logical’ topology. This refers to how services (Layer 2) are routed in the network. Traditionally, Ethernet networks were completely auto-learning, as mentioned above. This is still a valid premise in a campus or metro environment, but enhancements allow for connection-oriented switching, with paths defined by identifying specific connections during route provisioning. This arrangement allows deterministic path identification, letting network operators choose the network path services to better manage latency or maximize fiber and bandwidth utilization. This becomes more relevant as network topologies change and grow. Connection-oriented switching gives the ability to offer network security on par with circuit-based applications, with the flexibility and robustness of packet switching. Combined with carrier-grade OAM tools, a network operator can define and monitor network performance while providing assured information technology infrastructures for critical missions. Finally, because Ethernet is a well-defined standard, new vendor equipment can be interconnected to existing networks with minimal testing. This ease of transition makes Ethernet an efficient, risk-free option.
Ethernet Configurations and Equipment Flexibility
Ethernet has widespread global deployment and can be thought of as the de facto data link standard for carrying TCP/IP-based traffic. Deployments are ubiquitous in the user appliance connection to the edge, campus networks, and the Wide Area Network (WAN). As such, equipment manufacturers have created a large range of devices with varying options for port type, speed, density, and multiple form factors. Ciena offers a wide breadth of carrier-grade Ethernet devices in its packet networking portfolio, as shown in Figure 2.
Resiliency
Networks carrying mission-critical information require high tolerance to network disruptions such as fiber cuts or equipment outage. Network resiliency is essential to the ability to support critical command and control applications, data center connectivity, and even desktop applications.
Due to Ethernet’s ability to be deployed in various topologies, it is possible to set up multiple physical paths to specific network elements. This flexibility allows network operators to provide resilient paths in the network.
Traditionally, Ethernet LAN technology used spanning tree to manage failures in a network, often taking up to a minute to reconverge after a failure. This lag is not acceptable in today’s highly demanding networks. Operators are looking for failover rates closer to the TDM gold standard of 50 ms to ensure applications such as VoIP are not impacted. The MEF utilizes standards such as IEEE 802.1Qay Provider Backbone Bridging – Traffic Engineering (PBB-TE), MPLS-TP, and ITU-T G.8032 Ethernet Ring Protection (ERP) to ensure
reconvergence times below 50 ms in the event of failure. These failover times are applicable to any network topology—mesh, ring, or linear—yet this configuration does not preclude using unprotected hub-and-spoke architectures where redundancy is not required.
Bandwidth Scalability and Granular Bandwidth Control
Today’s Ethernet networks provide interfaces from 10 Mb/s to 10GbE and beyond, with emerging 40 and 100GbE interfaces. Because Ethernet hardware enjoys such large-volume deployments, even 10GbE connections are becoming reasonably priced. The change from lower to higher speeds is basically seamless; in many cases, the equipment’s basic configurations support multiple connectivity options. This means that GbE and 10GbE ports exist on the same hardware, the latter of which does not need to be used until it is required. Additionally, it is now possible to upgrade from GbE to 10GbE with the simple addition of a software license. This option is becoming a more popular trend among carrier-grade Ethernet switches.
Ciena’s solution provides unprecedented levels of service classification. Ciena’s service aggregation switches provide up to 64 class-of-service levels, allowing greater flexibility than the typical eight found within competitive offerings. In addition, operators can configure certified MEF-compliant Committed Information Rate (CIR), Excess Information Rate (EIR), and burst parameters.
An example of a Ciena innovation is the use of service templates defining QoS parameters. For instance, a provider’s ‘Silver’ service can be changed easily from 40 Mb/s to 50 Mb/s. Every service configured to Silver is automatically changed, dramatically reducing the number of
configuration/provisioning steps required by the operator.
Service Security
Beyond network firewalls, gateways, and other mechanisms aimed at defending the borders of a network, security also can be enhanced by incorporating protection into the network architecture. The core structures of the Ethernet extensions built into the frameworks by the MEF let the Ethernet standard scale to carrier networks, allowing robust security architectures to be built in the campus environment. Layer 2 VPNs can be used to segregate traffic, limit the potential for contention and congestion to affect critical traffic, and ensure traffic prioritization decisions remain private.
Secure services are classified in the Ethernet standards defined by the MEF by appending VPN tags, which differentiate the VPN service type and priority, to an expanded Ethernet frame. This service can be scaled by consecutively adding layers of VPN tags to create network VPNs that are logically isolated from edge VPNs.
The ability to stack VPN tags was crafted by IEEE 802.1ah to enable both the customer and the network operator to set up VPNs independently, with no possibility of mixing between the domains. This technique, also known as MAC-in-MAC, can be added as a security element in the Ethernet network. By establishing a customer VPN inside a particular security zone or subnetwork and then using PBB in the network, details of the internal topology of the subnetwork can be isolated from other elements of the network.
Ciena also has implemented a patented technology, virtual switching, into its carrier-grade Ethernet portfolio. Virtual switching logically partitions a physical Ethernet switch into separate switching domains; each virtual switch forwards traffic independently of the others, providing a unique mechanism for isolating user traffic that is superior to other available techniques.
Traditional multiple Virtual
Bridging Instance Ciena’s Virtualized Architecture
These core elements of the Ethernet standard, when used in conjunction with Layer 2 encryption, provide powerful tools to enable network operators to build assured network security architectures into their networks.
In addition to PBB-TE, there is a variety of encapsulation mechanisms such as MPLS-TP available to create these VPN tags; in each case, user traffic is separated and there is a clear segregation between end-user traffic and core network traffic. The selection of the encapsulation mechanism depends on a
number of factors, including the need to interact with Layer 3 infrastructure, the extent of the security requirements, and the need for robust segregation in the network.
Operations and Maintenance
A key element of MEF standards development was to build a framework that allows Ethernet networks to operate in manner similar to legacy TDM transport networks, including high-level or OAM capabilities. This includes the ability to monitor,
Model NNI/UNI Ports UNI Ports Total
Gb/s Power Form Factor
318x (2) 100M/Gig SFP (8) 10/100M RJ45 (8 or 16) T1/E1 (2) Front Plug AC /DC 1U 3190 (40) 100M/Gig SFP (2+2) 10Gig SFP+ (16/32) STM-1/OC3 (4/8/32) STM-4/OC12
84 (2) Front Plug AC/DC 3U
3902 (1) 100M/Gig SFP (1) 10/100/1000M RJ45 2 External AC Adpter NID
3911 (2) 100M/Gig SFP (8) 10/100/1000M RJ45 10 AC / DC Option Outdoor
3916 (2) 100M/Gig SFP (2) 100M/Gig SFP + (2) 100M/Gig SFP/RJ45
6 Front Fixed AC (single or dual) Front Fixed DC
1U ETSI (~10” wide)
3920 (4) 100M/Gig SFP (8) 10/100/1000M RJ45 12 Front Fixed AC/ DC 1U ETSI
3930 (2) 1Gig/10G SFP+ (4) 100M/Gig SFP + (4) 100M/Gig SFP/RJ45
28 (2) Front Plug AC /DC 1U ETSI
3931 (2) 1Gig/10G SFP+ (4) 100M/Gig SFP + (4) 10/100/1000 RJ45
28 (2) AC or DC Option Outdoor
3932 (2) 1Gig/10G SFP+ (4) 100/1G SFP/RJ45 (4) 100/1G SFP (16) T1/E1
30 (2) Front Plug AC/DC 1U ETSI
3940 (4) 100M/Gig SFP/RJ45 (20) 100M/Gig SFP/RJ45 24 (2) Rear Plug AC /DC 1U
3960 (2) 10G XFP (2) 10G XFP + (8) 100M/Gig SFP/RJ45
48 (2) Rear Plug AC /DC 1U
5140 (4) 100M/Gig SFP/RJ45 (20) 100M/Gig SFP/RJ45 24 Front Fixed DC 2U ETSI Ext Temp
5142 (4) Gig/10G SFP+ (20) 100/Gig SFP/RJ45 60 Front Plug AC/DC 1U ETSI
5150 (2) Dual 10G XFP Option Slots (48) 100M/Gig SFP 88 (2) Front Plug DC 2U ETSI Ext Temp
5160 (24) Gig SFP/10Gig SFP+ (UNI or NNI)
240 (2) Front Plug AC /DC 1U ETSI Ext Temp
5305 (5) Slots->(10) 10G or (120) Gig N/A 50 (2) Front Plug AC/DC 6U
5410 (10) Slots -> (40) 10G or (320) Gig or (10) 100G
N/A 1000 (2) Rear Fixed DC 22U
diagnose, and centrally manage the network using standards-based, vendor-independent implementations. Ciena has been a pioneer in introducing and pushing relevant OAM protocols in standards bodies.
With the addition of comprehensive OAM capabilities, Ethernet offers a complete feature set that allows network operators to troubleshoot networks to ensure Service Level Agreements (SLAs) are met. IEEE, IETF, ITU-T, and MEF describe mechanisms that allow an operator to poll the status of an end-to-end service to provide detailed metrics. The OAM tools also provide the ability to reactively
troubleshoot the network. Upon discovering a fault or service degradation, operators can rapidly isolate and correct issues, providing higher availability for business-critical services. OAM encompasses the ability to rapidly turn up services and devices. Ciena’s solution provides a truly automated provisioning capability. All devices shipped have the intelligence by default to request specific software and configuration templates from the NOC. Templates are extensible and incredibly powerful, allowing many customizable parameters to be pushed down when the device is turned on.
Ciena Packet Networking Portfolio
Ciena offers a wide breadth of carrier-grade Ethernet devices in its packet networking portfolio, shown in the table below. These solutions are particularly robust in the government campus and metro LAN environment, meeting the requirements of most enterprises. Included are options for high-density, low-footprint switches, high port count (48 Ethernet ports or more) for high-density fan-out, and small, single port desktop devices. A variety of chassis configurations, powering options and physical interfaces are available, including multi-speed copper RJ45 or fiber-based options, which can be changed with the simple addition of an SFP module.
Conclusion
Miami-Dade County required a modernized network infrastructure to support its growing population and blend of changing government services. Operating independent IT networks would not allow for rapid growth with affordable economics. In deploying a converged optical Ethernet network architecture utilizing Ciena’s Packet Networking technology, the county government was able to improve operations across several functional areas while also improving operating efficiency and public service. Further, their infrastructure is well suited to grow and change as their application mix continues to evolve.
About the Customer
Miami-Dade County encompasses more than 2,000 square miles, with a population exceeding 2.5 million people. Located along the southeast tip of the Florida peninsula, one-third of Miami-Dade County is located in Everglades National Park. With a work force of more than 25,000 employees dedicated to providing the best services possible, the county has made it more convenient for residents to obtain service anytime via its Web portal (www.miamidade.gov). Citizens can make tax payments, search maps, report a problem, request new traffic signs, schedule solid waste collections, track building plans, and even check flight information at Miami International Airport—all online.
Ciena may from time to time make changes to the products or specifications contained herein without notice. Copyright © 2014 Ciena® Corporation. All rights reserved. CS075 1.2014
