A Network Design Primer

(1)

K-20 Network Engineering 6/30/15 K-20 Constituent Network Design Recommendation

K-20 Constituent

Network Design

Recommendations

Recommendations for K-20 Constituents to take into

account when doing network design to help create a more

(2)

Chris Johnson

K-20 Network Engineering 1 K-20 Constituent DDoS Protection

Contents

Network Goals ... 2

Common Components ... 2

Recommendations to protect and defend and analyze ... 2

Example Network Designs ... 5

Documentation ... 8

Network Diagram ... 8

Equipment Information Repository ... 8

Network Run Book ... 8

Communications Plans ... 8

Conclusion ... 9

For more information ... 9

Resources ... 9

Figure 1: Simple Network Design ... 5

Figure 2: Multiple ISP Network Design ... 6

(3)

Chris Johnson

Network Goals

In today’s online world, services vital to organizations performing their day to day operations – web presence, VoIP, Real Time Collaboration, outsourced payroll, Telemedicine, standardized tests – are becoming more dependent upon network communications. Outages of many sorts – looped ports, overloaded firewalls, saturated links, DDoS attacks – can bring an organization to a screeching halt, as such specific design strategies should be deployed to enhance the survivability of services. This document aims to help create a more fully functional network which has the specific goals.

 Dependable  Scalable  Defendable  Stable  Manageable

Common Components

A network which is designed with the following components should be able to provide the functionality needed while keeping aligned with the aforementioned goals.

 Routers

 Switches

 VLANS

 Private network space

 Public network space

 Subnets

 Access Control Lists

 Demilitarized Zone (DMZ)

 Firewalls

 Network Address Translation (NAT)

 Log Collector(s)

 Traffic Policy Devices

The proper implementation and use of these components will help to create a highly scalable, dependable, defendable and stable network that can help to simplify analyzing, managing and mitigating outages.

Recommendations to protect and defend and analyze

The following are a list of recommendations that organizations can institute to help defend

themselves against becoming a victim of an attack, participant in an attack on another organization, or to help identify the flows involved in an attack.

Hardware:

 For router(s), use a router which is able to easily process all the data flows expected for not only normal operations, but to include operations when in a failed or attacked state

(4)

Chris Johnson

 For router(s), use a router which is able to provide some robust Access Control List (ACL) capabilities

 For stateful and stateless firewall(s), use a firewall which will have enough processing power to handle all data flowing through it and will should not buckle under extreme load

 For stateful firewall(s), use a firewall which is able to inspect all traffic and proxy all necessary protocols which may need translations help such as SIP, H.323, etc…

 Use redundant network infrastructure configurations where possible

 Use managed switches

 Use switches which are compatible with your chosen Layer 2 loop elimination protocols

 Use switches which are able to perform storm control protocols

 Configure all hardware to export their logs to an external log collection server, in addition to on-device log stores

All network segments:

 Where possible, try to create non-loopable Layer 2 (L2) segments

 Where L2 loop elimination is not possible, use L2 loop elimination technologies (STP, PVST, RSTP, TRILL, etc…)

 Use broadcast storm control technologies (BPDU guard, Stormcontrol, etc…)

 Monitor network segment for network performance metrics (throughput, errors, levels, etc…)

 Log performance statistics violations of network segment to logging servers (allowable bandwidth exceeded,

 Utilize smart log analyzers to “detect” suspicious activities (Fluentd, Sagan, Splunk, etc…)

 Turn off unnecessary services (Chargen, Telnet, etc…)

 Set up easily modifiable rate limiting infrastructure Network Infrastructure Links

 Segregate network infrastructure Links to dedicated ports (not shared with other subnets)

 Use authentication schemes in accordance with dynamic routing protocols (OSPF, BGP, etc…)

 Mirror traffic to deep packet inspection engine (SNORT, Suricata, etc…)

 Log data flows to an analysis server (NTOP, nProbe, cflowd, StealthWatch, plixer, etc…)

 Monitor link utilization for historical, trending and real time statistics User Subnets

 Segregate user subnets to their own VLANS

 NAT user subnets with a many:1 private:public outbound

 Know and understand the type of traffic that should be traversing the user subnet

 Don’t allow inbound connections from the internet to user subnets

 Utilize host firewall and anti-virus on all hosts in user subnet

 Utilize a firewall or Access Control Lists (ACL) to prevent inbound connections on the user subnets

 Use a stateful inspection firewall to protect OSI layers 4-7 from more complex attacks Private Server Subnets

(5)

Chris Johnson

 Segregate private server subnets to their own VLANS

 Possibly NAT private server subnets with a many:1 private:public outbound

 Utilize host firewall and anti-virus on all servers in the private server subnet

 Filter access to services which should not be accessed remotely (RDP, fileserver, etc…)

 Utilize a firewall or ACL on the private server subnet to prevent any access into the private subnet from the internet

 Utilize a firewall or ACL on the private server subnets to prevent access to anything but the authorized services

Public Server Subnets

 Segregate public server subnets to their own VLANS

 NAT server subnets with a 1:1 private:public space

 For stateful firewalls, utilize a firewall which is able to inspect all traffic and proxy all necessary protocols which may need translations help such as SIP, H.323, etc…

 Know and understand the services running on the servers in the server subnets

 Utilize host firewall and anti-virus on all servers in the public server subnet

 Filter access to services which should not be accessed remotely (RDP, fileserver, etc…) VoIP Subnets

 Segregate VoIP communications infrastructure to their own VLANS

 Know and understand the services running on the servers in the VoIP subnets

 NAT VoIP subnets with a Many:1 private:public and only allow outbound connections

 Utilize a host firewall on all servers on VoIP subnets to only allow traffic to specific services DMZ Subnets

 Relocate high-target resources to a DMZ, if service off-site is not possible

 Utilize a host firewall on all DMZ servers to only allow traffic to specific services

 Filter access to services which should not be accessed remotely (RDP, fileserver, etc…) Offsite Subnets

 Off-site high-target resources (DNS, CRM, Web, Mail, etc…) to alternate service locations (colocation, SaaS, etc…)

 Use firewall and anti-virus as applicable on offsite services (SaaS, DaaS, IaaS, etc…) Monitoring

(6)

Chris Johnson

 Utilize smart log analyzers to “detect” suspicious activities (Fluentd, Sagan, Splunk, etc…)

 Utilize DDoS mitigation services to detect and clean dirty traffic

 Use IPS / IDS to prevent and detect intrusions

Example Network Designs

Below are some example network designs which incorporate a number of the proposed

recommendations above to help create a more easily managed and defendable network which can lead to lessened impact by negative network events and ensure service availability. These designs are intended to be more logical designs with an understanding that there may be many instances of

particular portions of the designs within an organization. While each of these designs focus on specific aspects of network design, parts of them can be combined and merged with each other to provide the appropriate network design that best fits the needs of the organization.

FIGURE 1:SIMPLE NETWORK DESIGN

Private Server Subnet With X:1 NAT On firewall or router Public and Private Network Space firewall router private switch or VLAN dmz switch or VLAN private switch or VLAN web server file server DNS Server log collector monitor server Internet Provider laptops PC Private User Subnet With X:1 NAT On firewall or router collaboration server mail server

Public Subnet Via 1:1 NAT or Public addresses. Should have Firewall or ACLs on the Router limiting access to

only the available services Servers should be

running local firewalls

As can be seen in the above diagram, the users and servers are segmented onto different VLANs to prevent local broadcast problems interfering with each other and taking out the entire network. The use of Layer 2 protocols such as spanning tree (STP, RSTP, PVSTP, etc…) and broadcast storm

mitigation protocols (BPDU Guard, Storm Control, etc…) helps to prevent a looped port or an errantly broadcasting device from taking out all user and server subnets. Some services which may be high bandwidth or may not interact well with firewalling technologies are configured on a DMZ connected to the router and are able to be protected with local firewalls and ACLs or firewall filters on the router. The use of dynamic routing protocols such as OSPF may or may not be utilized in this

(7)

Chris Johnson

situation depending on the complexity and diameter of the network, for example if there were multiple routers or multiple firewalls servicing all the network segments. The use of dynamic protocols helps to ensure that as new segments are added or new routers are added that static routes do not have to be updated. Per user, subnet, or service rate limits may be placed on the firewall or router to ensure that there are bounds on an errant devices or that may be trying to consume all the network resources. The router and firewall in this scenario should be logging

information about equipment performance, observed network events, user data flows, and any other information that is pertinent to the organization to the log collectors and monitoring servers so that real time stats and historical information can be reviewed for forecasting and forensic investigation. This option is a fairly common option for a medium sized organization which has a single internet connection and cannot afford to offsite services or purchase additional internet connections. FIGURE 2:MULTIPLE ISPNETWORK DESIGN

Private Server Subnet With X:1 NAT Public Network firewall router private switch or VLAN dmz switch or VLAN private switch or VLAN web server file server collaboration server log collector Monitor server Internet Provider laptops PC Private User Subnet With X:1 NAT

router _{Internet Provider}

Public Subnet Via 1:1 NAT or Public addresses. Should have Firewall or ACLs on the Router limiting access to

only the available services Servers should be

running local firewalls

Phones private switch or VLAN Voice server Private Voice Subnet With X:1 NAT And only outbound

Connections From Voice server

allowed

While still implementing the monitoring and Layer 2 and Layer 3 protocols as referenced in the network design above, the above diagram has segregated services which may have been high likelihood targets or provide critical services to their own internet connectivity. This may be a separate connection coming into the organizations existing data center and utilizing VLANs and a

(8)

Chris Johnson

separate router, or this may be a case where the organization has collocated the services to an external vendor. This prevents attacks to those high target servers from impacting the day to day operations of the users and local servers required for their day to day operations. It also provides a boundary so that issues which may arise within the local network are less likely to be able to

negatively impact services which may be critical for business functions or brand identity. Also of note: in this scenario, the organization has deployed Voice over IP (VoIP) services, and have segregated those services onto their own VLAN to ensure that voice services are as protected as possible. This type of network design is quite often used by organizations which may have some of its web presence hosted externally.

FIGURE 3:SAASNETWORK DESIGN

Private Server Subnet With X:1 NAT Public Network firewall router private switch or VLAN switch private switch or VLAN web server file server DNS

log collector Internet Provider

laptops PC Private User Subnet With X:1 NAT SaaS Provider Servers should be running local firewalls

As is the case with the previously mentioned network design, the design above has collocated some of its services to third party SaaS providers. This affords the organization the ability to have an external company manage and maintain the security policies and infrastructure for critical systems without taking on the complexity and cost of maintaining the appropriate security infrastructure. One other feature that is often offered by the SaaS providers is guarantees of uptime, specifically that 100% uptimes are guaranteed for critical services and brand identity. The cost of the SaaS services vary widely based upon throughput, number of servers required, levels of response, services being protected, etc… While this design helps to ensure that services are always available, that does come at additional cost.

(9)

Chris Johnson

Documentation

One of the most important things about managing a network is making sure that it is documented properly. Documentation of the network consists of a number different information stores, but a few types that are essential to ensure that all individuals are working with the same understanding and goals are listed below.

Network Diagram

All networks should have a detailed network diagram. Included above are some generic network diagrams, but the network diagrams that are generated for an organizations network should contain information about what servers and services are located on which systems and how they all

interconnect. This should be in an easy to read pictorial format. This network diagram may be broken up into a number of smaller more detailed documents for clarity’s sake, but there should be an overarching diagram which should reference detailed documents. There are a number of

examples of what these diagrams may look at here in the link in the resources at the end of this document.

Equipment Information Repository

As part of the documentation for the network, there should be an equipment information repository. This information should be in an easy to review format and should be centrally located so all

individuals of the organization know what equipment is active and the functions that equipment performs. This should also contain information such as warranty and support information. This information is operational information that may be referenced by the Network Run Book for appropriately responding to an incident within the network.

Network Run Book

Another key document to ensure proper network management and stability is the Network Run Book. This book defines all the processes that are utilized to manage and maintain the activities associated with the network. This includes functions such as adding in new segments, new servers, new routers, new routes, new customers, new VPN connections, etc… This book should be the authoritative guide for how to execute and react to incidents. This document should reference well defined

communication plans, if there are to be communications about activities governed by this document. In addition to the Communications Plans for each of the activities in the Network Run Book, there should be a well-defined Roles and Responsibilities matrix developed. This should contain

information about who is Responsible, Accountable, Contributing and Informed (RACI) about the procedures in the Network Run Book.

Communications Plans

Communications plans are vital for disseminating information about the going-on in a network. A well-defined and executed communication plan should be developed to ensure that when there are network issues that information can be broadcast and notifications can occur in a timely manner. The organization should have clearly defined roles and methodologies of communications, bearing in mind that normal means of communications may be hindered because of network unavailability, therefore non-traditional forms of communications, or ‘old school’ forms may be more reliable.

(10)

Chris Johnson

Conclusion

While this document is intended to present a number of recommendations for how to architect and implement a network, it is by no means authoritative in all circumstances, nor does it address every situation that may arise. The architecture and management of a network is an ever evolving process that changes with technologies, people, organizational needs, and any other number of factors. The main thing to be cognizant of is that the network should be constructed to be scalable, flexible, defendable, and manageable.

For more information

If you have more questions, comments or requests about the materials covered in this document or for additional documents, please contact the your organizations K-20 liaison, your sector

representative, or the K-20 Program Office. If the matter is regarding an operational issue, please contact the K-20 NOC at [email protected] or +1 (888) 934-5551.

Resources

Below are a list of resources which go into great detail regarding various aspects of the technologies mentioned in this document. These links are by no means the authoritative repository, they are meant to be a stepping off place for further research and evaluation.

Here are some resources for some of the lower level components that should be utilized, these are very basic building blocks to help build up more complex networks.

Router http://www.webopedia.com/TERM/R/router.html

Switch http://www.webopedia.com/TERM/S/switch.html

Firewall http://www.webopedia.com/TERM/F/firewall.html

VLANs http://www.cse.wustl.edu/~jain/cis788-97/ftp/virtual_lans/ http://serverfault.com/questions/188350/how-do-vlans-work OSI Model http://www.webopedia.com/quick_ref/OSI_Layers.asp

Access Control Lists (ACL) https://en.wikipedia.org/wiki/Access_control_list

Network Diagram Examples http://creately.com/diagram-community/popular/t/network-diagram

Network Run Book http://www.webopedia.com/TERM/R/run_book.html https://en.wikipedia.org/wiki/Runbook

Network Run Book Example

https://contursiconsulting.com/documents/XYZ_Directory_Runook.doc

Below are some articles which go into comparisons of various logging and analysis tools with their strengths, weaknesses and caveats to them:

Flow collection tools http://netflow.caligare.com/applications.htm

Log Analyzer tools http://www.predictiveanalyticstoday.com/list-security-event-management-log-analysis-software/