TTM4240: Advanced Network Control and Management
This compendium is an effort to summarize the contents of the course TTM4240 Advanced network control and administration. The compendium does not make any guarantees to be correct or to include everything needed for the exam, so if you find any mistakes or anything that should be included, please edit the page to fix or add them.
Chapters are ordered by how they were presented during the fall 2021 course.
Network device internals
This course mostly focuses on three different network devices:
- Routers (core devices for the internet)
- Switches (transparently allows more hosts/routers on a single interface)
- Hosts (devices that connect to a router, like a PC or a server)
Inside a router
We usually divide the inside of a router into four logical parts:
- Input ports: Receives data, consults the lookup table, and forwards the data to the correct output port (see next subsection for details). Input ports usually have a queue, in case the correct output port is busy.
- Switching fabric: Connects input ports and output ports. Can be viewed as a small network inside the router.
- Output ports: Sends out data. In most cases, ports are bidirectional, so all physical ports are both input and output ports.
- Routing processor: Performs the intelligence of the router, such as running routing protocols, building the lookup and forwarding table etc.
The lookup table of the router is used by input ports to see where an incoming packet should be forwarded. The table usually contains entries with a triplet of (address, mask, interface). The address specifies a prefix (what the destination address should start with), a mask (how long the prefix is), and the interface/output port the packet should be forwarded to on a match. On non-core network routers, a default entry is usually added for packets that don't match any other rule. Packets are forwarded using a longest-prefix match, which is based on the most specific table entry (longest mask).
A router has the following lookup table:
Notice that the last line adds a default route for packets with no other destination. It's often easier to see how the table works if the addresses are converted to binary:
A zero in the mask specifies that that digit of the IP address is ignored. A mask always takes the form of a string of 1-s, followed by a string of 0-s. A more compact way of writing masks is therefore just to write the length of the string of 1-s directly. For example, instead of writing "Address:
255.255.255.0", the same info can be written as
192.168.1.0/24. This notation is referred to as CIDR notation.
If a packet with destination
192.168.1.3 arrives at this router, it should be forwarded to eth2, as this address has been added to the table directly. It is not forwarded to eth1, since the first row of the forwarding table has a longer prefix (32 bits instead of 24 bits).
If a second packet arrives with the address
192.168.1.6, it will be forwarded to eth1.
If a packet with the destination address
172.16.128.200 arrives at the router, it will be forwarded to eth0. Notice that the mask for the third row of the table is 25 bits long, not 24. With the specifies prefix, this gives the range of addresses from
172.16.128.127. The default route is therefore chosen, even if there was almost a match.
Content-Addressable Memory (CAM)
The lookup process in the previous subsection needs to be performed for every incoming packets. In today's networks, with potentially multiple terabits of data per second for a single ISP-grade router, this needs to happen very quickly. A software implementation is therefore not fast enough. Instead, the lookup process is implemented in hardware, using the Content-Addressable Memory technique.
CAM uses dedicated hardware for the search and comparison between the lookup table and destination addresses of incoming packets. This gives a time complexity of O(1), meaning that the correct search result will be found in constant time. The downside of CAM is that it requires dedicated hardware for the patterns (but the patterns themselves can be configured using software). This increases the size, cost, and power usage of the router.
Addressing and planning
An IPv4 address (what we usually refer to as just IP address) has a length of 32 bits. With each bit having two different states, this gives
Traditionally, IPv4 addresses have been classified into three different classes based on subnet mask length.
- Class A: 8 bit subnet mask. 16777216 addresses. Handed out to large ISP's.
- Class B: 16 bit subnet mask. 65536 addresses. Handed out to small ISP's and large corporations.
- Class C: 24 bit subnet mask. 256 addresses. Shared among users.
However, as networks have evolved, other subnet mask lengths have been used. For example, a customer with 600 routers make better use of a 22-bit (1024 addresses) subnet mask than a class B subnet.
Some IP ranges have been reserved as private. These are never used as external addresses, so they can be used in internal networks. These internal networks can be connected to the open internet using a single shared address by utilizing NAT (see next subsection). The following ranges are reserved for internal use:
Other reserved ranges
0.0.0.0/8is reserved to refer to "current network"
127.0.0.0/8is reserved for loopback addresses (where
127.0.0.1is the most widely used)
184.108.40.206/4(class D addresses) is reserved for multicast (not part of this course)
255.255.255.255) are reserved for future use and research. They are known as class E addresses.
Network Address Translation (NAT)
To save on IPv4 addresses, internet users are usually allocated only a single external IP address for their household. To be able to connect multiple devices to the internet with only one address, NAT is used. When NAT is used, the router replaces source and destination addresses (translates) of network packages between internal and external addresses. To remember which packets should be sent to which device on the local network, TCP and UDP ports are used. This violates the principle that the router should only work on layer 3 of the TCP/IP protocol stack, but has been a requirement for the internet to scale to the size it is today.
IPv6 is the latest version of the internet protocol. The main difference is the increased address size: from 32 to 128 bits. This gives a total of
Rollout of IPv6 is increasing, but large parts of internet traffic is still IPv4. To remedy this, several tunneling techniques can be used to send IPv6 packets over an IPv4 connection. This usually happens by just wrapping the IPv6 packet inside of a larger IPv4 packet. The most well known technology used today is NAT64, which can be used in order to tunnel ipv4 traffic over ipv6 networks embeding ipv4 addresses. The downsite of NAT64 is that some protocols embed literal ipv4 addresses and therefor can't be used with NAT64. Examples of protocols are FTP, SDP and Websockets.
To make a network manageable as more routers and hosts are added, proper planning of both topology and addressing is required. Creating an address plan consists of defining rules for which IP addresses should be used for what. For example, deciding what addresses should be used for internal links, loopbacks, external links, and customers (for an ISP).
Interior Gateway Protocols (IGPs)
Interior gateway protocols are protocols that allow routers within an AS (see next subsection) to discover what else is connected within the AS. For a small AS, this could maybe be configured manually, but this does not scale well, and is more prone to human error.
Definition of an AS
RFC1930 defines an Autonomous System (AS) as:
An AS is a connected group of one or more IP prefixes run by one or more network operators which has a SINGLE and CLEARLY DEFINED routing policy.
In practice, when we talk about an AS, we refer to an ISP or company network with several routers, where a single entity (the owner) has control over the routing and addressing plan. A single AS might use multiple IGPs within its own AS, and probably communicates with other AS's using an EGP (which is almost always BGP).
Routing Information Protocol (RIP)
RIP was one of the first attempts at an interior gateway protocol, and does show signs of old age in terms of scalability and features. RIP is a distance vector protocol, with hop count (amount of routers a packet visits on its path) as metric. Hop count does not consider link capacity, so a single 1 Mbps link would have been preferred over two 10 Gbps links in the eyes of RIP.
RIP (by default) works by broadcasting its distance vector every 30 seconds. In a large network, this creates large traffic bursts twice every minute. In addition, hop counts are limited to a maximum of 15 hops, which makes it impossible to use RIP in large networks. However, of the widely used IGPs, RIP has the least requirements for configuration, and is easy to get running.
RIP does not support IPv6. An extension of RIP, known as RIPng, was created to support IPv6 networks.
Open Shortest Path First (OSPF)
OSPF is a protocol for use within an AS to let routers and devices discover each other, and maintain routes when routers go down. It is widely used in large enterprise networks, while IS-IS is more common in ISP networks. OSPF runs directly on top of IP, without using TCP or UDP for transport. Support for IPv6 in OSPF required a new version of the protocol, OSPFv3.
OSPF is a "link-state" based protocol. This means that instead of transmitting known distances to other routers (like a "distance vector" protocol such as RIP), only the status of links is transmitted, and each router calculates distances and paths to other routers.
OSPF has to be configured on each router, but when it has been enabled on all participating routers, it operates without human interaction. Each router is configured to belong to an area, which is used to create some separation within the AS. OSPF requires that the area with id 0 is the backbone area. All other areas should have some direct or indirect connection to the backbone area, creating a "core" or "center" of the AS.
While RIP uses only hop count as a metric, OSPF calculates a cost of routes. This cost is usually related to link bandwidth, but can be configured by the operator. This makes it easier to prioritize links, and creates more scalable paths through the network.
OSPF classifies routers into four categories:
- Internal routers (IR): A router that exists within an OSPF area, without direct connections to anything outside the area.
- Area border router (ABR): A router that is located at the border of an OSPF area, and has interfaces to more than one OSPF area.
- Backbone router (BR): A router that is located in area 0. if it has connections to other areas as well, it is both a BR and an ABR.
- AS Border Router (ASBR): A router that has interfaces both within the AS, and to other AS's. Usually runs OSPF towards the AS, and BGP to external routers.
The hello message is the most widely transmitted message type in OSPF. On startup of the OSPF process, hello messages are used by neighboring routers to discover each other. They are broadcasted to all connected interfaces where OSPF is enabled, and a reply is counted as a valid neighbor. When the protocol is running, hello messages are used to detect link failures: if a router does not receive a hello message from another router after a dead interval amount of time, the link is considered down.
When a connection between two routers has been established using hello messages, a database exchange is performed. At this stage, both routers in the new connection transmit their own routing database in the form of one or more DB Description packet(s). Each DB Description packet contains one or multiple LSA's, so after the exchange is complete, the database is synchronized between the routers.
To allow OSPF networks to scale, a Designated Router (DR) and Backup Designated Router (BDR) is used to reduce load. The routers are selected according to some simple rules:
- If there is no DR, make the BDR into the DR.
- If there is no BDR, hold an election to select a new BDR (has to be different from the DR).
The election is based on DR priority value set by the network administrator. The router with the highest priority will be chosen. If multiple routers have the same priority, the one with the lowest OSPF ID (in IPv4 networks this is usually a loopback IP address, but could be any integer) is chosen.
A consequence of these simple rules is that when a new connection is established, two elections are held right after each other (first select a BDR, make the BDR the DR, before finally selecting another BDR).
The role of the designated router is to receive LSA's from other directly connected routers, and distribute them among other routers on the same interface (such as multiple routers connected on the same switch). This has the potential on reducing bandwidth usage of OSPF, but places more load on the DR.
IS-IS (Intermediate System - Intermediate System) is an alternative to OSPF. Both protocols are link-state protocols, and work in a similar way. Unlike OSPF, IS-IS does not run over IP, but rather runs directly on layer 3 of the protocol stack. IS-IS does not make any assumptions about the underlying network, and therefore only required a small extension to support IPv6.
Exterior Gateway Protocols (EGPs) / BGP
While IGPs like OSPF and IS-IS is used for routing within an AS, EGP's are used for routing traffic between AS's. In practice, the only EGP in use is the Border Gateway Protocol (BGP). BGP is manually configured on every router that runs BGP, and runs as a TCP session on port 179. Two routers with a direct BGP connection is said to be neighboring BGP peers. BGP can either run between routers in different AS's (called External BGP, eBGP), or between two routers in the same AS (internal BGP, iBGP).
Usually, all BGP-speaking routers in an AS run both eBGP towards some other AS, and iBGP towards every other BGP router within the local AS. This requires (virtual) links between all BGP routers within an AS, giving lots of connections for a large network. One way to reduce the load, is to use a route reflector, which effectively turns the internal iBGP mesh network into a star topology, with the reflector in the middle.
BGP is a path-vector routing protocol, a bit similar to RIP. But unlike RIP, paths in BGP are very dependent on the network operator's wishes. We often say that BGP is a policy-based routing protocol, where policies could be reasons outside the network, such as economics. BGP routers are identified using their AS number.
The path vectors only include reachability information, not routing information.
The difference is that instead of advertising which hosts and links exist, like in an IGP, only reachable address ranges are advertised. The ranges included in the path vector also includes which AS numbers need to be traversed to reach said range. For example, NORDUnet (AS2603) may advertise that they can reach the IP range
220.127.116.11/22 by routing traffic through itself, Uninett (AS224), and Studentersamfundet (AS58302). Other routers would then know that to reach the IP range belonging to Studentersamfundet, the traffic would have to pass through two more AS's on their way after being sent to NORDUnet.
Network management fundamentals
Definition used in this course:
Network management includes the deployment, integration and coordination of the hardware, software, and human elements to monitor, test, poll, configure, analyze, evaluate, and control the network and element resources to meet the real-time, operational performance, and Quality of Service requirements at a reasonable cost.
In short, network management refers to the acts of configuring and monitoring network devices.
One of the challenges in network management, is that devices from different manufacturers usually use different software and administration systems. In order to make it possible to efficiently manage a network with devices from different manufacturers, common protocols and standards are developed. For devices that don't support standardized protocols, it is also possible to place an agent between the manager and the managed device to "translate" between general protocols and specific protocols.
Network managing models
We usually look at four different network management models:
- The organizational model defines the components of a managed network system. It defines objects (managed devices), agents (receive commands from the manager and use them to configure the device. A SNMP server is an agent), and managers (sends commands to the agents).
- The information model describes which data is managed. For example, the structure of the MIB (see below).
- The communication model defines how commands and data is transmitted over the network.
- The functional model defines five application areas within network management: configuration, fault, performance, security, and accounting.
Command line interfaces
A common way to manage a network device, is to use a command line interface (CLI). These are often very specific to the device, with little standardization. It is possible to automate CLI usage using scripts, but these are prone to break as the CLI is changed. A CLI is also very vulnerable to human error when humans type in commands. However, a CLI is often flexible, and easier to extend than a protocol.
Throughout the rest of this compendium, the term OID will be used to refer to an "Object Identifier". This is a standardized string of digits that is used to refer to some "object" or "thing". OID's are structured in a tree, where each number refers to the index of the child node to traverse down.
Most OID's in this course starts with
18.104.22.168, which refer to the internet.
Managed Information Base (MIB)
The MIB is a database within the router that stores information in the router. It is structured as a tree structure, based on OID. We usually interact with the MIB using SNMP. It is part of the information model.
Genericity vs Transparency
When designing some management API, a choice of transparency and generality has to be made.
Transparency describe how precise parameters are defined (higher transparency
|High genericity||Low genericity|
High transparency is more flexible with what to request, but requires more difficult parsing, and more documentation over supported features.
Simple Network Management Protocol (SNMP)
Simple Network Management Protocol (SNMP) is, as the name implies, a protocol used for managing network devices. It allows getting and setting various properties from the routers MIB based on OID's. SNMP is an old, widely used protocol, and is supported on many devices from many different vendors. However, what features can be retrieved and modified over SNMP varies greatly from device to device. For example, many devices only allow getting properties from the MIB, not setting anything. This makes SNMP mostly suited for monitoring.
SNMP runs over UDP, which causes the data transfer to be less reliable than it could have been. For example, if a manager does not receive any reply to their SNMP request, it is not possible to know if this was because the link is down, the SNMP agent (i.e. the SNMP server) was not running, or the community string (see next paragraph) was incorrect. In addition, transmitting large amounts of data is difficult, as any part of the transmission could fail.
SNMP does not perform much authentication. The only form of authentication used, is the community string. The community string can be set on the managed device, and it will only respond to SNMP requests that use the said community string. It can therefore be seen as a password.
SNMP message types
SNMP lives up to the "simple" in its name by only having four different messages. Of these, only the
trap message is sent from the SNMP agent to the manager unprovoked.
getmessage is used by the manager to retrieve a piece of information from a SNMP agent, using an OID. A
get-responseis sent as a reply by the agent with the requested information.
get-nextmessage is used by the manager when retrieving data that does not fit in a single UDP packet, the
get-nextrequest can be sent to receive the next part of the data.
setmessage is used by the manager to set a value in the MIB. Its parameters are an OID and the new value.
trapmessage is sent from an agent to the manager on certain conditions set by the manager. For example, it is possible to configure traps to be sent when an event occurs.
Network monitoring and measurements
Network monitoring, as the name implies, is the act of monitoring a network. Monitoring can involve everything from just monitoring resource usage, performing intrusion detection, or performing advanced analysis on obtained data. First, let's introduce the differences between monitoring and measuring:
- Measuring refers to obtaining measurements of some data. For example checking CPU usage on a router, or the average length of packets in a TCP session.
- Monitoring refers to the whole process of monitoring a network, which usually includes many measurements.
When doing network monitoring, a lot of questions needs to be answered, and careful planning is required. For example:
- What should we monitor?
- How do we measure these values?
- Where do we measure?
- How often do we perform these measurements?
- To which granularity should the measurements be performed?
- Does our monitoring violate user privacy?
We often distinguish between two types of measuring:
- Passive measurements: Passively analyze existing network traffic. Can be done using a dedicated device, or directly on a host, router (running protocols like SNMP and NetFlow), or other existing device on the network.
- Active measurements: Generate new traffic for performing measurements. An example is using the
pingtool for obtaining round-trip time, which injects new traffic in the network. If done on a large scale, this new traffic may affect the network, giving less precise measurements for a normal scenario.
Types of traffic analysis
When measurements are performed on a copy of the entire traffic in a link, there are several ways to analyze the traffic to obtain measurements. This can be chosen based on available resources (e.g. header-only analysis is much lighter than full packet monitoring).
When the full packets are analyzed, it is possible to obtain very detailed data about user activity. This is required for some types of monitoring, like intrusion detection, but has a high computational cost in high-bandwidth networks.
A different alternative is to just look at packet headers. With the header, it is possible to obtain information about the source and destination of the packet, size, and protocol. Using only headers is also enough to perform flow monitoring (see below).
Flow monitoring and NetFlow
When monitoring network traffic, it may be difficult to tell traffic patterns from packet data alone. One alternative is to aggregate packages into "flows". A "flow" can be seen as a session between two hosts, such as a TCP session or two-way UDP traffic with the same source and destination address/port. Many routers support aggregating flows, for example with NetFlow.
Netconf / Yang
As mentioned earlier, the Simple Network Management Protocol (SNMP) is mostly used for monitoring, not for configuration. Several other attempts have been made at creating a standardized protocol and language for configuring network devices. One of them is Netconf, with the included configuration modeling language YANG (short for Yet Another Next Generation data modeling language).
Netconf devices have to support running Netconf over SSH, but may also add support for other transport protocols like TLS. What these transport protocols have in common, is that they are secure and reliable, solving one of the problems of using SNMP for configuration.
Netconf assumes that the device's configuration can be represented as an object, or a document. Since this object can be quite large for a complex network setup, all Netconf methods support various filtering to only fetch or set parts of the configuration. Some important Netconf operations are:
get-config: Gets the current configuration, with an optional filter.
get-config, but also returns state information in addition to configuration.
edit-config: Edits some part of the current configuration.
copy-config: Copy some configuration from a source to a destination, for example to copy the current configuration over to startup configuration to preserve it after reboot.
Multiprotocol Label Switching (MPLS)
When routing traffic within a large network, checking the destination address of the packet and performing a lookup in the forwarding table may be a waste of resources. This is particularly a problem in core networks without default routes, where the lookup table might contain hundreds of thousands of entries. To prevent unnecessary lookups, MPLS can be used. In a MPLS-enabled network, routers at the edge of the network can find the correct path for the packet, and attach a "label" to it. When intermediate routers check the packet, they can look at only the short label on the packet to determine the next hop on the route.
MPLS works in a layered fashion, and a packet may receive multiple labels on its way to the destination. This works as a stack, where labels are pushed on top of the stack, and popped from the top when removed.
MPLS is not only used to prevent unnecessary lookups in the full flow table. Labels can also be used to create virtual links, for example to create an IP-level VPN connection, or to give specific instructions and QOS to different types of traffic.
MPLS reduces the requirement for routers to know the addresses of other routers, but introduces a new problem: How do we know what the different labels mean? For this, we use a Label Distribution Protocol (LDP).Some LDP's are extensions of existing protocols (for example MPLS-BGP uses BGP to distribute labels), while some are new for this specific purpose. One of the most common LDP's is MPLS-LDP, which is confusingly enough often just called LDP.
Label distribution is often only done within a single MPLS domain, such as within the backbone network of an AS. However, it is possible for two different core network to agree to exchange labels with each other, in order to improve performance for traffic that passes through both.
Software Defined Networking
Software Defined Networking (SDN) is a newer trend in networking. The main idea is to separate the control plane and data plane of the network. Traditionally, these have both been done in routers, where control has happened in the routing processor. With SDN, this is instead performed using an external controller. This makes the network more flexible, as it can be seen as a single programmable entity. It is also possible to more closely integrate the network with specific applications.
When talking about SDN, we often talk about four different interfaces.
- The southbound interface is the defining interface of SDN. This is the interface between software-controlled switches, and the controller. On this interface, a standardized protocol like OpenFlow is used.
- The northbound interface is the interface between the SDN controller and an application. Requirements for different applications differ by a lot, so this interface is not standardized, and such standardization would probably lead to more restrictions on what SDN could be used for.
- The westbound interface is the interface between SDN controllers belonging to different networks. This could be used to exchange information, such as monitoring results and routing information. For normal routing information, standardized protocols like BGP can be used.
- The eastbound interface is the interface between an SDN controller and a non-SDN network. For this interface, some kind of translation is needed, and it depends on the type of legacy network. For some networks, speaking normal BGP is enough.
Advantages of SDN
The main advantages and features about SDN are:
- Programmability: This is the key feature of SDN. The whole network can be programmed even after physical deployment without much hassle.
- Protocol independence: Since e.g. forwarding rules are entirely programmable, a SDN network is easy to adapt to new protocols. For example, upgrading from IPv4 to a dual IPv4 and IPv6 stack is possible by just updating the controller.
- Ability to dynamically alter network parameters: This is closely linked with programmability. The network can be altered dynamically, for example to respond better to specific types of traffic.
- Granularity: A SDN network can be controlled at different granularity. From just mapping input interfaces to output interfaces, to working on a per-packet or flow basis. This gives the network operator great flexibility in how the network is programmed.
- Elasticity: Since the controller runs entirely in software, it is possible to scale it up and down as a response to traffic changes. In a traditional network, this would probably have required to replace the routers, which is time-consuming, expensive, and (most likely) introduces downtime.
Internet design philosophy
The top level goal when the internet was designed initially, was to allow efficient multiplexing over existing communication networks. Packet switching was chosen as the technology. A large number of internet packet switches/gateways, now known as routers, should use store-and-forwarding on variable length data packets. Other goals were, in order:
- Survivability. The network should survive even if routers or links went down.
- Types of service. The internet should support a wide array of communication services. Realized as TCP for reliable transport, and UDP for unreliable but simple transport.
- Varieties of networks. The architecture should allow different network types, like mobile networks and enterprise-networks.
- Distributed management
- Low-effort host attachment. It should be easy to add a new router or host to the network, without having to reconfigure everything else.
- Resources should be accountable, for getting network operators paid.
The internet works on a best-effort basis. There is never any guarantees at IP level that data is transmitted successfully. TCP mitigates some of this problem by using retransmission and acknowledgments, but there's always a chance of data loss when using the internet.
In general, network neutrality is the idea that all packets on the internet are treated equally by internet service providers. The idea that for example Netflix should not be allowed to pay Telenor for users to experience better speeds for Netflix than Viaplay.
Since the beginning of the internet, service needs have changed a lot. Today, most traffic on the internet is video traffic. This is not because people only stream videos all day, but because video traffic is the most demanding traffic type in wide use (streaming an hour of high-quality video could use the same amount of data as visiting hundreds or thousands of websites).
Some network operators think that streaming service providers should pay them more, since they use most of the available network bandwidth. This violates the idea of net neutrality, and is illegal in many places (including Norway).
In the European Union, net neutrality is a legal requirement for ISP's. However, there exists an exemption for "specialized services". The idea of a specialized service is that ISP's should be allowed to do more than just sell internet access, while still having fair competition.
The full definition of a "specialized service" given by the EU is:
services other than internet access services which are optimised for specific content, applications or services, or a combination thereof, where the optimisation is necessary in order to meet requirements of the content, applications or services for a specific level of quality
Defining specialized services has been difficult, and there are gray areas.