Introduction to BGP and cloud networking

Introduction to BGP and cloud networking post thumbnail image

This post is intended for people that is just entering to BGP and cloud networking (i.e. SDN). We’ll bring an introduction to BGP fundamental concepts.  BGP is fundamental for SDN to scale out large networks in the Datacenter for end-points attached to containers and virtual servers.

Border Gateway Protocol (BGP)

Let’s start saying that BGP or BGP-4 is defined in the RFC-4271. RFC are documents that are released to the internet community. Some of they came from important institutions like the IETF and they are used as the foundation for the Internet Standards we know today.

BGP is the most popular and it’s used as a critical component in the Internet today. Basically it helps to route IP packets all around the Internet or inside your Network. It’s a protocol intended for large networks.

Workloads in today’s datacenter are mostly  located in virtual servers, and probably soon, in containers, forming a huge, dynamic and complex network. Adding that containers are ephemeral workloads, constantly scaling up or shutting down, adding, moving or removing network end-points every minute or second.  What better protocol like BGP to manage that changing and enormous challenge, very similar of what is happening in Internet. This is why BGP is the foundation of SDN technologies like Calico or Nuage Networks.

Autonomous systems are the building blocks of Internet

What are the Autonomous systems (AS):

  • AS’s are collections of IP prefixes. Like for examples if you have a AS with the number 330 and you have prefixes like and that belongs to them.
  • AS’s have their own routing policies and inter-exchange routing information between other AS’s
  • Public AS that are used to interconnect between different Service Providers requires to be registered by a Regional Internet Registry  (RIR)
  • There is also a private AS can be used when a single upstream exists
    • There’s a private range for private AS’s (64512 to 65534)
    • External routing policy is Identical for the private AS and the single private upstream.

How BGP the protocol works

BGP uses the TCP 179 port to communicate between the different routers. The routers inside an AS use that port to communicate between themselves and exchange routing information. When that happen we say their exchanging iBGP messages (i for internal). When the routers at the border need to exchange routing information between other border routers in other AS, then we call them eBGP messages (e for external).

Those BGP messages are used to:

  • Open a communication between routers
  • Update routing info between them
  • Notifies events like errors
  • Or just keep the communication alive (Keep Alive)

BGP Update Message

Now we’ll check more details on the most important message: Update. This message is use to advertise or/and withdraw new routes. This message is composed of a header and the following sections:

  • Withdraw routes: When the withdraw route length is “0” there are not routes to withdraw from the BGP table.
  • Path Attributes – Routes to be advertise shares the same attributes on this section – BTW, don’t go so hard on yourself trying to understand every term here. Work on the overall concept, like I did in my case, and then you can drill down in it:
    • Origin: Routing protocol. For example: IGP
      • BGP can relay on IGP, or static routes, to resolve nexthop address received in BGP updates from its peers inside an AS. Routes in the AS should be the same point of setup before to setup BGP.
      • You can use IS-IS or OSPF as an IGP option
    • AS_path: The path of different AS’s to reach certain destination. Each AS prepend itself into that path info and pass it to the next peer. Values can be:
      • Empty
      • 100
      • 100 200
    • Next_hop
    • Multi Exit Discriminator: MED is an optional non-transitive attribute. MED is a hint to external neighbors about the preferred path into an AS that has multiple entry points. Some routers, like Nokia SROS based ones, do their own math to have this value.
  • Network Layer reachability info: the network regarding the updated route including the prefix ( i.e. )

A high level description of that message

Image: Introduction to BGP and cloud networking – BGP Update Message (Source: ThousandEyes)

The next image show you how an Update message is composed when you captured.

Image: BGP and cloud networking – BGP Update Message (Source:

Now we’ll see how this update process works and update the forwarding table

How BGP update process works

As you can see in the following picture, the process of updating the routing information starts with a router receiving a RIB (Routing information Base) from its neighbor, with unprocessed routes. The router starts calculating preference and selecting them base on their resolution and create the local RIB.

That Local RIB is then incorporated to the Routing table adding the routes to the ones learned through local routing protocols (IS-IS, OSPF). Then, the new learned routes are also add to the local RIB and then send to the neighbors.

Image: Introduction to BGP and cloud networking – BGP Update Process

I’ve got this from Roman Dodin’s Blog to show you a more practical example. Check the following case on the picture – Also note you can see the Update messages sent between border router peers:

Image: Introduction to BGP and cloud networking – BGP Update Process in a two AS topology example from Roman Dodin’s blog

We have a BGP exchange between R1-R3. Next picture show you how a Local RIB is display in the R3 Nokia SROS Router (i.e. 7750). The “u” flag means the route was used in the routing table and “*” that passed validation checks.

Image: Introduction to BGP and cloud networking – Routing table in SROS from Roman Dodin’s blog

Now you can see the forwarding table in the same router:

Image: Introduction to BGP and cloud networking – Forwarding table in SROS from Roman Dodin’s blog

You can see the following flow chart with the process more related to Nokia SROS routers:

Image: Introduction to BGP and cloud networking – Flow Chart SROS in how routes are selected from Roman Dodin’s blog

How Routes are selected?

new unprocessed routes are selected based on the following criteria on descending order:

  1. Highest weight that is set by the local router
  2. Local preference that is set by the local router based on any criteria like commercial relationships.
  3. Shortest AS Path
  4. Origen type: Internal learned (IGP) routes are preferred
  5. Multi-Exit Discriminator. A preference set by the origen AS
  6. Additional tie-breaking and multi-path criteria.

What else?

You should check also community strings. It’s something more related the way some ISPs manage their customers and the way the want them to advertise routes.  That could be prepending  multiple times their AS or stop announcing to peers. That way, they can control preferred routes or the scale of entries in their network routers.

Image: Introduction to BGP and cloud networking – Community strings for preferred routes


This post used info from different sources:

Nokia (Alcatel-Lucent) BGP configuration tutorial. Part 1 – basic eBGP, iBGP from Roman Dodin. I recommend to check his post about communities also.

How BGP works from ThousandEyes App:  Great introduction and also a great demo at the end by Young Xu:

BGP Series part 1: Message Types by Adeolu Owokade


Tags: , ,

Leave a Reply

Related Post