GRE Tunnels & Dynamic Routing in NSX 6.4
There was a line in the NSX 6.4 release notes that caught my eye: “Support for BGP and static routing over GRE tunnels.” – First thought was “awesome, dynamic routing over a VPN”. But then I realized that that GRE tunnel in itself is a new feature as well, which the release notes don’t really mention. The VMware Docs website also doesn’t mention anything about it.
After a bit of digging, it appears that this feature has been added primarily to support the VMware on AWS architecture and connectivity towards that platform. But that doesn’t mean us mortals can’t use these tunnels to have some fun! ;-)
The configuration for these GRE tunnels is not available through the vCenter GUI, only via the NSX API. I’ll go through a couple of API calls to work with GRE tunnels, below. If you’re new to doing API calls to NSX, have a look here for an excellent tutorial using Postman. These will all be on an existing NSX Edge, with edge-id 68.
NOTE: The NSX Edge has to be of version 6.4. Upgrade it first, if it’s a lower version. You will get an error otherwise.
Getting a list of GRE Tunnels
To get a list of existing GRE tunnels on an Edge, use the following call:
|
1 |
GET https://nsx-manager/api/4.0/edges/edge-16/tunnels |
If you have a GRE tunnel configured, here’s an example of what the returning body will be like:
Creating a new GRE Tunnel
The configuration needed to create a new GRE tunnel is quite straightforward. It takes a source and destination, tunnel interface details and a health check. Here’s the API call:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
POST https://nsx-manager/api/4.0/edges/edge-68/tunnels Body: <tunnel> <name>GRE-1-Tunnel-to-GRE-2</name> <description>GRE-1 Tunnel to GRE-2</description> <type>gre</type> <enabled>true</enabled> <sourceAddress>10.8.230.100</sourceAddress> <destinationAddress>10.8.25.100</destinationAddress> <tunnelInterface> <mtu>1400</mtu> <tunnelAddresses> <tunnelAddress>100.194.0.1/24</tunnelAddress> </tunnelAddresses> </tunnelInterface> <greConfig> <enableKeepAliveAck>true</enableKeepAliveAck> </greConfig> <tunnelHealthCheck> <enabled>false</enabled> <type>ping</type> <interval>3</interval> <deadTimeMultiplier>4</deadTimeMultiplier> </tunnelHealthCheck> </tunnel> |
Below are the most important settings inside the API call:
| Setting | Description |
|---|---|
| sourceAddress | The IP address local on this NSX Edge that’s going to be used to connect from. |
| destinationAddress | The IP address that the NSX Edge will connect to. |
| tunnelAddresses | This IP address will be attached to the tunnel endpoint. Because this is a point-to-point connection; I used .1 for the first Edge and .2 for the second Edge. |
| tunnelHealthCheck | This section defines a ping healthcheck to the destinationAddress. If the healthcheck fails, the tunnel will be deactivated. Set interval to the second interval you want to monitor the destination and the deadTimeMultiplier to the times a healthcheck can fail before the tunnel is taken down. |
Repeat this process for the other NSX Edge, or configure your other GRE endpoint manually. Unfortunately, there doesn’t seem to be a way to get the status of the tunnels. To see the status of a tunnel, simply see if the other end of the tunnel is pingable:
* this is from GRE-2 (edge-69) with IP 100.194.0.2 to GRE-1 (edge-98) with IP 100.194.0.1
For more information on these API calls to manage the GRE tunnels, see the NSX 6.4 API Guide.
Dynamic Routing over GRE
Currently, the tunnels only support dynamic routing connectivity via BGP. I won’t cover all the details of the BGP neighbor and route redistribution configuration, but here’s a BGP howto. In this scenario, I built the following topology:
And after configuring BGP between the GRE tunnel IP addresses, I’m happy to report the BGP relation was established and the Edges were exchanging routes:
Conclusion
Even if you don’t want to bother with API calls to configure tunnels, there are 2 main things you should take away from this feature:
- NSX has always been API first. Support for configuring this in the GUI will soon follow.
- Now that it is possible to do BGP over a GRE tunnel, it’s likely that the IPsec tunnels will also support dynamic routing in the future.
Good times ahead!
Disclaimer: This is merely glass ball guesswork, as I am not involved with setting roadmaps. ;-)
3 Comments
Leave a Reply
You must be logged in to post a comment.
Hi Martijn, thanks for the interesting infos!
A couple of questions to the GRE tunnel topic:
– What about the MTU size? Can I define this somewhere?
– What about the DF bit? Will it be honored?
– Will the GRE tunnel source do fragmentation and the destination reassembling?
– How to configure this?
Thanks and ciao, Gerd
Hey Gerd,
The MTU size is defined in the section. I’ve had it at 1400 because the underlying network was 1500 bytes.
I have not tested the DF bit yet, still something I want to do. It would be cool if it didn’t honour it and reassembled the packets through the tunnel, so we can use VXLAN over 1500 MTU networks (it would also interesting to know what performance penalty that will bring).
Hi Martijn,
Can we have IPsec for the same GRE using below procedure,
If i create IPsec using same peer IP and interesting traffic with BGP peer IP , then create the GRE using API with same peer IP and tunnel IP as BGP peer IP.
Thanks & Regards, Manjunath