Trunking VLAN's towards Docker containers in VMware
This is what it will look like: a VM running Rocky Linux with one NIC in trunking mode, having presence on three VLAN’s. VLAN 50 is the main VLAN used for management access and VLAN 701 and VLAN 60 would be used for containerized applications - in my case cantainers will use maclan driver. Additional application VLAN’s can be added later, as needed.
Start by configuring trunking on vCenter port group: Depending on what you need you can trunk all VLAN’s towards the VM (not wise) or you can trunk just specific VLAN ranges:
You might need to set the policies as follows:
Next, reconfigure the NIC on the Rocky VM to work as a trunk:
[root@docker01 network-scripts]# more ifcfg-ens224
NAME=ens224
DEVICE=ens224
ONBOOT=yes
NETBOOT="yes"
TYPE=Ethernet
Then, configure VLAN 50 interface for management…
[root@docker01 network-scripts]# cat ifcfg-ens224.50
VLAN=yes
TYPE=Vlan
PHYSDEV=ens224
VLAN_ID=50
REORDER_HDR=yes
GVRP=no
MVRP=no
HWADDR=
IPADDR=192.168.50.10
NETMASK=255.255.255.0
GATEWAY=192.168.50.254
DNS1=192.168.50.100
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens224.50
UUID=bdfdd998-7359-4b5f-8ae7-7e3b7786bb22
DEVICE=ens224.50
ONBOOT=yes
PREFIX=24
RES_OPTIONS="rotate timeout:1 retries:1"
…and VALN701 interface:
[root@docker01 network-scripts]# cat ifcfg-ens224.701
VLAN=yes
TYPE=Vlan
PHYSDEV=ens224
VLAN_ID=701
REORDER_HDR=yes
GVRP=no
MVRP=no
IPADDR=192.168.70.29
NETMASK=255.255.255.240
GATEWAY=192.168.70.30
DNS1=192.168.70.1
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=no
IPV4_FAILURE_FATAL=no
IPV6INIT=no
IPV6_AUTOCONF=no
IPV6_DEFROUTE=no
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens224.701
UUID=0809f5a6-1b2a-4a37-b664-fa9c46634c92
DEVICE=ens224.701
ONBOOT=yes
PREFIX=28
RES_OPTIONS="rotate timeout:1 retries:1"
The same configuration should be performed for VLAN 60 interface adjusting settings as needed.
At this point there should be three functioning interfaces, ens224.50, ens224.60 and ens224.701. Next, you need to configure symmetric routing. Rather than taking a stab at explanation, here is a pretty good one instead.
So, in my case the following rule-* and route-* files are needed for VLAN 50 and VLAN 701. You need to perform similar configuration for any other VLAN’s you will need.
The following is routing for VLAN 50. Note the default entry; this is the management interface:
[root@docker01 network-scripts]# cat route-ens224.50
192.168.50.0/24 dev ens224.50 src 192.168.50.10 table rt50
default via 192.168.50.254 table rt50
Next, I specify rule under which the above routes will be utilized:
[root@docker01 network-scripts]# cat rule-ens224.50
from 192.168.50.10 prio 50 table rt50
Similarly the following files deal with VLAN701. Note the absence of default entry:
[root@docker01 network-scripts]# cat rule-ens224.701
from 192.168.70.29 prio 70 table rt701
[root@docker01 network-scripts]# cat route-ens224.701
192.168.70.16/28 dev ens224.701 src 192.168.70.29 table rt701
Now, I need to make sure the alternate routing tables are defined in rt_tables. This is simply a mapping file that say “a number maps to this friendly name”.
[root@docker01 network-scripts]# cat /etc/iproute2/rt_tables
#
# reserved values
#
255 local
254 main
253 default
0 unspec
#
# local
#
#1 inr.ruhep
50 rt50
60 rt60
70 rt701
There are more details on the content of two files here.
Like me, you might need to install NetworkManager-dispatcher-routing-rules package that will allow NetworkManager process the route-* and rule-* files.
Finally, below is a snippet from docker-compose.yml. Note IP address assignment for interface VLAN 701 and its definition towards the bottom utilizing macvlan driver.
version: "3"
services:
nginx:
container_name: nginx
image: nginx:mainline-alpine
restart: always
ports:
- 80:80
networks:
db:
vlan701:
ipv4_address: 192.168.70.28
...
networks:
vlan701:
name: VLAN701 dmz-app to expose external apps
driver: macvlan
driver_opts:
parent: ens224.701
ipam:
config:
- subnet: 192.168.70.16/28
gateway: 192.168.70.30
Depending on the environment you might need to fiddle with rp_filter, see more info below: