openstack使用linux bridge+vxlan模式的网络变化与分析.doc

目前网络上大部分讲解openstack的网络模块neutron的方式都是基于ovs进行讲解的，很少有涉及linux bridge的，由于本人初次接触openstack的版本是ocata，而官方文档的部署模式是基于linux bridge的，网上的资料较少，本文按照自己的思路进行描述，可能存在不对的地方。、先说创建过程：1. 创建外部网络provider。2. 创建外部网络provider的子网，创建过程中创建网桥brq-4c,同时创建dhcp命名空间，并且在网桥上创建端口tap-f3与DHCP服务连接，同时因为provider网络使用的网桥是有eth0提供，所以eth0转为网桥brq-4c的接口。IP地址转移为网桥的地址。3. 创建私网网络selfservice4. 创建私网网络selfservice的子网，创建过程中创建网桥brq-9d，同时创建DHCP命名空间，并且在网桥上创建端口tap-35与DCH服务连接，改网桥的vxlan的id为48，隧道为vxlan-485. 创建路由器router6. 将私网网络selfservice与路由器router连接，这个时候创建路由器router命名空间，同时在私网网桥brq-9d上新增一个端口tap-8f，同时可以通过路由器router查看上面有一个端口与私网selfservice一致。7. 在provider网络上创建router的网关，这个时候在provider的网桥brq-4c上创建一个端口tap-ba，同时在路由器上看到有对应的网关（注：要切换到admin权限下才能看到）。至此：控制（网络）节点上的网络配置全部创建完成，对应的网桥和端口如下：对应的命名空间如下：对应的网络如下：对应的路由如下：8. 创建虚拟机的时候，会在对应的计算节点创建对应的网桥，而控制（网络）节点不会有任何的变化，创建网桥的时候用什么网络的ID，就创建于空间（网络）节点名称一样的网桥，网桥如下：所有结构如下图所示再分解如下：一开始controller上网络接口的配置如下：rootwcontroller # ifconfigbrq2e52256f-38: flags=4163 mtu 1450 ether 22:46:9e:24:f7:9c txqueuelen 1000 (Ethernet) RX packets 20 bytes 1736 (1.6 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0brq3d648406-f4: flags=4163 mtu 1500 inet 192.168.211.11 netmask 255.255.255.0 broadcast 192.168.211.255 ether 06:82:7c:00:01:b2 txqueuelen 1000 (Ethernet) RX packets 189744 bytes 60059122 (57.2 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 56953 bytes 18259759 (17.4 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0brqd73f593e-3a: flags=4163 mtu 1450 ether 1e:a7:45:09:d2:25 txqueuelen 1000 (Ethernet) RX packets 68 bytes 3478 (3.3 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0docker0: flags=4099 mtu 1500 inet 172.17.0.1 netmask 255.255.0.0 broadcast 0.0.0.0 ether 02:42:68:2a:17:10 txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0eth0: flags=4163 mtu 1500 inet6 fe80:482:7cff:fe00:1b2 prefixlen 64 scopeid 0x20 ether 06:82:7c:00:01:b2 txqueuelen 1000 (Ethernet) RX packets 423257 bytes 136849834 (130.5 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 167688 bytes 32119313 (30.6 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0eth1: flags=4163 mtu 1500 inet 10.10.1.195 netmask 255.255.255.0 broadcast 10.10.1.255 inet6 fe80:70ff:fef4:16 prefixlen 64 scopeid 0x20 ether 02:00:70:f4:00:16 txqueuelen 1000 (Ethernet) RX packets 823534 bytes 263237259 (251.0 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 713441 bytes 219028904 (208.8 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0lo: flags=73 mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 :1 prefixlen 128 scopeid 0x10 loop txqueuelen 1 (Local Loopback) RX packets 7501903 bytes 2489191984 (2.3 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 7501903 bytes 2489191984 (2.3 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0tap13e6f3df-44: flags=4163 mtu 1450 ether da:53:12:6b:c6:c4 txqueuelen 1000 (Ethernet) RX packets 16 bytes 2608 (2.5 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 24 bytes 2958 (2.8 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0tap3993ded3-a9: flags=4163 mtu 1500 ether be:fd:f8:4f:4a:5d txqueuelen 1000 (Ethernet) RX packets 24318 bytes 2087968 (1.9 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 147064 bytes 11429892 (10.9 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0tap6f9999ab-75: flags=4163 mtu 1500 ether 82:81:c7:25:f0:f9 txqueuelen 1000 (Ethernet) RX packets 21920 bytes 1814175 (1.7 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 151249 bytes 11756457 (11.2 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0tapcfaba77e-18: flags=4163 mtu 1450 ether ba:d5:fd:76:36:39 txqueuelen 1000 (Ethernet) RX packets 31659 bytes 2370348 (2.2 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 24852 bytes 1947535 (1.8 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0tapeb1df5ee-9e: flags=4163 mtu 1450 ether 9e:d5:ce:08:48:92 txqueuelen 1000 (Ethernet) RX packets 17 bytes 1926 (1.8 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 64 bytes 4082 (3.9 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0vxlan-46: flags=4163 mtu 1450 ether 1e:a7:45:09:d2:25 txqueuelen 1000 (Ethernet) RX packets 9 bytes 1338 (1.3 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 60 bytes 2664 (2.6 KiB) TX errors 0 dropped 11 overruns 0 carrier 0 collisions 0vxlan-68: flags=4163 mtu 1450 ether 22:46:9e:24:f7:9c txqueuelen 1000 (Ethernet) RX packets 24858 bytes 1600645 (1.5 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 31659 bytes 1928386 (1.8 MiB) TX errors 0 dropped 9 overruns 0 carrier 0 collisions 0网桥结构如下：rootwcontroller # brctl showbridge namebridge idSTP enabledinterfacesbrq2e52256f-388000.22469e24f79cnotap13e6f3df-44tapcfaba77e-18vxlan-68brq3d648406-f48000.06827c0001b2noeth0tap3993ded3-a9tap6f9999ab-75brqd73f593e-3a8000.1ea74509d225notapeb1df5ee-9evxlan-46docker08000.0242682a1710nocompute1的网络接口rootwcom1 # ifconfigbrq2e52256f-38: flags=4163 mtu 1450 ether 5a:48:96:7a:35:31 txqueuelen 0 (Ethernet) RX packets 11 bytes 1116 (1.0 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0brqd73f593e-3a: flags=4163 mtu 1450 ether f2:ed:f1:a2:17:e8 txqueuelen 0 (Ethernet) RX packets 57 bytes 2682 (2.6 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0docker0: flags=4099 mtu 1500 inet 172.17.0.1 netmask 255.255.0.0 broadcast 0.0.0.0 ether 02:42:1e:6e:81:f9 txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0eth0: flags=4163 mtu 1500 inet 192.168.211.25 netmask 255.255.255.0 broadcast 192.168.211.255 inet6 fe80:40d:f6ff:fe00:1c0 prefixlen 64 scopeid 0x20 ether 06:0d:f6:00:01:c0 txqueuelen 1000 (Ethernet) RX packets 541257 bytes 402036673 (383.4 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 272370 bytes 22098834 (21.0 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0eth1: flags=4163 mtu 1500 inet 10.10.1.242 netmask 255.255.255.0 broadcast 10.10.1.255 inet6 fe80:20ff:fe29:18 prefixlen 64 scopeid 0x20 ether 02:00:20:29:00:18 txqueuelen 1000 (Ethernet) RX packets 527416 bytes 154641475 (147.4 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 607894 bytes 202236972 (192.8 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0lo: flags=73 mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 :1 prefixlen 128 scopeid 0x10 loop txqueuelen 0 (Local Loopback) RX packets 375937 bytes 19737009 (18.8 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 375937 bytes 19737009 (18.8 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0tape802bb66-5a: flags=4163 mtu 1500 inet6 fe80:fc16:3eff:fefb:ca0b prefixlen 64 scopeid 0x20 ether fe:16:3e:fb:ca:0b txqueuelen 500 (Ethernet) RX packets 18158 bytes 1495667 (1.4 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 18130 bytes 1290301 (1.2 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0vxlan-46: flags=4163 mtu 1450 ether f2:ed:f1:a2:17:e8 txqueuelen 0 (Ethernet) RX packets 60 bytes 2664 (2.6 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 9 bytes 1608 (1.5 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0vxlan-68: flags=4163 mtu 1450 ether 5a:48:96:7a:35:31 txqueuelen 0 (Ethernet) RX packets 22633 bytes 1387715 (1.3 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 18158 bytes 1786195 (1.7 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0因开始部署测试了几台虚拟机，导致环境很乱，无法重新每一部发生的过程，先清理环境取消浮动IP绑定，命令行：openstack server delete floating ip selfservice-instance 203.0.113.104删除浮动IPopenstack floating ip delete 192.168.211.209查看浮动IP列表 openstack floating ip list删除虚拟机openstack server delete selfservice-instance以上操作对于controller的接口和网桥没有任何变化，但是compute1的接口发生了变化rootwcom1 # brctl showbridge namebridge idSTP enabledinterfacesbrq2e52256f-388000.5a48967a3531novxlan-68brqd73f593e-3a8000.f2edf1a217e8novxlan-46docker08000.02421e6e81f9no可以发现tape802bb66-5a接口已经不存在了，说明随着虚拟机的删除，对接该虚拟机的tape802bb66-5a接口也删除了，但是网络还存在。开始删除网络：目前网络结构如下：另外还有一个router，计划先删内网，再删router，再删provider删除serfservice11、 因为selfservice1有和router对接，先删除接口neutron router-interface-delete router selfservice12、 查看对应的router接口3、 删除子网发现端口tapeb1df5ee-9e删除了，那删除的这个端口是干嘛的呢，其实是把对应的dhcp服务也删除了，DHCP服务的端口对应也删除了。这个时候compute没有任何变化查看对应服务4、 删除网络发现网桥brqd73f593e-3a也被删除了同时compute上的对应的网桥也被删除了5、 那么现在剩下的就是一个私网selfservice，一个路由router，一个公网provider，两个dhcp服务，对应如下：6、 开始删除私网selfservice,先删除在router中的接口7、 删除路由router注意发现两个网桥上都少了一个接口brq2e52256f-38 网桥上的tapcfaba77e-18；brq3d648406-f4网桥上的tap6f9999ab-75，这说明这两个接口都是网桥连接router的。8、 删除子网可以看到，删除子网后，网桥还在，但是少了一个接口，同时DHCP服务也少了一个，说明少的这个接口是连接dhcp服务的。这个时候computer是没有变化的9、 删除私网网络网桥被删除，网络被删除，同时computer上的网络也被删除了10、 删除provider网络子网可以发现，dhcp服务没有，网桥还