3.yeslab数据中心存储课程fcip协议与配置

1、.网络及产品介绍2.FC协议与配置3.iSCSI协议与配置FCoE技术介绍与配置Yeslab数据中心课程Storage系列作者：马海波现任明教教主Yeslab数据中心现任明教教主内容简介内容简介第一部分:FCIP Protocol Overview第二部分:FCIP 配置实验部分第一部分:FCIP Protocol Overview第一部分FCIP Protocol Overview马海波/现任明教教主Storage系列第一部分:FCIP ProtocolHybrid SAN ExtenImplemenionsOverview马海波/现任明教教主Storage系列第一部分:FCIP Proto

2、colFCIP: Fibre Channel over IPOverviewFibre Channel over IP (FCIP) FCIP provides a standard way of encapsulating FC frames withCP/IP,allowing islands of FC SANs to beerconnected over an IP-based networkSame protocol stack as Fibre Channel Think IPv6 over IPv4 GRE Tunnel Initiators &s are still Nativ

3、e FC (or FCoE)Used for FC SAN ExtenData replication over long distanceNative FC is sensitive to latency and dropsTCP already has built in congestion management, retransmis, etc.马海波/现任明教教主Storage系列第一部分:FCIP Protocol OverviewStorage Protocol Technologies catalog马海波/现任明教教主Storage系列第一部分:FCIP Protocol Ov

4、erviewFCIP Frame DetailMax Fibre Channel frame is 2148 bytes plus optional extrasFCIP will segment and reassemble FC frames if MTU too small (TCP payload on second or subsequent packets)Jumbo frames may increase performanceIP MTU of 2300 avoids splitting of TCP frames马海波/现任明教教主Storage系列第一部分:FCIP Pro

5、tocol OverviewFCIP Architecture M马海波/现任明教教主Storage系列第一部分:FCIP Protocol OverviewFCIP and VE PortsFCIP virtual E (VE) ports behave exactly like standard Fibre Channel Eports, except Channel.t the transporthis case is FCIP instead of Fibre马海波/现任明教教主Storage系列第一部分:FCIP Protocol OverviewFCIP LinksFCIP lin

6、ks consist of one or more TCP connections betntwo FCIP link endpo Channel frames.s. Each link carries encapsulated FibreBy default, the FCIP feature on any Cisco MDS 9000 Family switch creates two TCP connections for each FCIP link:One connection is used for data frames.The other connection is used

7、only for Fibre Channel controlframes,t is, switch-to-switch protocol frames (all Class F).This arrangement provides low latency for all control frames.Once the FCIP link is established, the VE port behavior isidentical to E port behavior for aller-switch communication(includingmanagement, zones, and

8、 VSANs).马海波/现任明教教主Storage系列第一部分:FCIP Protocol OverviewFCIP ProfilesThe FCIP profile contains information about the local IP address and TCP parameters. The profile defines the following information:The local connection pos (IP address and TCP port number)The behavior of the underlying TCP connection

9、s for all FCIPlinkst use this profile(定义使用这个profile的FCIP Links的底层TCP的连接行为)马海波/现任明教教主Storage系列第一部分:FCIP ProtocolResolving Differenbetn StorageOverviewand TCP NetworksStorage traffic:Quite bursty (流量丰足的)Latency sensitive (sync apps)延迟性敏感Requires high, instantaneous throughput(要求高的,即时的吞吐量)Traditional T

10、CP:Tries to be network soble(强调网络的互通)Tries to avoid congestion (overrunning downstream routers)强调避免拥塞Backs off when congestion detected(拥塞时回退)Slow to rup over long links (slow start andcongestion avoidance)由慢传送开始再逐渐增速马海波/现任明教教主Storage系列第一部分:FCIP Protocol OverviewFlow Control - FC & FCIPTCP uses a sl

11、iding window for flow control end to end betn FCIP entities BB_Credits not relevant over FCIPFC FCIP is typical powhere problems may occur,slow rup oftraditional TCP can cause FC frame expiry in some conditions: Multiple 1 or 2 Gbps FC sourto slow, high latency WAN path, big buffers Retransmiss, man

12、y sour马海波/现任明教教主Storage系列第一部分:FCIP Protocol OverviewMDS FCIP TCP BehaviorDesigned to reduce probability of drops(设计减少丢包可能性)Bursts are controlled through per flow sha controland congestion windowIncreased resince to drops (增加丢包的回复力)Uses SACK, fast retransmit and shaAggressive slow start mechanism(激进式

13、慢启制)Initial rate controlled by ”min-availab Max rate controlled by ”max-bandwidth”andwidth”Differenwith Normal TCP:When congestion occurs with other conventional TCP traffic, FCIP is more aggressive during recovery ( ”bullying”霸占 the other traffic)-Aggresis proportional to the min-availabandwidth co

14、nfiguration马海波/现任明教教主Storage系列第一部分:FCIP ProtocolFCIP TCP Packet Sha: MDS9000OverviewShr sends at a rate consumaby the downstreath:Immediay sends at ”minimum-bandwidth” rate (avoids early stages oftraditional slow start)Rs up to ”um-bandwidth” rate (using usual slow start andcongestion avoidance meth

15、ods)Requirements for shr to engage:Min-availabandwidth 1/20 max-bandwidthSACK (Selective Ack) must be enabledtcp max-bandwidth-mbps 1000 min-availabw-mbps 500 round-trip-time-ms 1马海波/现任明教教主Storage系列第一部分:FCIP Protocol OverviewFCIPerfaThe FCIPerface is the local endpoof the FCIPlink and a VE porterfac

16、e. All the FCIP and E portparameters are configured in context to the FCIP erface.The FCIP parameters consist of the following:The FCIP profile determines which Gigabit Ethernet port initiates the FCIP links and defines theTCP connection behavior.Peer information.Number of TCP connections for the FC

17、IP link.E port parameters trunking mode and trunk allowed VSAN list.马海波/现任明教教主Storage系列第一部分:FCIP ProtocolFCIP High-Availability Solutions (1)OverviewVRRP-Based High AvailabilityThe following characteristics set VRRP solutions apart from other solutions:If the active VRRP port fails, the standby VRRP

18、 port takes over the VRRP IP address.When the VRRP switchover happens, the FCIP link automatically disconnects and reconnects.This configuration has only one FCIP (E)ISL link.马海波/现任明教教主Storage系列第一部分:FCIP ProtocolFCIP High-Availability Solutions (2)OverviewEthernet PortChannel-BasedHigh AvailabilityT

19、he following characteristics set Ethernet PortChannel solutions apart from other solutions:The Gigabit Ethernet link-level redundancy ensures a transparent failover if one of the Gigabit Ethernet links fails.Two Gigabit Ethernet ports in one Ethernet PortChannel appear like one logical Gigabit Ether

20、net link.The FCIP link stays up during the failover.马海波/现任明教教主Storage系列第一部分:FCIP ProtocolFCIP High-Availability Solutions (3)OverviewFSPF-Based Load BalancingThe following characteristics set FSPF solutions apart from other solutions:Each FCIP link is a separate (E)ISL.The FCIP links can connect to

21、different switches across two SAN islands.The Fibre Channel traffic is load balanced across the FCIP link.马海波/现任明教教主Storage系列第一部分:FCIP ProtocolFCIP High-Availability Solutions (4)OverviewPortChannel-BasedLoad BalancingThe following characteristics set Fibre Channel PortChannel solutions apart from o

22、ther solutions:The entire bundle is one logical (E)ISL link.All FCIP links switches.he PortChannel should be across the same twoThe Fibre Channel traffic is load balanced across the FCIP linkshePortChannel.马海波/现任明教教主Storage系列第一部分:FCIP Protocol OverviewFCIP 与 CCIE DCExtend the SAN over the DCI E.g. F

23、C SAN over OTVActually SCSI over FCP over TCP over IP over Ethernet over MPLS over GRE over IP over EthernetHow does it apply to us?MDS as an FCIP GatewayN7K&N5K doesnt support FCIP featureAdvanced FCIP Features not mentioned such as FCIP Write Acceleration, FCIP T CompresAcceleration, FCIP第二部分:FCIP

24、 配置实验部分第二部分FCIP 配置实验部分马海波/现任明教教主Storage系列第二部分:FCIP 配置实验部分FCIP 实验拓扑马海波/现任明教教主Storage系列第二部分:FCIP 配置实验部分FCIP 实验需求实验需求:配置MDS1和MDS2之间两条FCIP链路,允许VSAN101和VSAN102通过这两条链路.由于G1/1链路是1Gbps独享带宽,优先使用它作为主要链路,由于G1/2是155M共享带宽,则使用它作为备用链路.调整适当MTU值以适应FC帧.IP参数如下:Device MDS1 MDS2Primary Link Address 172.16.10.1/24172.16.

25、10.2/24Secondary Link Addres 172.16.20.1/24172.16.20.2/24MDS FCIP Gateway Configuration步骤如下:Configure normal FC to Initiators &sConfigure IP connectivity betConfigure FCIP Tunnel- Configure FCIP Profilen MDSes- Configure FCIPerfaceTunnel now counts as a TE_port- Normal FC Switching design now apps马海

26、波/现任明教教主Storage系列第二部分:FCIP 配置实验部分MDS1上FCIP 配置 (1)MDS1上:MDS1(config)# vsan databaseMDS1(config-vsan-db)# vsan 101 name VSAN101 MDS1(config-vsan-db)# vsan 102 name VSAN102MDS1(config)# zone mode enhanced-zone vsan 101(保持与M9124A一致)MDS1(config)#er fc1/5MDS1(config-if)# switchport mode e MDS1(config-if)#

27、 switchport trunk mode onMDS1(config-if)# switchport trunk allowed vsan 101 MDS1(config-if)# switchport trunk allowed vsan add 102 MDS1(config-if)# no shuMDS1(config-if)# showfc1/5 is trunkinger fc1/5Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN)Receive B2B Credit is 255 Receive dat

28、a field Size is 2112 Beacon is turned offTrunk vsans (admin allowed and active) (101-102)Trunk vsans (up)(101-102)马海波/现任明教教主Storage系列第二部分:FCIP 配置实验部分MDS1上FCIP 配置 (2)MDS1(config)# feature fcipMDS1(config)#erface GigabitEthernet 1/1MDS1(config-if)# ip address 172.16.10.1 255.255.255.0 MDS1(config-if)#

29、 switchport mtu 2300MDS1(config-if)# no shutdownMDS1(config-if)#erface gigabitethernet 1/2MDS1(config-if)# ip address 172.16.20.1 255.255.255.0 MDS1(config-if)# switchport mtu 2300MDS1(config-if)# no shutdown马海波/现任明教教主Storage系列第二部分:FCIP 配置实验部分MDS1上FCIP 配置 (3)MDS1(config-if)# fcip profile 10 MDS1(con

30、fig-profile)# ip add 172.16.10.1 MDS1(config-profile)# show fcip profile 10 FCIP Profile 10ernet Address is 172.16.10.1 ( Listen Port is 3225TCP parameters SACK is enablederface GigabitEthernet1/1)PMTU discovery is enabled, reset timeout is 3600 sec Keep alive is 60 secMinimum retransmistimeout is 2

31、00 msum number of re-transmis Send buffer size is 0 KBs is 4um allowed bandwidth is 1000000 kbps Minimum availabandwidth is 500000 kbps Configured round trip time is 1000 usecCongestion window monitoring is enabled, burst size is 50 KB Auto jitter detection is enabled马海波/现任明教教主Storage系列第二部分:FCIP 配置实

32、验部分MDS1上FCIP 配置 (4)MDS1(config)# fcip profile 20 MDS1(config-profile)# ip add 172.16.20.1MDS1(config-profile)# tcp max-bandwidth-mbps 155 min-available- bandwidth-mbps 20 round-trip-time-ms 10MDS1(config-profile)# show fcip profile 20FCIP Profile 20ernet Address is 172.16.20.1 ( Listen Port is 3225T

33、CP parameters SACK is enablederface GigabitEthernet1/2)PMTU discovery is enabled, reset timeout is 3600 sec Keep alive is 60 secMinimum retransmistimeout is 200 msum number of re-transmis Send buffer size is 0 KBs is 4um allowed bandwidth is 155000 kbps Minimum availabandwidth is 20000 kbps Configur

34、ed round trip time is 10000 usecCongestion window monitoring is enabled, burst size is 50 KB Auto jitter detection is enabled马海波/现任明教教主Storage系列第二部分:FCIP 配置实验部分MDS1上FCIP 配置 (5)MDS1(config-profile)#erface fcip10MDS1(config-if)# use-profile 10MDS1(config-if)# peer-info ipaddr 172.16.10.2 MDS1(config-i

35、f)# switchport mode eMDS1(config-if)# switchport trunk allowed vsan 101 MDS1(config-if)# switchport trunk allowed vsan add 102 MDS1(config-if)# fspf cost 500 vsan 101MDS1(config-if)# fspf cost 500 vsan 102 MDS1(config-if)# no shutdownMDS1(config-if)#erface fcip20MDS1(config-if)# use-profile 20MDS1(c

36、onfig-if)# peer-info ipaddr 172.16.20.2 MDS1(config-if)# switchport mode eMDS1(config-if)# switchport trunk allowed vsan 101 MDS1(config-if)# switchport trunk allowed vsan add 102 MDS1(config-if)# no shutdown马海波/现任明教教主Storage系列第二部分:FCIP 配置实验部分MDS2上FCIP 配置 (1)MDS2上:MDS2(config)# vsan databaseMDS2(con

37、fig-vsan-db)# vsan 101 name VSAN101 MDS2(config-vsan-db)# vsan 102 name VSAN102MDS2(config)# feature fcipMDS2(config)#erface GigabitEthernet 1/1MDS2(config-if)# ip address 172.16.10.2 255.255.255.0 MDS2(config-if)# switchport mtu 2300MDS2(config-if)# no shutdownMDS2(config-if)#172.16.10.1MDS2(config

38、-if)#erface Gigabitethernet 1/2MDS2(config-if)# ip address 172.16.20.2 255.255.255.0 MDS2(config-if)# switchport mtu 2300MDS2(config-if)# no shutdownMDS2(config-if)#172.16.20.1马海波/现任明教教主Storage系列第二部分:FCIP 配置实验部分MDS2上FCIP 配置 (2)MDS2(config-if)# fcip profile 10 MDS2(config-profile)# ip add 172.16.10.2

39、MDS2(config)# fcip profile 20 MDS2(config-profile)# ip add 172.16.20.2MDS2(config-profile)# tcp max-bandwidth-mbps 155 min-available-bandwidth-mbps 20 round-trip-time-ms 10MDS2(config-profile)#erface fcip10MDS2(config-if)# use-profile 10MDS2(config-if)# peer-info ipaddr 172.16.10.1 MDS2(config-if)#

40、switchport mode eMDS2(config-if)# switchport trunk allowed vsan 101 MDS2(config-if)# switchport trunk allowed vsan add 102 MDS2(config-if)# fspf cost 501 vsan 101MDS2(config-if)# fspf cost 501 vsan 102MDS2(config-if)# no shutdown马海波/现任明教教主Storage系列第二部分:FCIP 配置实验部分MDS1上FCIP 配置 (3)erface fcip20MDS2(co

41、nfig-if)#MDS2(config-if)# use-profile 20MDS2(config-if)# peer-info ipaddr 172.16.20.2 MDS2(config-if)# switchport mode eMDS2(config-if)# switchport trunk allowed vsan 101 MDS2(config-if)# switchport trunk allowed vsan add 102 MDS2(config-if)# no shutdownMDS2(config-if)# showfcip10 is trunkingHardwar

42、e is GigabitEtherneter fcip 10Port WWN is 20:10:00:0d:ec:4a:80:00Peort WWN is 20:10:00:0d:ec:1a:44:c0Admin port mode is E, trunk mode is onsnmp link se traps are enabledPort mode is TE Port vsan is 1Speed is 1 GbpsTrunk vsans (admin allowed and active) (101-102)Trunk vsans (up) Trunk vsans (isolated

43、).(101-102) ()马海波/现任明教教主Storage系列第二部分:FCIP 配置实验部分MDS1上FCIP 配置 (4)MDS2(config-if)# showfcip20 is trunkingHardware is GigabitEtherneter fcip 20Port WWN is 20:14:00:0d:ec:4a:80:00Peort WWN is 20:14:00:0d:ec:1a:44:c0Admin port mode is E, trunk mode is onsnmp link se traps are enabledPort mode is TE Port vsan is 1Speed is 1 GbpsTrunk vsans (admin allowed and active) (101-102)Trunk vsans (up) Trunk vsans (isolated)Trunk vsans (initializing)er