rare-dev - Re: [rare-dev] [freertr] [RARE-users] backplane/stacking/dc functionality arrived to rare...

Subject: Rare project developers

List archive

Re: [rare-dev] [freertr] [RARE-users] backplane/stacking/dc functionality arrived to rare...

From: mc36 <>
To: rare-dev <>, "" <>
Cc: edgardcunha <>, Everson Borges <>, gabrieltetznermenegueti16 <>, "Moises R. N. Ribeiro" <>
Subject: Re: [rare-dev] [freertr] [RARE-users] backplane/stacking/dc functionality arrived to rare...
Date: Wed, 18 May 2022 09:45:53 +0200

hi,
so as discussed during the meeting @ marseille, i've reconfigured the 4 core
boxes of the lab to a big stack...
here comes the details:

all the switches got a vdc called ctrl... it's the single, redundant
controller of the 4 boxes...
you can think about it as 4 rp in an asr9k... the thing to note here is that you can do
"attach vdc ctrl"
on ams, bud, fra, poz, all will open, but only one will give you a prompt,
the rest will tell you
"this node is standby, active on ethernet8", much like when you try to
console a standby rp in an asr9k...
also you can do "reload vdc ctrl" on the current active and you'll see that
one of the rest will take
over, and the reloaded one will go standby during the bootup process...

this anycast controller needed an ip which is reachable regardless of where
it lives so i've
reconfigured the eth1 of all the boxes to do bridging... it became eth9
everywhere...
and more importantly, the controller needs a control connection too, where it
can communicate
that who's the active and sync the configuration... these are eth6..8 on each
switch, which
is a simple xconnect between the switches...

interface ethernet1
description out of band management port
bridge-group 11
exit
interface ethernet8
description controller redundancy
xconnect oob bvi11 pckoudp 172.16.26.151 99
exit
interface ethernet9
description out of band management port
bridge-group 11
exit
interface bvi11
description out of band management port
macaddr 0001.0a03.0303
vrf forwarding oob
ipv4 address 172.16.26.153 255.255.255.0
exit

each switch had a cpu port, and the bffwd.py connection... all of these need
to be tunneled to the
single controller... the cpuport became an xconnect, the bffwd became an
inter-vrf tcp port forward...

interface ethernet0
description CPU_PORT _NEVER_EVER_ CONFIGURE IT
xconnect oob bvi11 pckoudp 172.16.26.150 10
exit
server forwarder bffwd
port 9080
target vrf oob
target address 172.16.26.150
target port 9080
vrf p4
exit

these need to be caught at the controller itself, they became pwether for the
cpuports, and the
bffwd connections now all handled in a single p4lang server...

interface pwether0
description ams cpuport
pseudowire oob ethernet9 pckoudp 172.16.26.151 10
exit

regardless of where the controller lives actually, you can access it with
ctrl-rare from the bastion...
the sdn1..3 interfaces of the switches were renamed to sdnX1..3, where X is
the origin switch...
so ams was 10.1.1.1 so sdn11 is sdn1 of ams1, and fra was 2, so sdn21 is the
sdn1 of fra...
the "export-port" stanzas were prepended with "forwarder 0"... all the rest
of the config
from the switches were copy-paste... the sdn1,sdn2 of all the switches
previously was in
the core vrf, but from now, they belong to the stack vrf and perform the
backplane discovery...
no addressing or any specific config needed on them at all except to bring
them up...

the overall result, as expected, is that all the 4 swiches from now look a
like a single device,
with a single point of configuration... so the core vrf points to different
switches' external
facing interfaces, but you really don't have to deal with it at all... you
just configure them
as if it was a single switch...

g4pl#show ipv4 route CORE
typ prefix metric iface hop time
C 10.1.39.0/24 0/0 sdn33.1955 null 00:18:59
LOC 10.1.39.3/32 0/1 sdn33.1955 null 00:23:56
C 10.1.101.0/24 0/0 sdn13.101 null 00:23:57
LOC 10.1.101.1/32 0/1 sdn13.101 null 00:23:57
C 10.1.110.0/24 0/0 sdn13.102 null 00:23:44
LOC 10.1.110.1/32 0/1 sdn13.102 null 00:23:57
C 10.1.210.0/24 0/0 sdn23.103 null 00:23:44
LOC 10.1.210.1/32 0/1 sdn23.103 null 00:23:55
B 10.2.1.12/30 200/0 sdn33.1955 10.1.39.9 00:18:47
C 10.2.101.0/24 0/0 sdn23.101 null 00:23:55
LOC 10.2.101.1/32 0/1 sdn23.101 null 00:23:55
C 10.2.104.0/24 0/0 sdn23.104 null 00:23:55
LOC 10.2.104.2/32 0/1 sdn23.104 null 00:23:55
C 10.3.3.3/32 0/0 loopback0 null 00:24:46
C 10.3.12.0/24 0/0 sdn33.106 null 00:23:48
LOC 10.3.12.3/32 0/1 sdn33.106 null 00:23:56
C 10.3.105.0/24 0/0 sdn33.105 null 00:23:56
LOC 10.3.105.3/32 0/1 sdn33.105 null 00:23:56
C 10.4.12.0/24 0/0 sdn43.107 null 00:23:48
LOC 10.4.12.4/32 0/1 sdn43.107 null 00:23:56
O 10.5.5.5/32 110/20 sdn33.1955 10.1.39.9 00:15:42
O 10.6.6.6/32 110/20 sdn13.2200 10.22.0.2 00:23:51
C 10.7.66.0/24 0/0 sdn23.766 null 00:23:55
LOC 10.7.66.1/32 0/1 sdn23.766 null 00:23:55
O 10.8.8.8/32 110/20 sdn13.3068 10.30.68.200 00:23:46
B 10.8.101.0/24 200/0 sdn13.3068 10.30.68.200 00:23:40
O 10.9.9.9/32 110/20 sdn13.1214 10.12.14.47 00:23:46
B 10.10.10.0/30 200/0 sdn13.102 10.1.110.10 00:23:42
O 10.10.10.10/32 110/20 sdn13.102 10.1.110.10 00:23:48
B 10.10.99.0/24 200/0 sdn13.102 10.1.110.10 00:23:46
C 10.11.3.0/24 0/0 sdn13.701 null 00:23:57
LOC 10.11.3.1/32 0/1 sdn13.701 null 00:23:57
O 10.12.12.12/32 110/20 sdn33.106 10.3.12.12 00:23:48
C 10.12.13.0/24 0/0 sdn23.1213 null 00:23:55
LOC 10.12.13.1/32 0/1 sdn23.1213 null 00:23:55
C 10.12.14.0/24 0/0 sdn13.1214 null 00:23:57
LOC 10.12.14.1/32 0/1 sdn13.1214 null 00:23:57
C 10.19.16.0/24 0/0 sdn13.1916 null 00:23:57
LOC 10.19.16.1/32 0/1 sdn13.1916 null 00:23:57
C 10.22.0.0/24 0/0 sdn13.2200 null 00:23:57
LOC 10.22.0.1/32 0/1 sdn13.2200 null 00:23:57
C 10.30.68.0/24 0/0 sdn13.3068 null 00:23:41
LOC 10.30.68.1/32 0/1 sdn13.3068 null 00:23:57
C 10.55.9.0/24 0/0 sdn23.559 null 00:23:55
LOC 10.55.9.1/32 0/1 sdn23.559 null 00:23:55
O 10.99.99.99/32 110/30 sdn13.102 10.1.110.10 00:23:48
B 192.168.0.0/24 200/0 sdn13.102 10.1.110.10 00:23:46
B 192.168.18.0/24 200/0 sdn13.3068 10.30.68.200 00:23:40
B 192.168.19.0/24 200/0 sdn13.3068 10.30.68.200 00:23:40
B 192.168.255.4/30 200/0 sdn13.102 10.1.110.10 00:23:46

g4pl#

and as an igp/bgp peering node connected to the complex, i can say that i can
ping through it, as expected...

mchome-core>ping 10.99.99.99 vrf p4lab source loopback424
2022-05-18 09:43:06
pinging 10.99.99.99, src=10.5.5.5, vrf=p4lab, cnt=5, len=64, tim=1000, gap=0,
ttl=255, tos=0, sgt=0, flow=0, fill=0, sweep=false, multi=false
!!!!!
result=100%, recv/sent/lost/err=5/5/0/0, rtt min/avg/max/sum=70/70/72/353,
ttl min/avg/max=254/254/254, tos min/avg/max=0/0/0
mchome-core>

br,
cs

On 5/15/22 21:27, mc36 wrote:

hi,
backplane functionality finally arrived to rare... it happened by adding capability to
"server p4lang" to be able to handle
multiple dataplanes simultaneously... the target use-cases include line-card
based chassis systems, stacking, or management
of a whole data-center... all the current dataplanes support the
functionality, and they could be freely mixed in a cluster...
technically it's a very small addition where freerouter considers how to
export a given route/mac to a given dataplane...
when it looks up the out-interface, it just checks if the given interface is
local or not to the current asic, and if it
finds that it's remote, then it exports the rule as it would be exported over
an mpls vpn...
it points to a nexthop, which includes complex encapsulations like gre or
l2tp, so completely remote line-cards
could be configured to a cluster... the extra functionality above the export
is a very basic discovery
protocol run over the configured backplane ports... it periodically sends a
hello describing the chassis, forwarder
port and a random... upon reception, the given port is considered to be part
of the backplane... this visibility
then fed to a per forwarder spf (the same class used in isis, ospf and lsrp)
to calculate a per forwarder line-card
reachability to the other line-cards, which is used during the route/mac
exporting process...
http://sources.freertr.org/cfg/p4lang-rout211.tst is a routing test case (but
bridging is also supported)
here r1 is the controller router instructing r2..r4 dataplane vms... r4 is
configured as a backplane only
sparateing r2 and r3 who configured to have backplane and user ports in
common... the r5..r8 routers are
surrounding the dataplane vms and test this mini chassis functionality...
(just a small addition, r1 could be a redundant pair of freerouters with auto
config sync in between them,
much like in real routers... this also test covered:
http://sources.freertr.org/cfg/conn-redun.tst)
finally here are some show commands from the backplane enabled p4 exporter,
but basically they're
the same for all the routing protocols using the freerouter spf class...
br,
cs

r1#show p4lang p4 backplane-graph
sfdp -Tpng > net.png << EOF
graph net {
//fwd0
"fwd0" -- "fe80:bad::1/128" [weight=0]
//fwd1
"fwd1" -- "fwd3" [weight=10] [taillabel="prt128"]
"fwd1" -- "fe80:bad::2/128" [weight=0]
//fwd2
"fwd2" -- "fwd3" [weight=10] [taillabel="prt128"]
"fwd2" -- "fe80:bad::3/128" [weight=0]
//fwd3
"fwd3" -- "fwd2" [weight=10] [taillabel="prt128"]
"fwd3" -- "fwd1" [weight=10] [taillabel="prt129"]
"fwd3" -- "fe80:bad::4/128" [weight=0]
}
EOF

r1#
r1#show p4lang p4 backplane-tree 1
`--fwd1
`--fwd3
`--fwd2

r1#show p4lang p4 backplane-tree 2
`--fwd2
`--fwd3
`--fwd1

r1#show p4lang p4 backplane-tree 3
`--fwd3
|`--fwd2
`--fwd1

r1#
r1#show p4lang p4 backplane-ports 1
port metric ready remote iface
p4lang port 128 10 true 3 p4lang port 129

r1#show p4lang p4 backplane-ports 2
port metric ready remote iface
p4lang port 128 10 true 3 p4lang port 128

r1#show p4lang p4 backplane-ports 3
port metric ready remote iface
p4lang port 128 10 true 2 p4lang port 128
p4lang port 129 10 true 1 p4lang port 128

r1#
r1#show p4lang p4 backplane-route 1
typ prefix metric iface hop time
null fe80:bad::3/128 0/20 #128 fe80:bad::4 00:18:29
null fe80:bad::4/128 0/10 #128 fe80:bad::4 00:18:29

r1#show p4lang p4 backplane-route 2
typ prefix metric iface hop time
null fe80:bad::2/128 0/20 #128 fe80:bad::4 00:18:30
null fe80:bad::4/128 0/10 #128 fe80:bad::4 00:18:30

r1#show p4lang p4 backplane-route 3
typ prefix metric iface hop time
null fe80:bad::2/128 0/10 #129 fe80:bad::2 00:18:31
null fe80:bad::3/128 0/10 #128 fe80:bad::3 00:18:31

r1#
r1#show p4lang p4 backplane-spf 1
category value
reach fwd1 fwd2 fwd3
reachable 3
unreach fwd0
unreachable 1
stub fwd0 fwd1 fwd2
segrou
nosegrou fwd0 fwd1 fwd2 fwd3
bier
nobier fwd0 fwd1 fwd2 fwd3
topostr fwd0,false,0,1 fwd1,true,1,1 fwd2,true,1,1 fwd3,true,2,1
topoid a8dc4d8a
last 2022-05-15 19:02:05 (00:18:41 ago)
fill 0
calc 0
table 1
run 4

r1#show p4lang p4 backplane-topology 1
node category value addition
fwd0 reach false 0
fwd0 segrou 0
fwd0 bier 0
fwd0 prefix fe80:bad::1/128 0
fwd1 reach true 1
fwd1 segrou 0
fwd1 bier 0
fwd1 neigh fwd3 10
fwd1 prefix fe80:bad::2/128 0
fwd2 reach true 1
fwd2 segrou 0
fwd2 bier 0
fwd2 neigh fwd3 10
fwd2 prefix fe80:bad::3/128 0
fwd3 reach true 2
fwd3 segrou 0
fwd3 bier 0
fwd3 neigh fwd2 10
fwd3 neigh fwd1 10
fwd3 prefix fe80:bad::4/128 0

r1#
r1#show running-config server p4lang p4
server p4lang p4
dataplanes 4
forwarder 0 remote ::
forwarder 1 export-vrf v1 1
forwarder 1 export-port sdn11 1 10 0 0 0
forwarder 1 export-port sdn12 2 10 0 0 0
forwarder 1 export-port sdn13 3 10 0 0 0
forwarder 1 export-port sdn14 4 10 0 0 0
forwarder 1 export-port bundle1 dynamic 0 0 0 0
forwarder 1 interconnect ethernet2
forwarder 1 backplane bundle1 10
forwarder 1 remote 10.11.12.111
forwarder 2 export-vrf v1 1
forwarder 2 export-port sdn21 1 10 0 0 0
forwarder 2 export-port sdn22 2 10 0 0 0
forwarder 2 export-port sdn23 3 10 0 0 0
forwarder 2 export-port sdn24 4 10 0 0 0
forwarder 2 export-port bundle2 dynamic 0 0 0 0
forwarder 2 interconnect ethernet4
forwarder 2 backplane bundle2 10
forwarder 2 remote 10.12.13.111
forwarder 3 export-vrf v1 1
forwarder 3 export-port sdn31 1 10 0 0 0
forwarder 3 export-port sdn32 2 10 0 0 0
forwarder 3 export-port sdn33 3 10 0 0 0
forwarder 3 export-port sdn34 4 10 0 0 0
forwarder 3 export-port bundle4 dynamic 0 0 0 0
forwarder 3 export-port bundle3 dynamic 0 0 0 0
forwarder 3 interconnect ethernet6
forwarder 3 backplane bundle4 10
forwarder 3 backplane bundle3 10
forwarder 3 remote 10.13.14.111
vrf v9
exit
!

r1#

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#313): https://groups.io/g/freertr/message/313
Mute This Topic: https://groups.io/mt/91125392/6006518
Group Owner:
Unsubscribe: https://groups.io/g/freertr/unsub []
-=-=-=-=-=-=-=-=-=-=-=-

Re: [rare-dev] brainstorming about stacking in rare/freertr, (continued)

List archive

Re: [rare-dev] [freertr] [RARE-users] backplane/stacking/dc functionality arrived to rare...