Hi Csaba/All,
There is some exciting news to share.
FreeRTR transits traffic colorfully!
The configs/code-change/bug-report can be found in the attachments.
Topology:
CE1 (10.1.1.1)
|
[PE11----P1----ASBR11] AS1: JUNOS
|
|
AS2: FREERTR [ASBR21----P2----ASBR22]
|
|
AS3: JUNOS [ASBR31----P3----PE31]
|
(10.11.11.11) CE2
AS1, AS3: PE/ASBR have RSVP Gold and Bronze tunnels towards each other.
Free Router Deployment and Config:
ASBR11
|EBGP-CT
|
| IBGP-CT IBGP-CT V1:ISIS,LDP
ASBR21---------P2(RR)---------ASBR22
V1:ISIS1,LDP V1:ISIS,LDP |
V2:ISIS2,LDP |EBGP-CT
V3:ISIS3,LDP ASBR31
ASBR21:
CT routes learned from ASBR22 will resolve based on the following
route-policy to VRF V2 and V3 based on color via
LDP with ISIS2 and ISIS3 respectively
CT routes will be advertised from V2 and V3 post resolution.
vrf definition v2
rd 68106:168427528
clr4import 100
clr4export 100
clr6import 100
clr6export 100
label4mode per-prefix
label6mode per-prefix
exit
!
vrf definition v3
rd 68106:168427529
clr4import 200
clr4export 200
clr6import 200
clr6export 200
label4mode per-prefix
label6mode per-prefix
exit
!
route-policy ibgp-in
sequence 10 if extcomm 2562:0:100
sequence 20 set vrf v2 ipv4
sequence 30 pass
sequence 40 enif
sequence 50 if extcomm 2562:0:200
sequence 60 set vrf v3 ipv4
sequence 70 pass
sequence 80 enif
sequence 90 drop
exit
router bgp4 1
vrf v1
local-as 64512
router-id 10.21.21.21
no safe-ebgp
address-family unicast
nexthop recursion 3
!
neighbor 10.11.121.1 remote-as 64511
neighbor 10.11.121.1 local-as 64512
neighbor 10.11.121.1 address-family ctp
neighbor 10.11.121.1 distance 20
neighbor 10.11.121.1 send-community standard extended
!
neighbor 10.20.20.20 remote-as 64512
neighbor 10.20.20.20 local-as 64512
neighbor 10.20.20.20 address-family ctp
neighbor 10.20.20.20 distance 200
neighbor 10.20.20.20 update-source loopback0
neighbor 10.20.20.20 traffeng
neighbor 10.20.20.20 send-community standard extended
neighbor 10.20.20.20 route-policy-in ibgp-in
!
afi-clr v2 enable
afi-clr v2 redistribute bgp4 1
!
afi-clr v3 enable
afi-clr v3 redistribute bgp4 1
!
freertr-asbr21(cfg)#show ipv4 route v1
typ prefix metric iface hop time
B 10.10.10.10/32 68106:168427528 200/0 ethernet3@v2:4<< 10.21.12.1 00:09:43
B 10.10.10.10/32 68106:168427529 200/0 ethernet3@v3:4<< 10.21.12.1 00:09:43
NOTE: This needed code modification in FreeRtR to have the route-policy-in apply for CTP family.
The code diff is attached.
freertr-asbr21#show mpls forwarding
label vrf iface hop label targets bytes
702088 v2:4 ethernet4 10.21.22.1 727749(LDP-PUSH),235868(CT-SWAP) 0
942958 v3:4 ethernet5 10.21.32.1 727749(LDP-PUSH),700462(CT-SWAP) 865368
ASBR22:
CT routes learned from ASBR21 will resolve via best effort
LDP in V1 (Same as previous BGP-CT InterOp Demo)
BUGS:
@ASBR-21: ASBR11-ASBR21 EBGP session flap due to Bad Packet
from FreeRTR to JUNOS.
This issue does not arise when the config is initially loaded
but about 5 minutes since the EBGP session towards ASBR11
comes up. The initial CT advertisements go correctly with
right encoding and packet length and ASBR11 accepts and
forms the End-to-End Path. Ping succeeded on End to End Path.
However, ASBR11 after ~5 mins, gets malformed update from
FreeRTR and terminates the session. This needs to be fixed.
I have attached the malformed packet and full-configs.
Thanks,
-Nats-
From:
mc36 <>
Date: Tuesday, February 28, 2023 at 11:29 AM
To: <>, Krzysztof Szarkowicz <>, Kaliraj Vairavakkalai <>, Natrajan Venkataraman <>, Anton Elita <>
Cc: Reshma Das <>
Subject: Re: [rare-dev] BGP CT interop - Colorful Resolution
[External Email. Be cautious of content]
it took more than expected sorry....
please find attached the topology what i now pretty sure will work with the 2 ebgp vpnv4 and 1 bgp-ct for the colors...
at this time i used more descriptive vrf names like core, blue (a color of the core) red (same) and cust[1,2]
the cpes (r1, r2, r6, r7) are dump default route, the r3 is a single pe, have a ebgp-bgp-ct and 2 ebgp-vpnv4,
whereas r4 have just ebgp-bgp-ct, plus isis toward r5... r5 have ibgp-bgp-ct to r4, plus ebgp-vpnv4 to r3...
when i initiate some ping flood, i see different counters in "display mpls forwarding" in r4 so they're separated...
r1#ping 3.3.3.5 vrf cust1 size 1111 repeat 11111111
r2#ping 4.4.4.5 vrf cust2 size 1111 repeat 11111111
now i have some smaller stuff but soon i'll study if i can add the "afi-vrf cust1 resolve-in blue" easily or not :)))
thanks,
cs
On 2/28/23 11:16, Krzysztof Szarkowicz wrote:
> +Anton
>
> --
>
> Krzysztof Grzegorz Szarkowicz, JAWS PLM, Solutions Architect | Phone: +49 89 203 012 127
>
> Please consider my current time zone, when calling: CET (UTC+01:00)
>
> https://easylink.juniper.net/slicing
>
> *From: *mc36 <>
> *Date: *Tuesday, 2023-02-28:Tu at 11:09
> *To: *Kaliraj Vairavakkalai <>, Natrajan Venkataraman <>
> *Cc: *Reshma Das <>, <>, Krzysztof Szarkowicz <>
> *Subject: *Re: [rare-dev] BGP CT interop - Colorful Resolution
>
> [External Email. Be cautious of content]
>
>
> hi,
>
> after a quick look at the vrf import routine, this is where the nexthop's table is set:
>
https://urldefense.com/v3/__https://github.com/rare-freertr/freeRtr/blob/master/src/net/freertr/rtr/rtrBgpVrfRtr.java*L300__;Iw!!NEt6yMaO-gk!Hlv7SV6fqBveGXhRqzVJ0LNnqBQztmRhbiz326LViyZKETqIWnRNylAXw0tkX6-h2IHVbMIvNA46$ <https://urldefense.com/v3/__https:/github.com/rare-freertr/freeRtr/blob/master/src/net/freertr/rtr/rtrBgpVrfRtr.java*L300__;Iw!!NEt6yMaO-gk!Hlv7SV6fqBveGXhRqzVJ0LNnqBQztmRhbiz326LViyZKETqIWnRNylAXw0tkX6-h2IHVbMIvNA46$>
>
> maybe a new variable int that class like (afi-vrf blue) "underlays <vrf1> [vrf2]...[vrfn]" will do the trick permanently...
>
>
https://urldefense.com/v3/__https://github.com/rare-freertr/freeRtr/blob/master/src/net/freertr/rtr/rtrBgp.java*L2690__;Iw!!NEt6yMaO-gk!Hlv7SV6fqBveGXhRqzVJ0LNnqBQztmRhbiz326LViyZKETqIWnRNylAXw0tkX6-h2IHVbHBkgr18$ <https://urldefense.com/v3/__https:/github.com/rare-freertr/freeRtr/blob/master/src/net/freertr/rtr/rtrBgp.java*L2690__;Iw!!NEt6yMaO-gk!Hlv7SV6fqBveGXhRqzVJ0LNnqBQztmRhbiz326LViyZKETqIWnRNylAXw0tkX6-h2IHVbHBkgr18$>
>
> i'll experiment with that after i brought up the currently planned eantc topology.... :)
>
> br,
> cs
>
>
>
> On 2/28/23 10:31, mc36 wrote:
> > hi,
> >
> > i did some real quick tests and this is really just what i said initially,
> >
> > so the original bgp behavior with the next-hop-unchanged toward route reflectors...
> >
> > now i'll play a bit with my original idea about the two ebgp between pe1 and pe2 in the eantc topology...
> >
> > that one is im pretty sure will work for the interop event, and as we'll be onsite, after we configured up everything real quick,
> >
> > we could hack together on something that lifts this 2 ebgp requirement in freerouter... :)
> >
> > thanks,
> >
> > cs
> >
> >
> > On 2/27/23 22:37, Kaliraj Vairavakkalai wrote:
> >> Those are kind words Csaba . I m glad we are in the right direction.
> >>
> >>> i'll try that asap !
> >>
> >> Cool!. Will wait for your experiment. Fingers crossed.
> >>
> >> Thanks
> >>
> >> Kaliraj
> >>
> >> *From: *mc36 <>
> >> *Date: *Monday, February 27, 2023 at 11:37 AM
> >> *To: *Kaliraj Vairavakkalai <>, Natrajan Venkataraman <>
> >> *Cc: *Reshma Das <>, <>, Krzysztof Szarkowicz <>
> >> *Subject: *Re: BGP CT interop - Colorful Resolution
> >>
> >> [External Email. Be cautious of content]
> >>
> >>
> >> + krzysztof && rare-dev as it's a core rare-dev question, thanks you!!!
> >>
> >> hi,
> >>
> >> you're crazy good, seemingly you know my shit better than me.... :)))))))
> >>
> >> so that nexthop recursive was added when i given support for the original bgp behavior of nexthop processing:
> >>
> >> that is, when you no-next-hop-self toward the route reflectors on your peering node...
> >>
> >> freerouter by default assumes next-hop-self without that knob configured...
> >>
> >> but you're right, basically that recursive knob could be reused for the bgp-ct then i wont need two ebgp at eantc tests....
> >>
> >> i'll try that asap !
> >>
> >> ps: i <3 your private build XDDD
> >>
> >> thanks,
> >> cs
> >>
> >>
> >>
> >>
> >>
> >> On 2/27/23 19:56, Kaliraj Vairavakkalai wrote:
> >>> Also, Csaba
> >>>
> >>> Just some random observations:
> >>>
> >>> This config looks interesting:
> >>>
> >>>
> >>>
> >>> r4(cfg-rtr)#neighbor 2.2.2.3 address-family vpnuni
> >>> r4(cfg-rtr)#ne?
> >>> neighbor - specify neighbor parameters
> >>> * nexthop - specify next hop tracking parameter*
> >>>
> >>> r4(cfg-rtr)#nexthop ?
> >>> prefix-list - filter next hops
> >>> *recursion - specify recursion depth*
> >>> route-map - filter next hops
> >>> route-policy - filter next hops
> >>>
> >>> r4(cfg-rtr)#nexthop recursion ?
> >>> <num> - maximum rounds
> >>>
> >>> r4(cfg-rtr)#nexthop recursion 4
> >>>
> >>> and, from the code, this function arguments:
> >>>
> >>> /**
> >>>
> >>> * fix nexthops on a route entry
> >>>
> >>> *
> >>>
> >>> * @param <T> class of address
> >>>
> >>> * @param imp route entry to update
> >>>
> >>> * * @param recurs where to look up nexthops recursively*
> >>>
> >>> * * @param nexthops table where look up resolved nexthops*
> >>>
> >>> * @param recurn maximum recursion depth
> >>>
> >>> * @return true if failed, false if ready
> >>>
> >>> */
> >>>
> >>> public static <T extends addrType> boolean doNexthopFix(tabRouteEntry<T> imp, tabRoute<T> recurs, tabRoute<T> nexthops, int recurn) {
> >>>
> >>> (in tabRoute.java)
> >>>
> >>> if we pass in the color vrf table v2 as recurs argument, will that result in doing bgp nexthop resolution based on entries in that v2 table only?
> >>>
> >>> Basically,
> >>>
> >>> * from CLI, configure a community -> color-vrf/table mapping.
> >>> * When route is received, based on community, pick the color-table, and pass it as recurs in the above function.
> >>> * That should result in resolving bgp nexthop over rsvp/ldp routes in that color-table?
> >>>
> >>> Then we wouldn t need the import policy to match on RD and set vrf nexthop. And this method may work for all bgp families.
> >>>
> >>> I have a dev setup with freeRtr code ready, where I wanted to experiment doing the above. Just wanted to update you.
> >>>
> >>> You can tell if I am understanding the code right, or if I am going in wrong direction. Also, if you agree with the concept,
> >>>
> >>> You will be able to code it faster.
> >>>
> >>> bash-3.2$ dk rtr0
> >>>
> >>> root@edc770cfeb8a:/opt/freertr# telnet localhost 2323
> >>>
> >>> Trying 127.0.0.1...
> >>>
> >>> Connected to localhost.
> >>>
> >>> Escape character is '^]'.
> >>>
> >>> welcome
> >>>
> >>> line ready
> >>>
> >>> freertr#show ver
> >>>
> >>> freeRouter v23.2.25-cur, done by cs@nop.
> >>>
> >>> *private build - kaliraj -*****
> >>>
> >>> place on the web:
https://urldefense.com/v3/__http://www.freertr.org/__;!!NEt6yMaO-gk!H3SnIGYzFbrhxrhXtZ2diMvam2kBs7WeQgDvTqgTCewafLQi7u86cFptGheS5qQy_NBOxPhg$
> <https://urldefense.com/v3/__http:/www.freertr.org/__;!!NEt6yMaO-gk!H3SnIGYzFbrhxrhXtZ2diMvam2kBs7WeQgDvTqgTCewafLQi7u86cFptGheS5qQy_NBOxPhg$>
> >> <https://urldefense.com/v3/__http:/www.freertr.org/__;!!NEt6yMaO-gk!H3SnIGYzFbrhxrhXtZ2diMvam2kBs7WeQgDvTqgTCewafLQi7u86cFptGheS5qQy_NBOxPhg$
> <https://urldefense.com/v3/__http:/www.freertr.org/__;!!NEt6yMaO-gk!H3SnIGYzFbrhxrhXtZ2diMvam2kBs7WeQgDvTqgTCewafLQi7u86cFptGheS5qQy_NBOxPhg$>>
> >>>
> >>> license:
https://urldefense.com/v3/__http://creativecommons.org/licenses/by-sa/4.0/__;!!NEt6yMaO-gk!H3SnIGYzFbrhxrhXtZ2diMvam2kBs7WeQgDvTqgTCewafLQi7u86cFptGheS5qQy_M31zlre$
> <https://urldefense.com/v3/__http:/creativecommons.org/licenses/by-sa/4.0/__;!!NEt6yMaO-gk!H3SnIGYzFbrhxrhXtZ2diMvam2kBs7WeQgDvTqgTCewafLQi7u86cFptGheS5qQy_M31zlre$>
> >> <https://urldefense.com/v3/__http:/creativecommons.org/licenses/by-sa/4.0/__;!!NEt6yMaO-gk!H3SnIGYzFbrhxrhXtZ2diMvam2kBs7WeQgDvTqgTCewafLQi7u86cFptGheS5qQy_M31zlre$
> <https://urldefense.com/v3/__http:/creativecommons.org/licenses/by-sa/4.0/__;!!NEt6yMaO-gk!H3SnIGYzFbrhxrhXtZ2diMvam2kBs7WeQgDvTqgTCewafLQi7u86cFptGheS5qQy_M31zlre$>>
> >>>
> >>> quote1: make the world better
> >>>
> >>> I just got started, able to modify the version string and see the result.
> >>>
> >>> Thanks
> >>>
> >>> Kaliraj
> >>>
> >>> *From: *Natrajan Venkataraman <>
> >>> *Date: *Saturday, February 25, 2023 at 11:52 AM
> >>> *To: *mc36 <>
> >>> *Cc: *Kaliraj Vairavakkalai <>, Reshma Das <>
> >>> *Subject: *BGP CT interop - Colorful Resolution
> >>>
> >>> Hi Csaba,
> >>>
> >>> We tried experimenting the new free-rtr code for colorful resolution with the configuration that you had shared. However we are hitting a few roadblocks that needs to be addressed
> >>> in free-rtr code.
> >>>
> >>> The topology is as same as the demo that was performed in IETF 115 where resolution was happening via LDP best effort for BGP CT route in ASBR21 received from ASBR22
> (10.22.22.22).
> >>> However, we modified the demo based on the configs shared by you for colorful resolution.
> >>>
> >>> We have attached the Old and New configs for your reference for ASBR21. Rest of the configs (P2, ASR22) are more or less the same. You can refer to the topology from the demo
> >>> video.
> >>>
> >>> Thanks,
> >>>
> >>> -Nats-
> >>>
> >>> DETAILS:
> >>>
> >>> ````````````
> >>>
> >>> THIS IS THE NON-WORKING CASE WITH RSVP:
> >>>
> >>> =======================================
> >>>
> >>> The following is the summary list of what we are trying to achieve
> >>>
> >>> @ ASBR 21,
> >>>
> >>> - Configure VRF V2 and V3 to import based on BGP-CT RD/RT
> >>>
> >>> freertr-asbr21#show running-config vrf v2
> >>>
> >>> vrf definition v2
> >>>
> >>> rd 68106:168427528
> >>>
> >>> rt4import 68106:168427528
> >>>
> >>> rt6import 68106:168427528
> >>>
> >>> exit
> >>>
> >>> freertr-asbr21#show running-config vrf v3
> >>>
> >>> vrf definition v3
> >>>
> >>> rd 68106:168427529
> >>>
> >>> rt4import 68106:168427529
> >>>
> >>> rt6import 68106:168427529
> >>>
> >>> exit
> >>>
> >>> - We created RSVP tunnels tunnel1 and tunnel2 in VRF v2 and v3 respectively (However we keep tunnel VRF same as forwarding VRF)
> >>>
> >>> freertr-asbr21#show running-config interface tunnel1
> >>>
> >>> interface tunnel1
> >>>
> >>> description lsp_asbr21_asbr22
> >>>
> >>> tunnel vrf v2
> >>>
> >>> tunnel source loopback2
> >>>
> >>> tunnel destination 10.22.22.22
> >>>
> >>> tunnel mode p2pte
> >>>
> >>> vrf forwarding v2
> >>>
> >>> ipv4 address 10.121.121.121 255.255.255.0
> >>>
> >>> mpls enable
> >>>
> >>> no shutdown
> >>>
> >>> no log-link-change
> >>>
> >>> exit
> >>>
> >>> freertr-asbr21#show running-config interface tunnel2
> >>>
> >>> interface tunnel2
> >>>
> >>> description lsp_asbr21_asbr22
> >>>
> >>> tunnel vrf v3
> >>>
> >>> tunnel source loopback3
> >>>
> >>> tunnel destination 10.22.22.22
> >>>
> >>> tunnel mode p2pte
> >>>
> >>> vrf forwarding v3
> >>>
> >>> ipv4 address 10.221.221.221 255.255.255.0
> >>>
> >>> mpls enable
> >>>
> >>> no shutdown
> >>>
> >>> no log-link-change
> >>>
> >>> exit
> >>>
> >>> !
> >>>
> >>> freertr-asbr21#show ipv4 rsvp v2 summary
> >>>
> >>> source id
> >>> subgroup id target id
description
> >>>
> >>> 10.121.21.21 8906
> >>> :: 0 10.22.22.22 902663016 freertr-asbr21:tunnel1
> >>>
> >>> freertr-asbr21#show ipv4 rsvp v3 summary
> >>>
> >>> source
> >>> id subgroup id target
> id description
> >>>
> >>> 10.221.21.21 12715
> >>> :: 0 10.22.22.22 793731422 freertr-asbr21:tunnel2
> >>>
> >>> - Then we added static routes for 10.22.22.22 in V2 and V3 to point to tunnel1 and tunnel2.
> >>>
> >>> (This we assume, prevents the need for rewriting the nexthop in policy ibpg-in)
> >>>
> >>>
freertr-asbr21#show ipv4 route v2
> >>>
> >>>
typ
> prefix metric iface
> >>> hop time
> >>>
> >>>
S
> 10.22.22.22/32 1/0 tunnel1 10.121.121.122 02:19:56
> >>>
> >>>
freertr-asbr21#show ipv4 route v3
> >>>
> >>>
typ
> prefix metric iface
> >>> hop time
> >>>
> >>>
S
> 10.22.22.22/32 1/0 tunnel2 10.221.221.222 02:20:18
> >>>
> >>> - Then we added policy ibgp-in to filter BGP CT routes to resolve on VRFs V2 and V3 based on RD
> >>>
> >>> (However we found that the following policy is not applying on the BGP CT routes, therefore BGP CT routes do not have a way to set their respective resolving
> VRFs V2 and
> >>> V3)
> >>>
> >>> freertr-asbr21(cfg)#show running-config route-policy ibgp-in
> >>>
> >>> route-policy ibgp-in
> >>>
> >>> sequence 10 if rd 68106:168427528
> >>>
> >>> sequence 20 set vrf v2 ipv4
> >>>
> >>> sequence 30 pass
> >>>
> >>> sequence 40 enif
> >>>
> >>> sequence 50 if rd 68106:168427529
> >>>
> >>> sequence 60 set vrf v3 ipv4
> >>>
> >>> sequence 70 pass
> >>>
> >>> sequence 80 enif
> >>>
> >>> sequence 90 drop
> >>>
> >>> exit
> >>>
> >>> !
> >>>
> >>>
router bgp4 1
> >>>
> >>> neighbor 10.20.20.20 route-policy-in ibgp-in
> >>>
> >>> neighbor 10.20.20.20 vpn-route-policy-in ibgp-in <<<
> >>>
> >>> What we are looking for is an installation of SWAP + PUSH for BGP-CT (SWAP) over RSVP (PUSH) similar to the working scenario (Option B) instead of what is being tried above which
> >>> we assume is (Option A + B). The following section depicts the working case and expected behavior.
> >>>
> >>> THIS IS THE WORKING CASE WITH LDP:
> >>>
> >>> ==================================
> >>>
> >>> freertr-asbr21#show ipv4 bgp 1 ctp database
> >>>
> >>>
> prefix
hop metric
> >>> aspath
> >>>
> >>> 10.2.2.2/32 68098:33685512 10.11.121.1 20/100/0/0 64511
> >>>
> >>> 10.2.2.2/32 68098:33685513 10.11.121.1 20/100/0/0 64511
> >>>
> >>> 10.10.10.10/32 68106:168427528 10.22.22.22 200/100/0/0 64513
> >>>
> >>> 10.10.10.10/32 68106:168427529 10.22.22.22 200/100/0/0 64513 <<<<<
> >>>
> >>> freertr-asbr21#show ipv4 bgp 1 ctp database 10.10.10.10/32 68106:168427529
> >>>
> >>> id category value
> >>>
> >>>
> vrf
v1:4
> >>>
> >>> ipver
4
> >>>
> >>>
> rd
68106:168427529
> >>>
> >>> alt0 nexthop 10.22.22.22
> >>>
> >>> alt0 extended community 2562:0:200
> >>>
> >>> alt0 remote label 833274 <<<<<
> >>>
> >>> Here, the local labels are seen as part of the LDP database.
> >>>
> >>> freertr-asbr21#show ipv4 ldp v1 database
> >>>
> >>> prefix local remote
hop
> >>>
> >>> 10.10.10.10/32 898649 451252 785360 10.21.12.1
> >>>
> >>> 10.10.10.10/32 37034<< 451252 833274 10.21.12.1
> >>>
> >>> freertr-asbr21#show mpls forwarding
> >>>
> >>> label vrf iface hop
> label targets bytes
> >>>
> >>> 37034<< v1:4 ethernet3 10.21.12.1 451252 833274
0
> >>>
> >>> freertr-asbr21#show mpls forwarding 37034
> >>>
> >>> category value
> >>>
> >>> label 37034
> >>>
> >>> key vrfUni-vrf unicast <<<
> >>>
> >>> working true
> >>>
> >>> forwarder v1:4
> >>>
> >>> interface ethernet3
> >>>
> >>> nexthop 10.21.12.1
> >>>
> >>> remote label 451252 833274 // SWAP 833274 THEN PUSH 451252
> >>>
> >>> Juniper Business Use Only
> >>>
> >>>
> >>> Juniper Business Use Only
> >>>
> >>
> >>
> >> Juniper Business Use Only
> >>
>
>
> Juniper Business Use Only
>
