Rare project developers

["Moises R. N. Ribeiro" <>]

Csaba,

Very detailed descriptions indeed...thanks, it seems you've most of the bits 
and pieces (and steps to follow) in your head already; and what is needed is 
just time (and peace of mind) to implement/test/validate them. Looking 
forward to seeing the prototype...

My general perception is that the backplane ports will have similar problems 
as regular ports. However, the key aspects is that you don't need to follow 
conventional protocols on them. Then, you'll have the degree of freedom (and 
yet the problems) of reinventing networking when creating an underlay for 
interconnecting disaggregated freeRtr instances. But it seems you think it is 
worth the effort of "reinventing the wheel of networking" because you can do 
it simple, a minimalist approach for unifying such instances (or virtual or 
physical equipment), so that you can apply end-to-end policies and improve 
management by approaching (multiple instances) at a single point 
(centralized-like controller).

I understood also that a physical port (e.g., 100Gbps) can be configured as a 
backplane port and reach another (physical) element also with a physical port 
dedicated to backplane. But the latter port, for instance, can bridge with 
other backplane connections (either local or remote) reaching also this 
physical port. In other words, point-to-point is not required for backplane 
interconnects. It is clear to me that those "interconnect cables" could be 
overlay (virtual) links too. And that backplane ports can coexist with data 
plane ports.

As discussed before, challenges ahead on Bridging/Routing and interlocking 
backplane configurations. I oversee also issues when trying to mimic/provide 
non-blocking for (part of) your backplane....Prioritization, Diffserv, 
preemptive strategies and traffic engineering toward backplane packets would 
be fundamentally limited by what (limited) P4 enables you to do.      


Regards,
Moises



----- Mensagem original -----
De: "cs" <>
Para: "Moises Renato Nunes Ribeiro" <>
Cc: "edgardcunha" <>, "Everson Borges" 
<>, "gabrieltetznermenegueti16" 
<>, "rare-dev" <>
Enviadas: Quinta-feira, 5 de maio de 2022 3:27:01
Assunto: Re: brainstorming about stacking in rare/freertr

hi,
good questions, as usual...
so here are some more answers as per current state...
so i'm still happy with the previous statements.... when i meant, "blindly 
applied to the packet processing" i meant that
right now, when a packet enters the dataplane, the source port (and vlan) 
selects the service and sets the vrfid or bridgeid...
in case of backplane ports, simply there will be a new control that 
overwrites these values directly from the backplane header...
the rest of the packet processing (ttl decrement, etc) will follow as usual, 
so if one configures a look, the packets still
get dropped and wont flood forever... regarding the backplane underlay, i 
just given an example as gre, but normally one
will run it on a native 100g port... i just decided about relocating the 
header because i dont have direct 10g links here
but i use a router on the stick so if i want to introduce it in my homenet, i 
need to stick to a basic ethernet format...
(that is, i must run the backplane stuff over a vlan subinterface to be able 
to interconnect the nodes of me here)
regarding the rest, now the control plane ideas also seems to be also 
concluded... in the first stage, the deaggregated
nature of routers will be kept, and one will have to run an igp on top of the 
configured backplane ports....
furthermore, that one will also have to run a mp-bgp on top of that igp, and 
configure up vpls and vpnv4/vpnv6...
and the only thing that will happen, is when a route points to a backplane 
interface, then the new kind of nexthop,
will be programmed out, and the switches will perform the backplane header 
insertion... at this stage, the whole
fabric is still decentralized and need to be configured at a lot of different 
places, but i'll be able to sort out
the very basics of the fabric related stuff: packet processing at first place 
and issues i dont see right now... :)
and then, in the second stage i'll simply add a blind relaying function to 
the serv p4 p4, it'll be used by the
centralized controlling freerouter to set up the member switches' fibs, and 
then, the next functionality will be
that the serv p4 p4 will get the possibility to get clustered.. that is, 
there will be a single freerouter somewhere
in the fabric with serv p4 sw1..swX config stanzas... these will have the 
relaying serv p4 p4s from the switched connected,
and, there will be a serv p4 main, who will simply have all the sw1..swX 
clustered to... the fib creation will happen
within the serv p4 main from the locally present vrfs/bridges, and the serv 
p4 swNs will simply relay these messages,
and in case the nexthop points to a different switch, they'll simply rewrite 
the nexthop info with the fabric kind of
nexthop... and at this point, the stuff is ready, and it was done in very 
small, incremental steps... so that's why
i still think that it's feasible and will be easy-peasy....
br,
cs



On 5/5/22 05:41, Moises R. N. Ribeiro wrote:
> Csaba,
> 
> 
> I am not sure I've grasped what you have in mind...so, here come the 
> (usual) silly and naive questions/comments/requests:
> 
> Config interlock/lockdown sounds interesting, however, can you 
> overseen/comment on how stiff the whole config system becomes with such 
> interdependent approach? Oh, you see, TACACS and other centralization (for 
> authentication/coordination/orchestration) measures will follow 
> soon...thus, it is possible that this modification you're proposing now may 
> push freeRtr away from the pure distributed approach you'd managed to 
> preserve.
> 
> GRE seems to be your interconnection default/paradigm for the backplane 
> "virtual bus/cabling. On the backplane "connector/port" itself, will it be 
> a single entrance or multiple points of attachment? It sounds like an 
> entrance (bridge_id) that can be turned (internally) into multi-port 
> element/user/local port (by that bridge), right?
> 
> Provided that info in the backplane header are "blindly applied to the 
> packet processing state", what are your thoughts on backplane (route) 
> loops? How that could be prevented/detected/mitigated? A user decide to 
> stitch/interconnect backplanes not using L3 elements...Or is it compulsory 
> to have VRFs (and they would drop packets by TTL) to do that? I'm assuming 
> that "bridge" is Br-like element in virtualized environments. What exactly 
> is a "switch"? Is it a fully functional L2 element with STP, for instance?
> 
> Finally, I am trying to think of what kind of expressiveness (i.e., 
> topologies/slices/architectures) your proposal enables to be implemented 
> using the elements IDs you mentioned (e.g, switch, VRF, bridge, 
> nexthop)...any illustrative example in mind to help us visualize that?
> 
> 
> Regards,
> Moises
> 
> 
> ----- Mensagem original -----
> De: "cs" <>
> Para: "Moises Renato Nunes Ribeiro" <>
> Cc: "edgardcunha" <>, "Everson Borges" 
> <>, "gabrieltetznermenegueti16" 
> <>, "rare-dev" <>
> Enviadas: Quarta-feira, 4 de maio de 2022 10:44:35
> Assunto: Re: brainstorming about stacking in rare/freertr
> 
> hi,
> 
> On 5/4/22 13:24, Moises R. N. Ribeiro wrote:
>>
>> Not sure how it works in blades and stacks, perhaps permission to config 
>> "backplane networking" should require a second level security password. 
>> Sort of admin/SU to play with parameters that are beyond tenant (regular 
>> user, not privileged user) scope.
>>
> not a bad idea, i was also thinking to introduce a kind of lockdown under 
> various config modes... :)
> until that, freerouter have tacacs command authorization, that is, every 
> command could be forbidden by a central authority...
> 
>> Apparently, you are creating a backplane protocol/encapsulation and will 
>> stick/remove an extra header to packets that should go through them. In a 
>> general case, i.e., not assuming freeRtR network, P2P/overlay/proxy kind 
>> of tunnels for stitching freeRtr backplanes of distributed instances do 
>> freeRtr would do the trick. PolKA/SR would do that too, in case other 
>> instances in between to lead to the destination freeRtr backplane, right? 
>> But it requires PolKA/SR-enabled nodes in between. And that might beg for 
>> a centralized (PCE-like) control and multidomain strategies the near 
>> future.
> 
> yeahh so polka also could do the that part to just route the fabric packets 
> but still, it seems out of scope for polka...
> i'm thinking for now about a header to be inserted after the ethertype (to 
> be able to go over a gre for example):
> 
> bit<16>target_switch_id
> bit<16>target_vrf_id
> bit<16>target_bridge_id
> bit<16>target_nexthop_id
> 
> and in case it's present (that is, the packet arrived from a previously 
> backplane configured port),
> it just blindly get applied to the packet processing state, and that's all 
> what needed in the dataplanes...
> 
> br,
> cs
> 
> 
>>
>> Regards,
>> Moises
>>     
>>
>> ----- Mensagem original -----
>> De: "cs" <>
>> Para: "Moises Renato Nunes Ribeiro" <>
>> Cc: "edgardcunha" <>, "Everson Borges" 
>> <>, "gabrieltetznermenegueti16" 
>> <>
>> Enviadas: Quarta-feira, 4 de maio de 2022 2:04:35
>> Assunto: Re: brainstorming about stacking in rare/freertr
>>
>> hi,
>>
>> On 5/4/22 01:44, Moises R. N. Ribeiro wrote:
>>
>>>
>>> Did we get it right?
>> yeahhh, you got it 100% right...
>>
>>
>>> If so, I am afraid to tell you this is "mission impossible"...just give 
>>> it up and buy Cisco....freeRtr will never be able to....(just in case 
>>> you're driven by challenges and low
>>> steam blackmails)   :-)
>> i spent the last day thinking through things here & there and i still see 
>> it feasible...
>> and basically we're almost there, just some very small&(seemingly)easy 
>> functionalities are missing:
>> - backplane header functionality to the dataplanes to pass the packets 
>> back & forth natively...
>> - the connection capability between multiple p4 servers to share the 
>> local&remote nexthop info...
>>
>> br,
>> cs
>>
>>
>>
>>> Regards,
>>> Moises
>>> image.png
>>>
>>>
>>>       fyi
>>>
>>>
>>>       -------- Forwarded Message --------
>>>       Subject: brainstorming about stacking in rare/freertr
>>>       Date: Tue, 3 May 2022 09:09:44 +0200
>>>       From: mc36 < <>>
>>>       Reply-To:  <>
>>>       To:  <> 
>>> < <>>
>>>
>>>       hi,
>>>
>>>       during the migration to sde990 we started using the port metadata 
>>> to silent a warning about casting portid to subifid...
>>>       this seems to open up some more possibilities, especially i'm now 
>>> thinking about introducing a new have_backplane flag... :)
>>>       the idea here is to one would be able to configure a set of 
>>> switches to form a cluster at their control and data planes
>>>       much like in cisco aci or their stackwise stuff... here are the 
>>> some details i'm thinking about right now:
>>>       - on the (on the fly configurable!) backplane ports, we'll use a 
>>> small prepended backplane header much like on the cpu-port,
>>>             but here with the extra field about the target memberid... 
>>> this way the solution will handle well the line-card or
>>>             multi-chassis modes of operation, because the backplane 
>>> packets become routeable based on the memberid alone...
>>>       - all the member switches will have configured with an uniform 
>>> server p4lang configuration exclusively dealing with
>>>             the backplane port(s) properties and nothing else... under 
>>> the hood, they'll run a discovery and relay the for-us
>>>             packets toward the elected controller, and relay the rest of 
>>> the freertr-middleware communication too...
>>>       - there would be (pair of) a dedicated servers, also configured as 
>>> part of the backplane to be able to be elected,
>>>             who will take control over the whole topology, and act as a 
>>> single point of configuration...
>>>       - think about running a p4emu or p4dpdk instance, and a p4lang to 
>>> control that, but with something more
>>>       - all the member switches will have a server p4 sw1..swX instances 
>>> where you export your ports vrfs and bridges,
>>>             but the trick here would be that these vrf and bridge 
>>> instances can span multiple switches without any extra
>>>             configuration needed...
>>>       - this could be achieved in the dataplanes by having a new kind of 
>>> nexthop which will prepend the backplane header,
>>>             filling in the target member and the remote nexthopid...
>>>       - do in this way, we would be able to express even much tricker 
>>> things like, at sw1, i express a member/nexthop pointing
>>>             to a dpdk3 node where the nexthop involves some crypto and 
>>> transmit to a nexthop who's also a backplane kind, with member
>>>             to sw5 and nexthop to a local port...
>>>
>>>       any idea?
>>>       thanks,
>>>       cs
>>>
>>>