Rare project developers

[Alexander Gall <>]

Implementation of this feature was as simple as expected, so I went
ahead with it. The change for the SDE package is here:

https://github.com/alexandergall/bf-sde-nixpkgs/commit/b434db2711f387253048a8002156d16614947051

I added the Netberg Aurora 710 as a random example. Adding that
platform to RARE is a one-liner:

https://bitbucket.software.geant.org/projects/RARE/repos/rare-nix/commits/ee71a2eacf9b730245d37ce7e64378c808992762

What's missing here (i.e. in the "properties.nix" file of the SDE
package) is the serial device and speed as well as the PCI address of
the interface that will be used as management port. Without that, a
switch installed with ONIE will not be reachable. This information
needs to be completed before the platform becomes usable unless there
is a BMC like on the WEDGE.

-- 
Alex

On Tue, 5 Jul 2022 15:06:57 +0200, Alexander Gall <> said:

> Here is a (not so) brief summary for this list of what I took out of a
> VC I've had with csaba and Gawen regarding the "bspless deployment". I
> hope it contains some useful information for the folks here :)

> First, to clarify what exactly we're talking about, because there was
> a misunderstanding among us when we talked about "QSFP modules working
> or not".  For csaba it means that he can bring a port up and send and
> receive packets. For Gawen and me it means that as well (of course)
> but also that the software can access the plugin through its I2C
> management interface.

> For SFP/QSFP and friends (but not, e.g. QSFP-DD) it is true that at
> least for some types of plugins, all you need is the physical
> connection of the serdes lanes to bring up a link. You won't be able
> to interact with the plugin in any way, e.g. to determine it's
> properties, monitor operational parameters or, in the case of QSFP,
> disable and enable lasers (among other things). Even just resetting
> the module is not possible since that requires driving a separate pin
> which is also platform dependent (usually through a GPIO controller).

> Providing access to the QSFP plugins (this is meant to include SFPs
> via an adapter) via I2C is a large part of what a BSP is used for. Due
> to the nature of I2C, this can't be handled by generic code.

> The Intel SDE uses an API to encapsulate the platform-specific stuff
> in a few shared libraries. Every BSP produces an instance of these
> libraries for a specific platform. In a regular BSP-based
> installation, bf_switchd uses those libraries to interact with the
> platform, e.g. the QSFP modules but also power supplies and thermal
> sensors.

> The BSP also provides another element that's specific to a platform
> called the "port map", which contains very low-level information about
> the physical wiring of the Tofino serdes ports, e.g. which lane of a
> Tofino MAC block is connected to which lane of a QSFP cage. This
> information is programmed into the ASIC as part of the initialization
> performed by bf_switchd (maybe similar to how auto-MDI/MDIX works, not
> sure). At least that's how I understand it, please correct me if I'm
> wrong :)

> Enter "bspless mode". The only thing that's absolutely necessary for
> bf_switchd to operate is to have the correct port map to match a given
> platform. Usually, this is provided by one of the platform-specific
> libraries from the platform's BSP.  If we build the SDE completely
> without BSP, those libraries are missing and so is the information
> about the port map.  In this case, bf_switchd assumes that we want to
> operate on the Tofino model and automatically switches to that mode,
> i.e. the P4 program will not be loaded on the ASIC.

> However, there is also a way to supply the port map as a JSON file
> through one of the build artifacts of the P4 program that we want to
> run. Csaba has explained how to do this earlier in this thread and
> also in the Intel forum
> (https://community.intel.com/t5/Intel-Connectivity-Research/Installing-SDE-without-corresponding-BSP/m-p/1365676#M3253).
> If bf_switchd can't find the platform libraries but it gets the port
> map via the program's configuration, it does not switch to the model
> but proceeds to load and run the program on the ASIC.

> This is what we call "bspless mode". Due to the missing libraries,
> bf_switchd cannot access the QSFP modules in any way. In the UCLI
> (bf-shell), the entire bf_pltfm context is missing, including
> everything concerning other parts of the chassis (power supplies, fans
> etc.).

> Csaba's suggestion is that we could support platforms for which we
> don't have a BSP in this bspless mode (provided that we can get our
> hands on the port map files), since even that very limited support is
> better than no support at all. I tend to agree that this could cover
> some cases where that's good enough, e.g. equipment in a lab or some
> other place that's close to the operator using non-fancy plugins like
> DAC cables.

> This should actually not be difficult to support with the RARE
> packaging, so I think I'll give that a try :)

> However, I believe that this particular road ends with Tofino1. I'm
> pretty sure that bspless mode is simply impossible for platforms that
> use QSFP-DD, i.e. starting with Tofino2.  These types of plugins
> require interaction with the host to operate properly. I know for a
> fact that that's the case for the SDE when running on Tofino2.

> I also believe that we should *not* go bspless for platforms that have
> a BSP but don't support the newest SDE that we would otherwise like to
> use (e.g. right now we're limited to 9.7.2 in our current RARE release
> due to this problem). bspless should only be a last resort for
> otherwise unsupported platforms.

> I'd welcome feedback of any kind :)

> Thanks for listening.

> -- 
> Alex


> On Mon, 4 Jul 2022 12:31:41 +0200, mc36 <> said:

>> well, not....
>> all you need is
>> - astock debian stable
>> - the sde
>> - and the patch you did against the sde
>> - the yaml you gave me to compile for asic but without bsp
>> - the portmap from stratum
>> - mentioning port in the bf-p4c's output bf_router.json

>> br,
>> cs





>> On 7/4/22 10:49, Gawen Davey wrote:
>>> This requires ONLPv2.
>>> So ONL, but I do have a build guide for ONLP on Ubuntu. But it may be for 
>>> ONLPv1. As far as I remember, ONLPv2 was essentially abandoned -- There 
>>> were two commits this year, then 
>>> it goes back to 2020. And we certainly haven't maintained anything ONL 
>>> for a long time.
>>> 
>>> If you want to give it a go anyway: 
>>> https://github.com/APS-Networks/ubuntu-onlp 
>>> <https://github.com/APS-Networks/ubuntu-onlp>
>>> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>> *From:* mc36 <>
>>> *Sent:* 04 July 2022 08:44
>>> *To:*  <>; Alexander Gall 
>>> <>
>>> *Cc:* Gawen Davey <>; Fr  d  ric LOUI 
>>> <>; Alexander Jeffries 
>>> <>
>>> *Subject:* Re: [rare-dev] Access
>>> look at this:
>>> 
>>> https://github.com/stratum/stratum/blob/753270b6061c5efe15ee60178169b8abe032f65e/stratum/hal/bin/barefoot/deb/start-stratum.sh#L54-L63
>>>  
>>> <https://github.com/stratum/stratum/blob/753270b6061c5efe15ee60178169b8abe032f65e/stratum/hal/bin/barefoot/deb/start-stratum.sh#L54-L63>
>>> especially the #L61, that is, everything except the wedges, they bring 
>>> them up without a bsp!!!
>>> qsfps work fine without bsp... serdes too... okk, sensors are really 
>>> missing but heyy...
>>> 
>>> also: 
>>> https://github.com/stratum/stratum/blob/main/stratum/hal/bin/barefoot/README.build.md
>>>  
>>> <https://github.com/stratum/stratum/blob/main/stratum/hal/bin/barefoot/README.build.md>
>>>   
>>> // ctrl+f: recommended mode
>>> 
>>> gawen, how you see it?
>>> 
>>> thanks,
>>> cs
>>> 
>>> 
>>> 
>>> On 7/4/22 09:20, Alexander Gall wrote:
>>>> On Fri, 1 Jul 2022 19:10:27 +0200, mc36 <> said:
>>>> 
>>>>> what about this one? :)
>>>>> https://github.com/stratum/stratum/blob/main/stratum/hal/config/x86-64-inventec-d10064-r0/port_map.json
>>>>>  
>>> <https://github.com/stratum/stratum/blob/main/stratum/hal/config/x86-64-inventec-d10064-r0/port_map.json>
>>>> 
>>>> Same thing. This will set up the serdes to match the board, but
>>>> without the BSP none of the i2c-related stuff works. Actually, the
>>>> Inventec is even more complicated because some of the
>>>> platform-dependent stuff is currently still supplied by the ONL layer
>>>> (i2c enumeration, HW sensors, QSFP eeprom access etc.), not the
>>>> BSP. And the ONL stuff doesn't work on Debian 11 out of the box :(
>>>>