Rare project developers

[Gawen Davey <>]

I am aiming to include support for the BF6064X in the future, but the priority is the BF2556X; there are more of these units in the field, and so far has been the only platform that people have requested 9.9.0 support. Sadly, I am the only developer working on the BSPs at the moment, so admittedly, development of this has been relatively glacial.

The SAL/Gearbox situation is a complicated one. This version of the BSP can run happily without the SAL, but like before, without the SAL, those ports won't work.

So yes, it should be able to work with the SAL, as is, although there are likely some port management code I will probably need to add/change. I have tested the prior (and unreleased) re-write of 9.7.0 (I started this first and then ported to 9.9.0) with the SAL and it seems to work.

In case you weren't aware, I can explain how everything is configured and how the system is supposed to work.

The gearbox chips are not electrically connected to the host mainboard, and therefore no accessible to the host. They are instead connected to the Tofino via GPIO pins, configured as MDIO. This setup is performed by the BSP.

The BSP will then create a Unix Domain Socket which the SAL would connect to, to issue "SMI" reads and writes.

Since the gearbox code was expecting PCIe or direct SMI access to the chips, the gearbox driver code had to be modified to connect and send requests to this socket, which is then used by the SAL.

This would explain weirdness like the ports being unusable if a program is loaded over BfRt. When this happens, the platform manager is shutdown and restarted, which will close the socket, and open a new one. The SAL, being unaware of this, will be trying to talk to a socket that doesn't exist anymore, rendering control of those ports impossible.

Unfortunately, no prior effort was made to harden this and make the driver resilient.

Further to this, the SAL itself hosted it's own socket server which the BSP would connect to request port states.

However, my personal opinion would be that state could be streamed to the BSP using the existing socket, this would make the BSP "independent" of the SAL. Combining this with a more resilient SMI driver would mean that bf_switchd​ could be started independently of the SAL, rather than having to issue a gRPC request to get everything going. Then the SAL could theoretically be run as a service rather than the whole application, although there are limitations to this in that I don't think you can call SDE APIs outside of bf_switchd​'s process space.

In any case, I have rewritten the socket server code in the BSP so that it could be easily extended to incorporate extra functions. 

---

From: Alexander Gall <>
Sent: 23 June 2022 12:16
To: Gawen Davey <>
Cc: Frédéric LOUI <>; <>; <>; Alexander Jeffries <>
Subject: Re: Access

 

Hi Gawen

On Thu, 23 Jun 2022 09:56:47 +0000, Gawen Davey <> said:

> At some point, I will have the Thrift bindings done. The reason they aren't in the beta are that (keeping in mind this BSP is built from
> scratch), is that there's a bit of engineering to do to understand the build infrastructure around it (I don't want to do just a copy and paste
> job), and wire everything together. There's some bits missing with respect to grabbing the PSU information, so it wouldn't be complete as the
> BSP stands at the moment. I hadn't quite gotten around to see if I could have empty services, or simply not attach any RPC processors when the
> SDE/BSP asks for it, but if you're actually using Thrift in RARE then I think finishing it is work that should be done.

We don't really use Thrift in RARE but the SDE examples do, which I
use as test cases for the SDE packages. That's how this dependency
comes in. But now that I think about it, maybe that's not actually
true any more (in fact, in could be that was only true for the p4-14
examples, I don't remember). It's just something that I used in the
beginning (back with 9.1.1) and never revisited :/

> However If you'd be happy with empty Thrift services for testing I can look at it next week.

> Having said that, I really want the BSP to be feature-complete when it is released, and that the build semantics do not differ at the point of
> use from pervious BSPs.

That sounds very good. Two more questions (you might have discussed
this on Slack but the history is no longer available): this BSP
doesn't support the gear box in the 2556, right (i.e. the "SAL" is no
longer included)? I know that model is basically EOL. I also see no
support for the bf6064x_t. What's the fate of that platform?

Regards,
Alex

> Gawen
> -----------------------------------------------------------------------------------------------------------------------------------------------
> From: Alexander Gall <>
> Sent: 23 June 2022 07:38
> To: Gawen Davey <>
> Cc: Frédéric LOUI <>; <>; <>; Alexander Jeffries
> <>
> Subject: Re: Access
 
> Hi Gawen

> On Wed, 22 Jun 2022 10:44:10 +0000, Gawen Davey <> said:

>> Oh and P.S.
>> I got so excited about the efuse issue that I neglected to tell you that you should build the sde with thrift disabled, as the BSP doesn't
>> currently have any bindings. You can't do this via the interactive p4studio prompt, but you can do so via p4studio profile apply $
>> {path_to_profile}​

> When you say "currently", you mean that this will not be necessary in
> the final (or a later) version?  We use a build system not based on
> p4studio where we create separate packages for each BSP we support
> (that's for the official RARE releases, csaba is using p4studio for
> testing right now). They share all platform-independent components,
> e.g. the bf-drivers package. In this case, we need to build a separate
> version of that package with thrift disabled. That's not a problem but
> we would like to avoid it as much as possible.

> --
> Alex

>> The profile I used yesterday was:

>> global-options:
>> asic: true
>> features:
>> drivers:
>> bfrt: true
>> grpc: true
>> thrift-driver: false
>> architectures: []
>>
> -----------------------------------------------------------------------------------------------------------------------------------------------
>> From: Gawen Davey <>
>> Sent: 22 June 2022 10:08
>> To: Frédéric LOUI <>; Alexander Gall <>
>> Cc: <>; <>; Alexander Jeffries <>
>> Subject: Re: Access
 
>> Yes, It will go on the portal. But it's beta, not ready for general release.

>> With respect to the latest error, looks like SDE 9.9.0 removed some code that handled a case where a part number is not properly burned into
>> the "efuse". This is not something that can be fixed by the BSP, and requires a patched bf-drivers package. So the SDE version as it comes
> from
>> Intel won't ever work.

>> I've got a script as a workaround. It unpacks the SDE and bf-drivers package, applies the necessary patch, and packages everything back up
>> again. It expects the original SDE tarball to be in the same directory as itself, and will replace it in-situ.

>> Probably don't need to create an entire SDE tarball, so the last three lines are probably unecessary.

>> In any case, I've raised a ticket with Intel, but this certainly sets back my plans. I don't want to add more scripts and instructions to our
>> documentation and downloads, so nothing will be going on the support portal until I hear back from Intel.

>> #! /usr/bin/env bash

>> set -eE

>> script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"

>> cd ${script_dir}

>> tar xvf bf-sde-9.9.0.tgz
>> cd bf-sde-9.9.0/packages
>> tar xvf bf-drivers-9.9.0.tgz
>> rm bf-drivers-9.9.0.tgz

>> patch -p0 <<'EOM'
>> Index: bf-drivers-9.9.0/src/lld/lld_efuse_tof.c
>> ===================================================================
>> --- bf-drivers-9.9.0.orig/src/lld/lld_efuse_tof.c
>> +++ bf-drivers-9.9.0/src/lld/lld_efuse_tof.c
>> @@ -117,6 +117,15 @@ int lld_efuse_tof_load(bf_dev_id_t dev_i
dev_p-> efuse_data.voltage_scaling =
>> extract_bit_fld_128(hi64, lo64, (245 - 128), (243 - 128));
 
>> +    /* If either pipes or ports are disabled in efuse we'll pretend
>> +   * this is a tofin-small */
>> +  if ((dev_p->efuse_data.pipe_disable != 0) ||
>> +      (dev_p->efuse_data.port_disable_map_lo != 0)) {
>> +    /* This is a temporary hack since some of the Tofino lite chip dont have
>> +       have the chip part number burned correctly in the efuse */
>> +    dev_p->efuse_data.chip_part_number = BFN_PART_NBR_BFNT10032D;  // T-3.2-Half
>> +  }
>> +
>> if (dev_p->efuse_data.chip_part_number == BFN_PART_NBR_BFNT10032D) {
>> // T-3.2-Half
>> uint64_t p = dev_p->efuse_data.port_disable_map_lo;
>> EOM

>> tar cvf bf-drivers-9.9.0.tgz bf-drivers-9.9.0
>> rm -rf bf-drivers-9.9.0
>> cd ${script_dir}
>> rm bf-sde-9.9.0.tgz
>> tar cvf bf-sde-9.9.0.tgz bf-sde-9.9.0

> --
> Alexander Gall, Network

> SWITCH
> Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
> phone +41 44 268 15 15, direct +41 44 268 15 22
> https://switch.ch https://swit.ch/linkedin  https://swit.ch/twitter