RARE user and assistance email list

[Marcos Felipe Schwarz <>]

Thanks for the suggestions Frederic,

I tried some of your suggestions already. Setting autoneg to 2 doesn't work, 
but it does with 1. I have another server on port 19 that doesn't have this 
issue.
I'll try the other tests and post the results

Another thing that I noticed is that when everything is worknig correctly and 
I can ping between Both servers with MTU 9000 and no drops, if I ping the GW 
IP at the Wedge I also can only get MTU 8190 to it. Is there a way to 
increase the MTU of a internal interface on RARE OS?

Regards,

Marcos Schwarz

-----Mensagem original-----
De:  <> 
Em nome de Frédéric LOUI
Enviada em: quarta-feira, 5 de janeiro de 2022 15:16
Para: 
Cc: Pedro Diniz <>; Alexander Gall <>; 
Jordi Ortiz <>
Assunto: Re: [RARE-users] Problem on ports after reboot

Did you try to remove configuration from P4lang stanza and sdn interface and 
tried to add it again ?
Instead of plug/unplug ?

It sounds like this is a physical BSP problem.

Unfortunately we did not experience such problem on our case … Can you drop 
QSFP info ? Also this can be related to Mellanox OFED driver.

We configured some 100GE port with CERN with Mellanox XConnect5 ofed driver 
and it worked flawlessly.

Is it the only port at 100GE ? Do you have additional servers ? 

In CHICAGO we have LEONI DAC cable with 100GE port and I set AUTONEG to ON.
(Otherwise the link does not come up)
Can you please try the following line ?

export-port sdn20 28 100 0 2 0

The problem is that we are also using LEONI DAC but as they are using a 
specific P4 switch the BSP is not working correctly.
(No info at all from bf_platform)

Maybe @Alex or @Jordi can share their experience as they have 100GE ports 
connected to Mellanox XCOnnect5 ? 
Unfortunately in my case I usually dealt with 10GE ports.

All in all, please to to enable autoneg and please let me know if this change 
something ...

All the best,
Frederic

> Le 5 janv. 2022 à 16:44, Marcos Felipe Schwarz <> a 
> écrit :
> 
> Dear all,
>  
> I’m having problems on my Wedge running RARE-OS.
> Everytime I reboot the port sdn20 comes dropping packets and with maximum 
> MTU 8190 (ping -M do -s 8162). To solve the issue I need to phisically 
> disconnect the cable and reconnect it. If I reboot again, the problem 
> reapear.
> Has any of you had a similar problem?
>  
> Port sdn20 (export-port sdn20 28 100 0 1 0) is configured with MTU 9000 and 
> conected through a Leoni DAC cable to a Mellanox ConnectX5. I’ve tried 
> changing the DAC cable from EdgeCore, and the problem persists.
> I have limited availability to the equipment, so I’d like to get some ideas 
> and commands to help troubleshoot the issue.
> This Friday I’ll be able to continue the tests, and I intend to:
>       • Change the connection to a different port and verify if this is an 
> port issue
>       • Set up logging and compare port information before and after the 
> problem
>       • Any other ideas that you guys can help me with to either 
> troubleshoot or mitigate the problem
>  
> Regards,
>  
> Marcos Schwarz
>  
> Gerente de P&D | R&D Manager
> Gerência de Execução de P&D em Ciberinfraestrutura | Management of R&D 
> Execution in Cyberinfrastructure Diretoria de Pesquisa e 
> Desenvolvimento | Board of Research and Development RNP - Rede 
> Nacional de Ensino e Pesquisa | Brazilian National Research and 
> Educational Network Promovendo o uso inovador de redes avançadas | 
> Promoting the innovative use of advanced networks http://www.rnp.br | 
> +55 (19) 3787- 3386 | Skype ID: marcos.f.sch Campinas - SP - Brasil | 
> E-mail: 
>  
> <RIO0001.txt>