RARE user and assistance email list

[Marcos Felipe Schwarz <>]

I was only making configuration changes to the device when the problem 
occurred.

One thing I did wrong, and may be related. I configured by accident:
server p4lang p4
  interface sdn7

so i run after, after it I configured sdn7 correclty, saved and rebooted: 
server p4lang p4
  no interface sdn7

I don't know if this may have broke something. Another thing that is that " 
vrf forwarding inet" is configured was configured on two interfaces bvi1 and 
snd7.104.

Regards,

Marcos Schwarz

-----Mensagem original-----
De: Frédéric LOUI <> 
Enviada em: segunda-feira, 23 de agosto de 2021 15:49
Para: Marcos Felipe Schwarz <>
Cc: ; Pedro Diniz <>; 
Fernando Nazareno Nascimento Farias <>; Michael Prieto 
Hernandez <>
Assunto: Re: RES: [RARE-users] RES: Wedge interconnection using Juniper DAC 
cable.

Hi Marcos,

I hope you are doing well.
I’m doing some forensics on RIO0001.

As from now, I see that there is a problem related to the interface between 
the controls plane (freertr) and the dataplane.

Indeed for unknown reasons bf_forwarder.py is initiating a GRPC session and 
immediately quit.
This is a weird observation and this is the first time I see this. At least 
with the basic ONIE installation.

I’m pretty sure I have already rebooted RIO0001 for and checked that 
RARE/freeRtr could restart flawlessly. 


So something has happened in terms of software (upgrade upgrade ...)

Do you know by any chances if anything happened on the WEDGE ?
Software change, upgrade etc.

I don’t have much time this evening/tonight but I’ll have another fresh look 
tomorrow.

À bientôt,
--  Frederic




> Le 23 août 2021 à 16:25, Marcos Felipe Schwarz <> a 
> écrit :
> 
> Hi Frederic and All,
>  
> I tryied to change my configuration to be tagged on VLAN 104:
> interface sdn7
> description WAN PORT -> JUNIPER
> lldp enable
> no shutdown
> log-link-change
> exit
> server p4lang p4
> export-port sdn7 180 100 3 1 0
> exit
> 
> interface sdn7.104
> vrf forwarding inet
> ipv4 address 200.143.232.255 255.255.255.254 exit server p4lang p4 
> export-port sdn7.104 201 exit
>  
> It didn’t work, so I save and rebooted.
>  
> Now all ports seams to be down.
>  
> <image001.png>
>  
> Not sure what I did to get to this state.
>  
> Regards,
> 
> Marcos Schwarz
>  
> De: Frédéric LOUI <> Enviada em: domingo, 22 
> de agosto de 2021 14:36
> Para: Marcos Felipe Schwarz <>
> Cc: ; Pedro Diniz 
> <>; Fernando Nazareno Nascimento Farias 
> <>; Michael Prieto Hernandez 
> <>
> Assunto: Re: [RARE-users] RES: Wedge interconnection using Juniper DAC 
> cable.
>  
> Hi Marcos,
>  
> I hope you are doing well.
> Thanks for taking this to the list so all potential new users can benefit 
> from everyone’s experience.
>  
> I just did had a quick look at your new connection addition and as 
> suggested by @MC36 () for this precise connection I manage
>  
> For LEONI  DAC cable you had with servers, FEC was not mandatory.
>  
> For JUNIPER Interconnection with AOC FEC=‘RS’ has been set.
>  
> bf-sde.bf_pltfm.qsfp> dump-info 7 
> Port                           = 7
> Module type identifier         = QSFP28 (0x11)
> Module                         = 100G AOC or 25GAUI C2M AOC
> 
> Vendor name                    = JUNIPER-INNO    
> Vendor P/N                     = TF-FC010-NJC    
> Vendor rev                     = 1A
> Vendor S/N                     = 1FCS4410022     
> Date & lot code                = 190307  
> Vendor OUI                     = 44-7C-7F
> 
> ## MODULE STATUS
> Status: 0x07 0x02
> 
> ## OTHER MODULE INFO
>   Extended ID:  0xcc    Ethernet Compliance:  0x80
>   Extended Ethernet Compliance:  0x01
>   Connector: 0x23
>   Spec compliance: 0x80 0x00 0x00 0x000x00 0x00 0x00 0x00
>   Encoding: 0x07
>   Nominal Bit Rate: 25500 MBps
>   Ext rate select compliance: 0x00
>   Length (SMF): 0 km
>   Length (OM3): 0 m
>   Length (OM2): 0 m
>   Length (OM1): 0 m
>   Length (Copper): 10 m
>   Device Tech: 0x00
>   Ext Module: 0x00
>   Wavelength tolerance: 0x07 0xd0
>   Max case temp: 0C
>   CC_BASE: 0x29
>   Options: 0x01 0x07 0xff 0xda
>   DOM Type: 0x08
>   Enhanced Options: 0x10
>   Reserved: 0x67
>   CC_EXT: 0xde
>   Vendor Specific:
>     37 34 30 2d 30 36 31 34  31 31 20 52 45 56 20 30
>     31 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> 
> ## INTERRUPT FLAGS
>     LOS: 0x00
>     Fault: 0x00
>     Temp: 0x00
>     Vcc: 0x00
>     Rx Power: 0x00 0x00
>     Tx Bias: 0x00 0x00
>     Reserved Set 3: 0x00 0x00
>     Reserved Set 4: 0x00 0x00
>     Reserved Set 5: 0x00 0x00
>     Vendor Defined: 0x00 0x00 0x00
> 
> ## POWER/TEMPERATURE
>   Power set/LPMODE:  0x01
>   TX disable bits: 0x00
>   TX rate select bits: 0x00
>   Temperature: 33.468750 C
>   Supply Voltage: 3.298300 V
>   Channel Data:      RX Power      TX Power       TX Bias
>     Channel 1:        0.83dBm       -infdBm        7.08mA
>     Channel 2:        0.80dBm       -infdBm        7.08mA
>     Channel 3:        0.98dBm       -infdBm        7.08mA
>     Channel 4:        0.87dBm       -infdBm        7.08mA
>     Reported RX Power is average power bf-sde.bf_pltfm.qsfp>
>  
> So far so good: :)
> As the screenshot shows the port transitioned from DOWN -> UP after 
> having set FEC to RS
>  
> RIO0001#conf t                                                              
>                                                                             
>                      
> RIO0001(cfg)#server p4lang p4                                               
>                                                                             
>                      
> RIO0001(cfg-server)#export-port sdn7 180 100 3 1 0                          
>                                                                             
>                      
> RIO0001(cfg-server)#end                                                     
>                                                                             
>                      
> ...
>  
>  <image002.png>
>  
>  
> LLDP is activated from the sdn7@WEDGE, and also it seems that the remote 
> end is sending some packet.
>  
> Ping However is not working (you used: 200.143.232.255/31 )
>  
> RIO0001#sh run sdn7                                                         
>                                                                             
>                      
> interface sdn7
>  description WAN PORT -> JUNIPER
>  mtu 1500
>  macaddr 0014.127b.3d43
>  lldp enable
>  vrf forwarding inet
>  ipv4 address 200.143.232.255 255.255.255.254  no shutdown  
> log-link-change  exit
>  
> RIO0001#ping 200.143.232.255 /vrf inet                                      
>                                                                             
>                      
> pinging 200.143.232.255, src=null, vrf=inet, cnt=5, len=64, tim=1000, 
> gap=0, ttl=255, tos=0, fill=0, sweep=false, multi=false, detail=false !!!!!
> result=100%, recv/sent/lost/err=5/5/0/0, rtt min/avg/max/total=0/0/1/2
> RIO0001#ping 200.143.232.254 /vrf inet                                      
>                                                                             
>                      
> pinging 200.143.232.254, src=null, vrf=inet, cnt=5, len=64, tim=1000, 
> gap=0, ttl=255, tos=0, fill=0, sweep=false, multi=false, detail=false .....
> result=0%, recv/sent/lost/err=0/5/5/0, rtt min/avg/max/total=10000/0/0/5002
> RIO0001#ping 200.143.232.254 /vrf inet                                      
>                                                                             
>                      
> pinging 200.143.232.254, src=null, vrf=inet, cnt=5, len=64, tim=1000, 
> gap=0, ttl=255, tos=0, fill=0, sweep=false, multi=false, detail=false .....
> result=0%, recv/sent/lost/err=0/5/5/0, rtt min/avg/max/total=10000/0/0/5001
> RIO0001#sh ipv4 arp sdn7                                                    
>                                                                             
>                      
> mac  address  time  static
> 
>  
> À bientôt,
> --  Frederic
> 
> 
> 
> 
> 
> Le 21 août 2021 à 12:39, mc36 <> a écrit :
> 
> hi,
> as far as i know, for 100g, the setting the fec is mandatory...
> regards,
> cs
> 
> 
> On 8/20/21 3:39 PM, Marcos Felipe Schwarz wrote:
> 
> Just correcting that the cable is AOC not DAC.
> Regards,
> Marcos Schwarz
> *De:*Marcos Felipe Schwarz
> *Enviada em:* sexta-feira, 20 de agosto de 2021 08:52
> *Para:* 
> *Cc:* Pedro Diniz <>; Fernando Nazareno 
> Nascimento Farias <>; Michael Prieto Hernandez 
> <>
> *Assunto:* Wedge interconnection using Juniper DAC cable.
> Hi everyone,
> We are using RARE OS at RNP on a EdgeCore Wedge, and we are trying to setup 
> an interconnect from it to a Juniper Router using a 100G DAC cable from 
> Juniper.
> I setup the port using the configuration below, but the cort continues 
> DOWN. IÂ’d like some help with the configuration or with some troubleshoot 
> options to try to find the problem.
> interface sdn7
> lldp enable
> vrf forwarding inet
> ipv4 address 200.143.232.255 255.255.255.254 no shutdown 
> log-link-change exit server p4lang p4 export-port sdn7 180 100 0 1 0 
> exit
> -----+----+---+----+-------+----+--+--+---+---+---+--------+----------------+----------------+-
> PORT |MAC |D_P|P/PT|SPEEDÂ  |FEC |AN|KR|RDY|ADM|OPR|LPBKÂ Â Â  |FRAMES 
> RXÂ Â Â Â Â Â  |FRAMES TXÂ Â Â Â Â Â  |E
> -----+----+---+----+-------+----+--+--+---+---+---+--------+----------------+----------------+-
> 7/0Â  |17/0|180|3/52|100GÂ Â  |NONE|Ds|Au|YES|ENB|DWN|Â  NONEÂ  |Â Â Â 
> Â Â Â Â Â Â Â Â Â Â Â  0|Â Â Â Â Â Â Â Â Â Â Â Â Â Â  0| Regards, 
> Marcos Schwarz *Gerente de P&D | /R&D Manager/* Gerência de 
> Execução de P&D em Ciberinfraestrutura | /Management of R&D 
> Execution in Cyberinfrastructure/ Diretoria de Pesquisa e 
> Desenvolvimento | /Board of Research and Development/ RNP - Rede 
> Nacional de Ensino e Pesquisa | /Brazilian National Research and 
> Educational Network/ Promovendo o uso inovador de redes avançadas | 
> /Promoting the innovative use of advanced networks/ http://www.rnp.br 
> <http://www.rnp.br/> | +55 (19) 3787-3386 | Skype ID: marcos.f.sch** 
> Campinas - SP - Brasil | E-mail:  
> <>