RARE user and assistance email list

[Frédéric LOUI <>]

Hi Marcos,

After a quick look I saw that someone removed the following line in ‘server 
p4lang p4’
...
vrf p4
…

So basically the control plane and dataplane communication could not occur.
As there was no VRF where this discussion would occur.

Before:
RIO0001#sh run p4l                                                            
 
server p4lang p4
 export-vrf inet 1
 export-bridge 1
 export-port sdn16 0 100 0 1 0
 export-port sdn15 8 100 0 1 0
 export-port sdn19 20 100 0 1 0
 export-port sdn20 28 100 0 1 0
 export-port bvi1 200 100 0 0 0
... 
interconnect ethernet0
 exit

P4 VRF configuration:
RIO0001#conf t                                                                
 
RIO0001(cfg)#server p4lang p4                                                 
 
RIO0001(cfg-server)#vrf p4                                                    
 
RIO0001(cfg-server)#end                                                       
 
RIO0001#wr                                                                    
 
% saving configuration
% success
RIO0001#    

After:
RIO0001#sh run p4l                                                            
 
server p4lang p4
 export-vrf inet 1
 export-bridge 1
 export-port sdn16 0 100 0 1 0
 export-port sdn15 8 100 0 1 0
 export-port sdn19 20 100 0 1 0
 export-port sdn20 28 100 0 1 0
 …
 interconnect ethernet0
 vrf p4
 exit
                
As a test I rebooted the WEDGE again in order to check that all restarted as 
expected.

Your config for VLAN 104 was prefect ! 
It is just that ‘vrf p4’ line disappeared

So congrats !

RIO0001#sh ipv4 arp sdn7.104                                                  
                                                                              
                 
mac             address          time      static
fc33.4211.c87b  200.143.232.254  00:01:14  false

RIO0001#show lldp neighbor                                                    
                                                                              
                 
interface  hostname  iface           ipv4  ipv6
sdn16                9803.9b7f.c758  null  null

RIO0001#ping 200.143.232.254  /vrf inet                                       
                                                                              
                 
pinging 200.143.232.254, src=null, vrf=inet, cnt=5, len=64, tim=1000, gap=0, 
ttl=255, tos=0, fill=0, sweep=false, multi=false, detail=false
!!!!!
result=100%, recv/sent/lost/err=5/5/0/0, rtt min/avg/max/total=0/15/57/78
RIO0001#sh run sdn7.104                                                       
                                                                              
                 
interface sdn7.104
 description RIO0001 -> JUNIPER@VLAN104
 lldp enable
 vrf forwarding inet
 ipv4 address 200.143.232.255 255.255.255.254
 no shutdown
 no log-link-change
 exit

Happy « freeRouting » !

All the best,

À bientôt,
--  Frederic




> Le 23 août 2021 à 20:49, Frédéric LOUI <> a écrit :
> 
> Hi Marcos,
> 
> I hope you are doing well.
> I’m doing some forensics on RIO0001.
> 
> As from now, I see that there is a problem related to the interface between 
> the controls plane (freertr) and the dataplane.
> 
> Indeed for unknown reasons bf_forwarder.py is initiating a GRPC session and 
> immediately quit.
> This is a weird observation and this is the first time I see this. At least 
> with the basic ONIE installation.
> 
> I’m pretty sure I have already rebooted RIO0001 for and checked that 
> RARE/freeRtr could restart flawlessly. 
> 
> 
> So something has happened in terms of software (upgrade upgrade ...)
> 
> Do you know by any chances if anything happened on the WEDGE ?
> Software change, upgrade etc.
> 
> I don’t have much time this evening/tonight but I’ll have another fresh 
> look tomorrow.
> 
> À bientôt,
> --  Frederic
> 
> 
> 
> 
>> Le 23 août 2021 à 16:25, Marcos Felipe Schwarz <> a 
>> écrit :
>> 
>> Hi Frederic and All,
>> 
>> I tryied to change my configuration to be tagged on VLAN 104:
>> interface sdn7
>> description WAN PORT -> JUNIPER
>> lldp enable
>> no shutdown
>> log-link-change
>> exit
>> server p4lang p4
>> export-port sdn7 180 100 3 1 0
>> exit
>> 
>> interface sdn7.104
>> vrf forwarding inet
>> ipv4 address 200.143.232.255 255.255.255.254
>> exit
>> server p4lang p4
>> export-port sdn7.104 201
>> exit
>> 
>> It didn’t work, so I save and rebooted.
>> 
>> Now all ports seams to be down.
>> 
>> <image001.png>
>> 
>> Not sure what I did to get to this state.
>> 
>> Regards,
>> 
>> Marcos Schwarz
>> 
>> De: Frédéric LOUI <> 
>> Enviada em: domingo, 22 de agosto de 2021 14:36
>> Para: Marcos Felipe Schwarz <>
>> Cc: ; Pedro Diniz <>; 
>> Fernando Nazareno Nascimento Farias <>; Michael 
>> Prieto Hernandez <>
>> Assunto: Re: [RARE-users] RES: Wedge interconnection using Juniper DAC 
>> cable.
>> 
>> Hi Marcos, 
>> 
>> I hope you are doing well.
>> Thanks for taking this to the list so all potential new users can benefit 
>> from everyone’s experience.
>> 
>> I just did had a quick look at your new connection addition and as 
>> suggested by @MC36 () for this precise connection I manage 
>> 
>> For LEONI  DAC cable you had with servers, FEC was not mandatory.
>> 
>> For JUNIPER Interconnection with AOC FEC=‘RS’ has been set.
>> 
>> bf-sde.bf_pltfm.qsfp> dump-info 7 
>> Port                           = 7
>> Module type identifier         = QSFP28 (0x11)
>> Module                         = 100G AOC or 25GAUI C2M AOC
>> 
>> Vendor name                    = JUNIPER-INNO    
>> Vendor P/N                     = TF-FC010-NJC    
>> Vendor rev                     = 1A
>> Vendor S/N                     = 1FCS4410022     
>> Date & lot code                = 190307  
>> Vendor OUI                     = 44-7C-7F
>> 
>> ## MODULE STATUS
>> Status: 0x07 0x02
>> 
>> ## OTHER MODULE INFO
>>  Extended ID:  0xcc    Ethernet Compliance:  0x80
>>  Extended Ethernet Compliance:  0x01
>>  Connector: 0x23
>>  Spec compliance: 0x80 0x00 0x00 0x000x00 0x00 0x00 0x00
>>  Encoding: 0x07
>>  Nominal Bit Rate: 25500 MBps
>>  Ext rate select compliance: 0x00
>>  Length (SMF): 0 km
>>  Length (OM3): 0 m
>>  Length (OM2): 0 m
>>  Length (OM1): 0 m
>>  Length (Copper): 10 m
>>  Device Tech: 0x00
>>  Ext Module: 0x00
>>  Wavelength tolerance: 0x07 0xd0
>>  Max case temp: 0C
>>  CC_BASE: 0x29
>>  Options: 0x01 0x07 0xff 0xda
>>  DOM Type: 0x08
>>  Enhanced Options: 0x10
>>  Reserved: 0x67
>>  CC_EXT: 0xde
>>  Vendor Specific:
>>    37 34 30 2d 30 36 31 34  31 31 20 52 45 56 20 30
>>    31 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
>> 
>> ## INTERRUPT FLAGS
>>    LOS: 0x00
>>    Fault: 0x00
>>    Temp: 0x00
>>    Vcc: 0x00
>>    Rx Power: 0x00 0x00
>>    Tx Bias: 0x00 0x00
>>    Reserved Set 3: 0x00 0x00
>>    Reserved Set 4: 0x00 0x00
>>    Reserved Set 5: 0x00 0x00
>>    Vendor Defined: 0x00 0x00 0x00
>> 
>> ## POWER/TEMPERATURE
>>  Power set/LPMODE:  0x01
>>  TX disable bits: 0x00
>>  TX rate select bits: 0x00
>>  Temperature: 33.468750 C
>>  Supply Voltage: 3.298300 V
>>  Channel Data:      RX Power      TX Power       TX Bias
>>    Channel 1:        0.83dBm       -infdBm        7.08mA
>>    Channel 2:        0.80dBm       -infdBm        7.08mA
>>    Channel 3:        0.98dBm       -infdBm        7.08mA
>>    Channel 4:        0.87dBm       -infdBm        7.08mA
>>    Reported RX Power is average power
>> bf-sde.bf_pltfm.qsfp> 
>> 
>> So far so good: :)
>> As the screenshot shows the port transitioned from DOWN -> UP after having 
>> set FEC to RS
>> 
>> RIO0001#conf t                                                             
>>                                                                            
>>                        
>> RIO0001(cfg)#server p4lang p4                                              
>>                                                                            
>>                        
>> RIO0001(cfg-server)#export-port sdn7 180 100 3 1 0                         
>>                                                                            
>>                        
>> RIO0001(cfg-server)#end                                                    
>>                                                                            
>>                        
>> ...
>> 
>> <image002.png>
>> 
>> 
>> LLDP is activated from the sdn7@WEDGE, and also it seems that the remote 
>> end is sending some packet.
>> 
>> Ping However is not working (you used: 200.143.232.255/31 )
>> 
>> RIO0001#sh run sdn7                                                        
>>                                                                            
>>                        
>> interface sdn7
>> description WAN PORT -> JUNIPER
>> mtu 1500
>> macaddr 0014.127b.3d43
>> lldp enable
>> vrf forwarding inet
>> ipv4 address 200.143.232.255 255.255.255.254
>> no shutdown
>> log-link-change
>> exit
>> 
>> RIO0001#ping 200.143.232.255 /vrf inet                                     
>>                                                                            
>>                        
>> pinging 200.143.232.255, src=null, vrf=inet, cnt=5, len=64, tim=1000, 
>> gap=0, ttl=255, tos=0, fill=0, sweep=false, multi=false, detail=false
>> !!!!!
>> result=100%, recv/sent/lost/err=5/5/0/0, rtt min/avg/max/total=0/0/1/2
>> RIO0001#ping 200.143.232.254 /vrf inet                                     
>>                                                                            
>>                        
>> pinging 200.143.232.254, src=null, vrf=inet, cnt=5, len=64, tim=1000, 
>> gap=0, ttl=255, tos=0, fill=0, sweep=false, multi=false, detail=false
>> .....
>> result=0%, recv/sent/lost/err=0/5/5/0, rtt min/avg/max/total=10000/0/0/5002
>> RIO0001#ping 200.143.232.254 /vrf inet                                     
>>                                                                            
>>                        
>> pinging 200.143.232.254, src=null, vrf=inet, cnt=5, len=64, tim=1000, 
>> gap=0, ttl=255, tos=0, fill=0, sweep=false, multi=false, detail=false
>> .....
>> result=0%, recv/sent/lost/err=0/5/5/0, rtt min/avg/max/total=10000/0/0/5001
>> RIO0001#sh ipv4 arp sdn7                                                   
>>                                                                            
>>                        
>> mac  address  time  static
>> 
>> 
>> À bientôt,
>> --  Frederic
>> 
>> 
>> 
>> 
>> 
>> Le 21 août 2021 à 12:39, mc36 <> a écrit :
>> 
>> hi,
>> as far as i know, for 100g, the setting the fec is mandatory...
>> regards,
>> cs
>> 
>> 
>> On 8/20/21 3:39 PM, Marcos Felipe Schwarz wrote:
>> 
>> Just correcting that the cable is AOC not DAC.
>> Regards,
>> Marcos Schwarz
>> *De:*Marcos Felipe Schwarz
>> *Enviada em:* sexta-feira, 20 de agosto de 2021 08:52
>> *Para:* 
>> *Cc:* Pedro Diniz <>; Fernando Nazareno 
>> Nascimento Farias <>; Michael Prieto Hernandez 
>> <>
>> *Assunto:* Wedge interconnection using Juniper DAC cable.
>> Hi everyone,
>> We are using RARE OS at RNP on a EdgeCore Wedge, and we are trying to 
>> setup an interconnect from it to a Juniper Router using a 100G DAC cable 
>> from Juniper.
>> I setup the port using the configuration below, but the cort continues 
>> DOWN. IÂ’d like some help with the configuration or with some troubleshoot 
>> options to try to find the problem.
>> interface sdn7
>> lldp enable
>> vrf forwarding inet
>> ipv4 address 200.143.232.255 255.255.255.254
>> no shutdown
>> log-link-change
>> exit
>> server p4lang p4
>> export-port sdn7 180 100 0 1 0
>> exit
>> -----+----+---+----+-------+----+--+--+---+---+---+--------+----------------+----------------+-
>> PORT |MAC |D_P|P/PT|SPEEDÂ  |FEC |AN|KR|RDY|ADM|OPR|LPBKÂ Â Â  |FRAMES RXÂ 
>> Â Â Â Â Â  |FRAMES TXÂ Â Â Â Â Â  |E
>> -----+----+---+----+-------+----+--+--+---+---+---+--------+----------------+----------------+-
>> 7/0Â  |17/0|180|3/52|100GÂ Â  |NONE|Ds|Au|YES|ENB|DWN|Â  NONEÂ  |Â Â Â Â Â 
>> Â Â Â Â Â Â Â Â Â  0|Â Â Â Â Â Â Â Â Â Â Â Â Â Â  0|
>> Regards,
>> Marcos Schwarz
>> *Gerente de P&D | /R&D Manager/*
>> Gerência de Execução de P&D em Ciberinfraestrutura | /Management of R&D 
>> Execution in Cyberinfrastructure/
>> Diretoria de Pesquisa e Desenvolvimento | /Board of Research and 
>> Development/
>> RNP - Rede Nacional de Ensino e Pesquisa | /Brazilian National Research 
>> and Educational Network/
>> Promovendo o uso inovador de redes avançadas | /Promoting the innovative 
>> use of advanced networks/
>> http://www.rnp.br <http://www.rnp.br/> | +55 (19) 3787-3386 | Skype ID: 
>> marcos.f.sch**
>> Campinas - SP - Brasil | E-mail:  
>> <>
>