rare-users - Re: [RARE-users] [freertr] Freertr Install On AWS and DPDK

Subject: RARE user and assistance email list

List archive

Re: [RARE-users] [freertr] Freertr Install On AWS and DPDK

From: "Christopher Adigun" <>
To: mc36 <>
Cc:
Subject: Re: [RARE-users] [freertr] Freertr Install On AWS and DPDK
Date: Fri, 6 Jan 2023 13:59:32 -0500
List-id: <freertr.groups.io>
Mailing-list: list ; contact

Ok, I think the rtr-hw.txt needs to be updated:

hwid xxx
tcp2vrf 23 host 23
line tty1 127.0.0.1 20022 127.0.0.1 20021
tcp2vrf 2323 p4 23 127.0.0.1
tcp2vrf 9080 p4 9080 127.0.0.1
proc p4emu /rtr/p4dpdk.bin --vdev=net_af_packet0,iface=ens3 --vdev=net_af_packet1,iface=veth1b --vdev=net_af_packet2,iface=veth0b -- 127.0.0.1 9080 2
int eth0 eth - 127.0.0.1 19999 127.0.0.1 19998
proc cpuport /rtr/pcapInt.bin veth0a 19998 127.0.0.1 19999 127.0.0.1

Each DPDK interface requires unique veth interfaces right?

In the above veth1b pair is not referenced in the proc cpuport /rtr/pcapInt.bin like the way veth0b.

On Fri, Jan 6, 2023 at 12:58 PM mc36 <> wrote:

hi,
niceee!

first of all, to have a better console experience, try the following: stty raw -echo < /dev/tty && java -jar .....

then to have the 2nd interface just do the following to the console:

conf t
int sdn2
macaddr <copy-from-hypervisor>
exit
serv p4 p4
export-port sdn2 1
end
wr

also to silence the console, remove the "line tty1" from -hw.txt

br,
cs

On 1/6/23 18:53, Christopher Adigun wrote:
> I have pulled the latest RPM build and I can see SDN1 interface up now:
>
>
> /[root@ip-172-31-38-199 freerouter]# /usr/lib/jvm/jre-11-openjdk/bin/java -jar /usr/share/java/rtr.jar routersc rtr-hw.txt rtr-sw.txt
>
>
> #### ##################
> ## ##
> ## ## ### ##### ##### ## ### ## ## ###
> #### ### ## ## ## ## ## ### ## ## ### ##
> ## ## ## ####### ####### ## ## ## ## ##
> ## ## ## ## ## ## ##
> ## ## ## ## ## ## ## ## ##
> ## ## ##### ##### ## ## ##
>
> freeRouter v23.1.5-cur, done by cs@nop.
>
> place on the web: http://www.freertr.org/ <http://www.freertr.org/>
> license: http://creativecommons.org/licenses/by-sa/4.0/ <http://creativecommons.org/licenses/by-sa/4.0/>
> quote1: make the world better
> quote2: if a machine can learn the value of human life, maybe we can too
> quote3: keep it stupid simple
> quote4: be liberal in what you accept, and conservative in what you send
> quote5: the beer-ware license for selected group of people:
> cs@nop wrote these files. as long as you retain this notice you
> can do whatever you want with this stuff. if we meet some day, and
> you think this stuff is worth it, you can buy me a beer in return
>
> info cfgInit.doInit:cfgInit.java:776 booting
> info cfgInit.doInit:cfgInit.java:958 initializing hardware
> warning lineTcpLine.rxtxOpen:lineTcpLine.java:73 reopening line on /127.0.0.1 <http://127.0.0.1> 20022 /127.0.0.1 <http://127.0.0.1> 20021
> info cfgInit.doInit:cfgInit.java:964 applying defaults
> info cfgInit.doInit:cfgInit.java:985 applying configuration
> info cfgInit.executeSWcommands:cfgInit.java:764 line 15: "ipv4 address 172.31.38.199 255.255.240.0" : % no vrf membership selected
> error cfgInit.doInit:cfgInit.java:993 1 errors found
> info cfgInit.doInit:cfgInit.java:1020 boot completed
> welcome
> line ready
> ebenezer#warning lineTcpLine.rxtxOpen:lineTcpLine.java:73 reopening line on /127.0.0.1 <http://127.0.0.1> 20022 /127.0.0.1 <http://127.0.0.1> 20021
> warning lineTcpLine.rxtxOpen:lineTcpLine.java:73 reopening line on /127.0.0.1 <http://127.0.0.1> 20022 /127.0.0.1 <http://127.0.0.1> 20021
> info cfgPrcss.doRound:cfgPrcss.java:509 restarting process cpuport
> info cfgPrcss.doRound:cfgPrcss.java:509 restarting process p4emu
> warning lineTcpLine.rxtxOpen:lineTcpLine.java:73 reopening line on /127.0.0.1 <http://127.0.0.1> 20022 /127.0.0.1 <http://127.0.0.1> 20021
> warning servP4langConn.doNegot:servP4langConn.java:312 neighbor 127.0.0.1 up
> warning lineTcpLine.rxtxOpen:lineTcpLine.java:73 reopening line on /127.0.0.1 <http://127.0.0.1> 20022 /127.0.0.1 <http://127.0.0.1> 20021
> warning lineTcpLine.rxtxOpen:lineTcpLine.java:73 reopening line on /127.0.0.1 <http://127.0.0.1> 20022 /127.0.0.1 <http://127.0.0.1> 20021
> warning lineTcpLine.rxtxOpen:lineTcpLine.java:73 reopening line on /127.0.0.1 <http://127.0.0.1> 20022 /127.0.0.1 <http://127.0.0.1> 20021
> warning lineTcpLine.rxtxOpen:lineTcpLine.java:73 reopening line on /127.0.0.1 <http://127.0.0.1> 20022 /127.0.0.1 <http://127.0.0.1> 20021
> warning lineTcpLine.rxtxOpen:lineTcpLine.java:73 reopening line on /127.0.0.1 <http://127.0.0.1> 20022 /127.0.0.1 <http://127.0.0.1> 20021
> warning lineTcpLine.rxtxOpen:lineTcpLine.java:73 reopening line on /127.0.0.1 <http://127.0.0.1> 20022 /127.0.0.1 <http://127.0.0.1> 20021
> show warning lineTcpLine.rxtxOpen:lineTcpLine.java:73 reopening line on /127.0.0.1 <http://127.0.0.1> 20022 /127.0.0.1 <http://127.0.0.1> 20021
> int
> ebenezer#show int
> ethernet0 is up, promisc
> description: p4 cpu port
> state changed 3 times, last at 2023-01-06 17:23:15, 00:00:10 ago
> last packet input never ago, output never ago, drop never ago
> type is ethernet, hwaddr=004f.2740.6100, mtu=1500, bw=100mbps
> received 0 packets (0 bytes) dropped 0 packets (0 bytes)
> transmitted 0 packets (0 bytes) macsec=false sgt=false/
> /sdn1 is up
> description:
> state changed 3 times, last at 2023-01-06 17:23:18, 00:00:07 ago
> last packet input never ago, output never ago, drop never ago
> type is sdn, hwaddr=0a18.f686.994b, mtu=9001, bw=8000kbps, vrf=host
> ipv4 address=172.31.131.170/20 <http://172.31.131.170/20>, mask=255.255.240.0, ifcid=361949270
> received 0 packets (0 bytes) dropped 0 packets (0 bytes)
> transmitted 0 packets (0 bytes) macsec=false sgt=false/
>
> ebenezer#warning lineTcpLine.rxtxOpen:lineTcpLine.java:73 reopening line on /127.0.0.1 <http://127.0.0.1> 20022 /127.0.0.1 <http://127.0.0.1> 20021
> warning lineTcpLine.rxtxOpen:lineTcpLine.java:73 reopening line on /127.0.0.1 <http://127.0.0.1> 20022 /127.0.0.1 <http://127.0.0.1> 20021
>
> /ebenezer#sho p4 p4 stat/
> /category value
> peer 127.0.0.1
> closed 0
> reconn 1
> since 2023-01-06 17:23:18
> for 00:09:37
> capability punting copp acl nat vlan bundle bridge pppoe hairpin gre l2tp route mpls vpls evpn eompls gretap pppoetap l2tptap vxlan ipip pckoudp srv6 pbr qos flwspc mroute
> duplab bier amt nsh polka racl inspect mpolka sgt vrfysrc gtp loconn tcpmss pmtud mlppp macsec ipsec openvpn wireguard
> platform p4emu/dpdk
> cpuport 2
> dynamic ifc 128 65535
> dynamic vrf 1 65535
> messages sent 46
> messages got 4976
> rounds done 588
> last done 2023-01-06 17:32:55 (00:00:00 ago)
> time took 0
> rounds skip 0
> last skip 1970-01-01 00:00:00 (never ago)
>
> ebenezer#
> ebenezer#
> ebenezer#sho p4 p4 port-na
> num name
> 0 0000:00:06.0
> 1 0000:00:07.0
> 2 net_af_packet1/
>
>
> How do I add the second DPDK interface (0000:00:07.0) in the rtr-hw.txt and rtr-sw.txt?
>
> Thanks
>
> On Thu, Jan 5, 2023 at 1:53 PM mc36 < <mailto:>> wrote:
>
> hi,
> i used to add these packages: clang, dpdk-dev, libpcap-dev, libbpf-dev, libmnl-dev and libssl-dev
> br,
> cs
>
> On 1/5/23 19:46, Christopher Adigun wrote:
> > Thanks..
> >
> > For future reference, can we get the full steps for compiling.. Encountered errors while attempting to build it (tried centos, fedora), I already installed the
> prerequisites you
> > mentioned.
> >
> > [fedora@ip-172-31-43-100 native]$ ./c.sh
> > arch=x86_64, cc=clang, mode=-O3, flag=-march=corei7, out=../../binTmp
> > compiling p4xdp_pass.
> > p4xdp_pass.c:2:10: fatal error: 'bpf/bpf_helpers.h' file not found
> > #include <bpf/bpf_helpers.h>
> > ^~~~~~~~~~~~~~~~~~~
> > 1 error generated.
> > compiling p4xdp_drop.
> > p4xdp_drop.c:2:10: fatal error: 'bpf/bpf_helpers.h' file not found
> > #include <bpf/bpf_helpers.h>
> > ^~~~~~~~~~~~~~~~~~~
> > 1 error generated.
> > compiling p4xdp_kern.
> > p4xdp_kern.c:2:10: fatal error: 'bpf/bpf_helpers.h' file not found
> > #include <bpf/bpf_helpers.h>
> > ^~~~~~~~~~~~~~~~~~~
> > 1 error generated.
> > compiling p4mnl_kern.
> > p4mnl_kern.c:2:10: fatal error: 'bpf/bpf_helpers.h' file not found
> > #include <bpf/bpf_helpers.h>
> > ^~~~~~~~~~~~~~~~~~~
> > 1 error generated.
> > compiling p4xdp_user.
> > p4xdp_user.c:12:10: fatal error: 'bpf/bpf.h' file not found
> > #include <bpf/bpf.h>
> > ^~~~~~~~~~~
> > 1 error generated.
> > compiling p4mnl_user.
> > p4mnl_user.c:13:10: fatal error: 'bpf/bpf.h' file not found
> > #include <bpf/bpf.h>
> > ^~~~~~~~~~~
> > 1 error generated.
> > precompiling p4emu_full.
> > In file included from p4emu_full.c:12:
> > ./p4emu_hdr.h:3:10: fatal error: 'openssl/conf.h' file not found
> > #include <openssl/conf.h>
> > ^~~~~~~~~~~~~~~~
> > 1 error generated.
> > precompiling p4emu_dbg.
> > In file included from p4emu_dbg.c:12:
> > ./p4emu_hdr.h:3:10: fatal error: 'openssl/conf.h' file not found
> > #include <openssl/conf.h>
> > ^~~~~~~~~~~~~~~~
> > 1 error generated.
> > precompiling p4emu_none.
> > In file included from p4emu_none.c:9:
> > ./p4emu_hdr.h:3:10: fatal error: 'openssl/conf.h' file not found
> > #include <openssl/conf.h>
> > ^~~~~~~~~~~~~~~~
> > 1 error generated.
> > precompiling p4emu_pcap.
> > p4emu_pcap.c:9:10: fatal error: 'pcap.h' file not found
> > #include <pcap.h>
> > ^~~~~~~~
> > 1 error generated.
> > precompiling p4emu_bench.
> > In file included from p4emu_bench.c:10:
> > ./p4emu_hdr.h:3:10: fatal error: 'openssl/conf.h' file not found
> > #include <openssl/conf.h>
> > ^~~~~~~~~~~~~~~~
> > 1 error generated.
> > precompiling p4emu_udp.
> > In file included from p4emu_udp.c:11:
> > ./p4emu_hdr.h:3:10: fatal error: 'openssl/conf.h' file not found
> > #include <openssl/conf.h>
> > ^~~~~~~~~~~~~~~~
> > 1 error generated.
> > precompiling p4emu_dpdk.
> > p4emu_dpdk.c:9:10: fatal error: 'dpdk/rte_config.h' file not found
> > #include <dpdk/rte_config.h>
> > ^~~~~~~~~~~~~~~~~~~
> > 1 error generated.
> > linking p4emu.
> > /usr/bin/ld: cannot find -lpcap
> > /usr/bin/ld: cannot find -lcrypto
> > clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
> > linking p4dbg.
> > /usr/bin/ld: cannot find -lpcap
> > /usr/bin/ld: cannot find -lcrypto
> > clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
> > linking p4pkt.
> > /usr/bin/ld: cannot find -lpcap
> > /usr/bin/ld: cannot find -lcrypto
> > clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
> > linking p4dpdk.
> > /usr/bin/ld: cannot find -lcrypto
> > /usr/bin/ld: cannot find -lrte_eal
> > /usr/bin/ld: cannot find -lrte_mempool
> > /usr/bin/ld: cannot find -lrte_mbuf
> > /usr/bin/ld: cannot find -lrte_ring
> > /usr/bin/ld: cannot find -lrte_ethdev
> > clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
> > linking p4dpdkDbg.
> > /usr/bin/ld: cannot find -lcrypto
> > /usr/bin/ld: cannot find -lrte_eal
> > /usr/bin/ld: cannot find -lrte_mempool
> > /usr/bin/ld: cannot find -lrte_mbuf
> > /usr/bin/ld: cannot find -lrte_ring
> > /usr/bin/ld: cannot find -lrte_ethdev
> > clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
> > linking p4dpdkPkt.
> > /usr/bin/ld: cannot find -lcrypto
> > /usr/bin/ld: cannot find -lrte_eal
> > /usr/bin/ld: cannot find -lrte_mempool
> > /usr/bin/ld: cannot find -lrte_mbuf
> > /usr/bin/ld: cannot find -lrte_ring
> > /usr/bin/ld: cannot find -lrte_ethdev
> > clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
> > linking p4bench.
> > /usr/bin/ld: cannot find -lcrypto
> > clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
> > linking p4udp.
> > /usr/bin/ld: cannot find -lcrypto
> > clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
> > compiling pcapInt.
> > pcapInt.c:9:10: fatal error: 'pcap.h' file not found
> > #include <pcap.h>
> > ^~~~~~~~
> > 1 error generated.
> > compiling pcap2pcap.
> > pcap2pcap.c:9:10: fatal error: 'pcap.h' file not found
> > #include <pcap.h>
> > ^~~~~~~~
> > 1 error generated.
> > compiling sender.
> > sender.c:5:10: fatal error: 'pcap.h' file not found
> > #include <pcap.h>
> > ^~~~~~~~
> > 1 error generated.
> > compiling mapInt.
> > compiling rawInt.
> > compiling tapInt.
> > compiling bundle.
> > compiling vlan.
> > compiling hdlcInt.
> > compiling stdLin.
> > compiling ttyLin.
> > compiling modem.
> > compiling ptyRun.
> > compiling dummyCon.
> > compiling daemonRun.
> >
> >
> >
> > On Thu, Jan 5, 2023 at 12:56 PM mc36 < <mailto:> <mailto: <mailto:>>> wrote:
> >
> > hi,
> > nevermind about recompiling.... i've made every error (except the memory allocations) to warning with the below change:
> > https://github.com/rare-freertr/freeRtr/commit/a59b34ecaa14ad276267cb0f5542678c219b0d71
> <https://github.com/rare-freertr/freeRtr/commit/a59b34ecaa14ad276267cb0f5542678c219b0d71>
> <https://github.com/rare-freertr/freeRtr/commit/a59b34ecaa14ad276267cb0f5542678c219b0d71 <https://github.com/rare-freertr/freeRtr/commit/a59b34ecaa14ad276267cb0f5542678c219b0d71>>
> > as far as i know the fedora is daily rebuilds itself automatically, so just wait a day and upgrade afterwards...
> > br,
> > cs
> >
> >
> > On 1/5/23 16:46, mc36 wrote:
> > > hi,
> > > let's dissect this again to two prats... the systemctl output is fully eal errors...
> > > messages starting with eal are from the dpdk... its quite cryptic but finally it starts,
> > > i know this becasue the final message is from p4dpdk noting that it needs at least 2 ports...
> > > it can be easily that without root, dpdk was not able to grab the nics...
> > > the second, when you run it by hand is more interesting...
> > > i recognize a lot of messages from p4dpdk trying to bring up the port then finally it terminates:
> > > https://github.com/rare-freertr/freeRtr/blob/master/misc/native/p4emu_dpdk.c#L542 <https://github.com/rare-freertr/freeRtr/blob/master/misc/native/p4emu_dpdk.c#L542>
> <https://github.com/rare-freertr/freeRtr/blob/master/misc/native/p4emu_dpdk.c#L542 <https://github.com/rare-freertr/freeRtr/blob/master/misc/native/p4emu_dpdk.c#L542>>
> > > here is my suggestion:
> > >
> > > sudo dnf install gcc clang dpdk-dev
> > > wget freertr.org/rtr.zip <http://freertr.org/rtr.zip> <http://freertr.org/rtr.zip <http://freertr.org/rtr.zip>>
> > > unzip rtr.zip
> > > cd misc/native
> > > editor p4emu_dpdk.c #comment out the whole line 542 completely containing the message "error setting promiscuous mode"
> > > ./c.sh
> > > sudo ../../binTmp/p4dpdk.bin --vdev=net_af_packet1,iface=veth0b -- 127.0.0.1 9080 2
> > >
> > > if it starts this way and you'll have the packets finally, i'll make it a warning,
> > > then you'll have just do your regular update to have it from the daily recompilations...
> > > thanks,
> > > cs
> > >
> > > On 1/5/23 16:01, Christopher Adigun wrote:
> > >> No improvement, the interface is still down.
> > >>
> > >> I decided to change the user from /*freerouter* /to /*root* /in the systemd file /usr/lib/systemd/system/freerouter.service, now I get the following error:
> > >>
> > >> freerouter.service - Main process of freerouter
> > >> Loaded: loaded (/usr/lib/systemd/system/freerouter.service; disabled; vendor preset: disabled)
> > >> Active: active (running) since Thu 2023-01-05 13:19:21 UTC; 4s ago
> > >> Main PID: 24397 (java)
> > >> CGroup: /system.slice/freerouter.service
> > >> 24397 /usr/lib/jvm/jre-11-openjdk/bin/java -jar /usr/share/java/rtr.jar router
> /etc/freerouter/rtr-
> > >> 24491 /bin/pcapInt.bin veth0a 19998 127.0.0.1 19999 127.0.0.1
> > >> Jan 05 13:19:23 ip-172-31-38-199.eu-west-1.compute.internal p4dpdk.bin[24486]: EAL: Requested device 0000:00:07.0 cannot be used
> > >> Jan 05 13:19:24 ip-172-31-38-199.eu-west-1.compute.internal p4dpdk.bin[24486]: TELEMETRY: No legacy callbacks, legacy socket not created
> > >> Jan 05 13:19:24 ip-172-31-38-199.eu-west-1.compute.internal p4dpdk.bin[24486]: EAL: Error - exiting with code: 1
> > >>
> > Cause:
> > >> Jan 05 13:19:25 ip-172-31-38-199.eu-west-1.compute.internal p4dpdk.bin[24857]: EAL: Cannot open VFIO container /dev/vfio/vfio, error 2 (No such file or directory)
> > >> Jan 05 13:19:25 ip-172-31-38-199.eu-west-1.compute.internal p4dpdk.bin[24857]: EAL: VFIO support could not be initialized
> > >> Jan 05 13:19:25 ip-172-31-38-199.eu-west-1.compute.internal p4dpdk.bin[24857]: EAL: Requested device 0000:00:06.0 cannot be used
> > >> Jan 05 13:19:25 ip-172-31-38-199.eu-west-1.compute.internal p4dpdk.bin[24857]: EAL: Requested device 0000:00:07.0 cannot be used
> > >> Jan 05 13:19:26 ip-172-31-38-199.eu-west-1.compute.internal p4dpdk.bin[24857]: TELEMETRY: No legacy callbacks, legacy socket not created
> > >> Jan 05 13:19:26 ip-172-31-38-199.eu-west-1.compute.internal p4dpdk.bin[24857]: EAL: Error - exiting with code: 1
> > >>
> > Cause:
> > >> Jan 05 13:19:26 ip-172-31-38-199.eu-west-1.compute.internal p4dpdk.bin[24857]: at least 2 ports needed
> > >>
> > >> So I decided to run *//bin/p4dpdk.bin --vdev=net_af_packet1,iface=veth0b -- 127.0.0.1 9080 2/* manually and I got the promiscuous error:
> > >>
> > >> /*[root@ip-172-31-38-199 log]# /bin/p4dpdk.bin --vdev=net_af_packet1,iface=veth0b -- 127.0.0.1 9080 2*/ EAL: Detected CPU lcores: 8 EAL: Detected NUMA nodes: 1 EAL:
> Detected
> > >> shared linkage of DPDK EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'PA' EAL: No available 1048576 kB hugepages reported EAL: VFIO
> support
> > >> initialized EAL: Using IOMMU type 8 (No-IOMMU) EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket 0) EAL: Probe PCI driver: net_ena (1d0f:ec20)
> device:
> > >> 0000:00:07.0 (socket 0) TELEMETRY: No legacy callbacks, legacy socket not created 8 cores and 3 ports detected... dpdk version: DPDK 21.11.2 openssl version: OpenSSL
> 1.1.1g
> > FIPS
> > >> 21 Apr 2020 connecting 127.0.0.1 9080. cpu port is #2 of 3... there will be 0 forwarding only cores... there will be 8191 mbufs, each 2048 bytes, 250 cached... there
> will
> > be 1024
> > >> rx and 1024 tx descriptors, 512 tx and 512 fwd mbufs... there will be 32 bursts and maybe 100 sleeps... opening mempool on socket 0... opening port 0 named
> 0000:00:06.0 on
> > socket
> > >> 0 on lcore 0 for rx and 0 for tx... devinfo: pmd=net_ena, mtu=128..9216, bufs=64, pktl=9234, rxque=10, txque=8, rxcapa=0000200e, txcapa=0000800e... configuring port:
> offloads
> > >> rx=00002000, tx=00008000, pktlen=2048 EAL: Error - exiting with code: 1 Cause: error setting promiscuous mode
> > >>
> > >> N.B - I have tested the dpdk interfaces in VPP and it works.
> > >>
> > >
> > >
> > >
> > >
> > >
> >
>
>
>
>
>

_._,_._,_

Groups.io Links:

You receive all messages sent to this group.

_._,_._,_

Re: [RARE-users] [freertr] Freertr Install On AWS and DPDK, (continued)

List archive

Re: [RARE-users] [freertr] Freertr Install On AWS and DPDK