Skip to Content.

rare-users - Re: [gn4-3-wp6-t1-wb-RARE] rare testing....

Subject: RARE user and assistance email list

List archive


Re: [gn4-3-wp6-t1-wb-RARE] rare testing....


Chronological Thread 
    < Chronological >      < Thread >
  • From: mc36 <>
  • To: , Tim Chown <>
  • Cc: Xavier JEANNIN <>
  • Subject: Re: [gn4-3-wp6-t1-wb-RARE] rare testing....
  • Date: Mon, 8 Mar 2021 12:42:55 +0100
hi,

On 3/8/21 11:31 AM, Tim Chown wrote:
Hi,

To recreate a realistic use case, have you tested how RARE performs when
adding additional RTT with something like tc? Say 70-100ms to pretend
we’re trying a 10gbps or even 100gps transfer to the US. And maybe throw
in some very small packet loss to enforce retransmits.

no, we did not performed such tests yet, and i personally on the opinion that
the more the tests, the better...
for now our focus here was to quickly check how tofino performs under load
with various features enabled...
and it just started, we measured routing for a day and mpls and pppoe for a
half... and we already have 4 more to go with: gre and l2tp
at the moment, the server here was only a tool, and the
vrf---dac---vrf---dac---vrf was only a poor man's spirent...


If we are considering using RARE for something like DMC, this sort of testing
will be useful. Raul can also look at this in the Jisc lab once we’ve got
physical access again.

and unfortunately just turned out that jordi have issues staying in umu so if
you have resources

at jisc to continue the testing, even in paralell with jordi, i would be glad
to hear about that!

thanks,
cs



Tim

On 6 Mar 2021, at 05:01, mc36 <> wrote:

hi,
we did 3.5 petabytes in the past 9.5 hours... it's a constant ~1tbps...
and, i just asked jordi to start the iperf in the other direction too...
let's see how tofino performs at 2tbps... :)
regards,
cs



wedge32#show platform | include up
uptime: since 2021-03-05 20:32:36, for 09:25:58
reload: code#2=upgrade finished

wedge32#show interfaces hwsummary | include up | summary
|~~~~~~~~|~~~~~~~~~~~~~~~~~~|~~~~~~~~~~~~~~~~~|
| column | summary | average |
|--------|------------------|-----------------|
| col0 | 0 | 0 |
| col1 | 0 | 0 |
| col2 | 3544198189299602 | 161099917695436 |
| col3 | 3518740514355259 | 159942750652511 |
| col4 | 0 | 0 |
|________|__________________|_________________|

wedge32#show interfaces hwtraffic | include up | summary
|~~~~~~~~|~~~~~~~~~~~~~|~~~~~~~~~~~~|
| column | summary | average |
|--------|-------------|------------|
| col0 | 0 | 0 |
| col1 | 0 | 0 |
| col2 | 90349835398 | 4106810699 |
| col3 | 89688649248 | 4076756784 |
| col4 | 0 | 0 |
|________|_____________|____________|

wedge32#show interfaces hwsummary | include up
| sdn10 | up | 322298036458272 | 1962862 | 0 |
| sdn11 | up | 322298048119971 | 1958796 | 0 |
| sdn12 | up | 0 | 319976098901523 | 0 |
| sdn13 | up | 0 | 319976018974458 | 0 |
| sdn14 | up | 322298048259114 | 1962227 | 0 |
| sdn15 | up | 322297986980604 | 1964487 | 0 |
| sdn16 | up | 2360842131 | 319975981009535 | 0 |
| sdn17 | up | 2790 | 319975981005468 | 0 |
| sdn18 | up | 322297923734442 | 1966547 | 0 |
| sdn19 | up | 322297957658274 | 1971270 | 0 |
| sdn2 | up | 0 | 0 | 0 |
| sdn20 | up | 3962 | 319975981002751 | 0 |
| sdn21 | up | 3962 | 319975980990948 | 0 |
| sdn22 | up | 322297910212818 | 1967185 | 0 |
| sdn23 | up | 322297971920229 | 1972948 | 0 |
| sdn24 | up | 1609 | 319975980993379 | 0 |
| sdn25 | up | 1423 | 319975981003827 | 0 |
| sdn26 | up | 322298017349115 | 1963328 | 0 |
| sdn5 | up | 0 | 320052252643048 | 0 |
| sdn6 | up | 322295689121433 | 252470 | 0 |
| sdn7 | up | 322298181599103 | 1955343 | 0 |
| sdn8 | up | 0 | 319976260377350 | 0 |
| sdn9 | up | 0 | 319976127585898 | 0 |

wedge32#show interfaces hwtraffic | include up
| sdn10 | up | 9894531523 | 54 | 0 |
| sdn11 | up | 9894531829 | 54 | 0 |
| sdn12 | up | 0 | 9823086692 | 0 |
| sdn13 | up | 0 | 9823086389 | 0 |
| sdn14 | up | 9894533356 | 54 | 0 |
| sdn15 | up | 9894533967 | 54 | 0 |
| sdn16 | up | 1108 | 9823085176 | 0 |
| sdn17 | up | 0 | 9823085782 | 0 |
| sdn18 | up | 9894533661 | 54 | 0 |
| sdn19 | up | 9894534272 | 54 | 0 |
| sdn2 | up | 0 | 0 | 0 |
| sdn20 | up | 0 | 9823086389 | 0 |
| sdn21 | up | 0 | 9823086085 | 0 |
| sdn22 | up | 9894533356 | 54 | 0 |
| sdn23 | up | 9894533050 | 54 | 0 |
| sdn24 | up | 0 | 9823085479 | 0 |
| sdn25 | up | 0 | 9823085176 | 0 |
| sdn26 | up | 9894533661 | 54 | 0 |
| sdn5 | up | 0 | 9823085728 | 0 |
| sdn6 | up | 9894534883 | 0 | 0 |
| sdn7 | up | 9894530913 | 54 | 0 |
| sdn8 | up | 0 | 9823085782 | 0 |
| sdn9 | up | 0 | 9823087601 | 0 |

wedge32#


On 3/5/21 1:36 PM, mc36 wrote:
sorry, not mbps.... khm.... 891 gbps! :)
On 3/5/21 1:35 PM, mc36 wrote:
hi,
please find an other intermediate result, now we're doing 891mbps in 2 pipes
of tofino,
and in rare's cli, also you can spot the same, but in bytes/seconds...
simply copy-paste the second table, paste it to zzz.csv and open in excel and
do the sum to see the result....
(and in the meantime i'll just started implementing "show anything | summary"
to make it more convenient...:)
regards,
cs


wedge32#terminal tablemode fancy
wedge32#show interfaces hwtraffic
  |~~~~~~~~~~~|~~~~~~~|~~~~~~~~~~~~~|~~~~~~~~~~~~~|~~~~~~|
  | interface | state | tx         | rx    Â
    | drop |
  |-----------|-------|-------------|-------------|------|
  | sdn1     | admin | 0          | 0Â
         | 0   |
  | sdn10    | up   | 10472892852 | 54     Â
   | 0   |
  | sdn11    | up   | 10472893157 | 54     Â
   | 0   |
  | sdn12    | up   | 0          |
10399062695 | 0Â Â Â |
  | sdn13    | up   | 0          |
10399062696 | 0Â Â Â |
  | sdn14    | up   | 10472892852 | 56     Â
   | 0   |
  | sdn15    | up   | 10472892852 | 56     Â
   | 0   |
  | sdn16    | up   | 1126       |
10399064213 | 0Â Â Â |
  | sdn17    | up   | 0          |
10399063606 | 0Â Â Â |
  | sdn18    | up   | 10472892852 | 56     Â
   | 0   |
  | sdn19    | up   | 10472893462 | 54     Â
   | 0   |
  | sdn2     | up   | 0          |
0Â Â Â Â Â Â Â Â Â Â | 0Â Â Â |
  | sdn20    | up   | 0          |
10399064211 | 0Â Â Â |
  | sdn21    | up   | 0          |
10399064513 | 0Â Â Â |
  | sdn22    | up   | 10472893157 | 54     Â
   | 0   |
  | sdn23    | up   | 10472894073 | 54     Â
   | 0   |
  | sdn24    | up   | 0          |
10399064513 | 0Â Â Â |
  | sdn25    | up   | 0          |
10399063907 | 0Â Â Â |
  | sdn26    | up   | 10472892852 | 54     Â
   | 0   |
  | sdn32    | admin | 0          | 0 Â
        | 0   |
  | sdn5     | up   | 0          |
10399057485 | 0Â Â Â |
  | sdn6     | up   | 10472893157 | 0     Â
    | 0   |
  | sdn7     | up   | 10472891325 | 54     Â
   | 0   |
  | sdn8     | up   | 0          |
10399058753 | 0Â Â Â |
  | sdn9     | up   | 0          |
10399061481 | 0Â Â Â |
  |___________|_______|_____________|_____________|______|

wedge32#terminal tablemode csv
wedge32#show interfaces hwtraffic
interface;state;tx        ;rx        ;drop
sdn1Â Â Â Â ;admin;0Â Â Â Â Â Â Â Â Â ;0Â Â Â Â Â Â Â
  ;0
sdn10   ;up  ;10453863378;54        ;0
sdn11   ;up  ;10453863378;54        ;0
sdn12   ;up  ;0         ;10379147609;0
sdn13   ;up  ;0         ;10379147609;0
sdn14   ;up  ;10453861545;54        ;0
sdn15   ;up  ;10453860629;54        ;0
sdn16   ;up  ;1111      ;10379147913;0
sdn17   ;up  ;0         ;10379147912;0
sdn18   ;up  ;10453859713;54        ;0
sdn19   ;up  ;10453858797;54        ;0
sdn2    ;up  ;0         ;0     Â
   ;0
sdn20   ;up  ;0         ;10379147913;0
sdn21   ;up  ;0         ;10379147912;0
sdn22   ;up  ;10453858186;54        ;0
sdn23   ;up  ;10453856964;54        ;0
sdn24   ;up  ;0         ;10379147305;0
sdn25   ;up  ;0         ;10379150337;0
sdn26   ;up  ;10453859713;54        ;0
sdn32Â Â Â ;admin;0Â Â Â Â Â Â Â Â Â ;0Â Â Â Â Â Â Â Â
 ;0
sdn5    ;up  ;0         ;10379149693;0
sdn6    ;up  ;10453859102;0         ;0
sdn7    ;up  ;10453865210;54        ;0
sdn8    ;up  ;0         ;10379149732;0
sdn9    ;up  ;0         ;10379146699;0

wedge32#
wedge32#attach process bfswd


bf-sde.pm> rate-show
-----+----+---+----+-------+---+-------+-------+---------+---------+----+----
PORT |MAC |D_P|P/PT|SPEED |RDY|RX Mpps|TX Mpps|RX Mbps |TX Mbps |RX
%|TX %
-----+----+---+----+-------+---+-------+-------+---------+---------+----+----
5/0Â |19/0|164|3/36|100GÂ Â |UP |Â Â 6.59|Â Â 0.00| 81074.55|Â Â
  0.00| 81%| 0%
6/0Â |18/0|172|3/44|100GÂ Â |UP |Â Â 0.00|Â Â 6.59|Â Â Â Â
0.00| 81086.94|Â 0%| 81%
7/0Â |17/0|180|3/52|100GÂ Â |UP |Â Â 0.00|Â Â 6.59|Â Â Â Â
0.00| 81088.38|Â 0%| 81%
8/0Â |16/0|188|3/60|100GÂ Â |UP |Â Â 6.59|Â Â 0.00| 81090.89|Â Â
  0.00| 81%| 0%
9/0Â |15/0| 56|1/56|100GÂ Â |UP |Â Â 6.59|Â Â 0.00| 81096.41|Â Â
  0.00| 81%| 0%
10/0 |14/0| 48|1/48|100GÂ Â |UP |Â Â 0.00|Â Â 6.59|Â Â Â Â
0.00| 81094.57|Â 0%| 81%
11/0 |13/0| 40|1/40|100GÂ Â |UP |Â Â 0.00|Â Â 6.59|Â Â Â Â
0.00| 81094.63|Â 0%| 81%
12/0 |12/0| 32|1/32|100GÂ Â |UP |Â Â 6.59|Â Â 0.00| 81091.92|Â Â
  0.00| 81%| 0%
13/0 |11/0| 24|1/24|100GÂ Â |UP |Â Â 6.59|Â Â 0.00| 81087.92|Â Â
  0.00| 81%| 0%
14/0 |10/0| 16|1/16|100GÂ Â |UP |Â Â 0.00|Â Â 6.59|Â Â Â Â
0.00| 81087.79|Â 0%| 81%
15/0 | 9/0|Â 8|1/ 8|100GÂ Â |UP |Â Â 0.00|Â Â 6.59|Â Â Â Â
0.00| 81082.34|Â 0%| 81%
16/0 | 8/0|Â 0|1/ 0|100GÂ Â |UP |Â Â 6.59|Â Â 0.00| 81081.42|Â Â
  0.00| 81%| 0%
17/0 | 7/0|Â 4|1/ 4|100GÂ Â |UP |Â Â 6.59|Â Â 0.00| 81084.60|Â Â
  0.00| 81%| 0%
18/0 | 6/0| 12|1/12|100GÂ Â |UP |Â Â 0.00|Â Â 6.59|Â Â Â Â
0.00| 81097.20|Â 0%| 81%
19/0 | 5/0| 20|1/20|100GÂ Â |UP |Â Â 0.00|Â Â 6.60|Â Â Â Â
0.00| 81111.92|Â 0%| 81%
20/0 | 4/0| 28|1/28|100GÂ Â |UP |Â Â 6.60|Â Â 0.00| 81131.37|Â Â
  0.00| 81%| 0%
21/0 | 3/0| 36|1/36|100GÂ Â |UP |Â Â 6.60|Â Â 0.00| 81153.14|Â Â
  0.00| 81%| 0%
22/0 | 2/0| 44|1/44|100GÂ Â |UP |Â Â 0.00|Â Â 6.60|Â Â Â Â
0.00| 81152.80|Â 0%| 81%
23/0 | 1/0| 52|1/52|100GÂ Â |UP |Â Â 0.00|Â Â 6.60|Â Â Â Â
0.00| 81154.04|Â 0%| 81%
24/0 | 0/0| 60|1/60|100GÂ Â |UP |Â Â 6.60|Â Â 0.00| 81156.22|Â Â
  0.00| 81%| 0%
25/0 |31/0|184|3/56|100GÂ Â |UP |Â Â 6.60|Â Â 0.00| 81165.10|Â Â
  0.00| 81%| 0%
26/0 |30/0|176|3/48|100GÂ Â |UP |Â Â 0.00|Â Â 6.60|Â Â Â Â
0.00| 81169.09|Â 0%| 81%
-----+----+---+----+-------+---+-------+-------+---------+---------+----+----
Pipe1| RX RATE  52.75Mpps 648883Mbps| TX RATE  52.75MppsÂ
648875Mbps
Pipe3| RX RATE  19.78Mpps 243330Mbps| TX RATE  19.78MppsÂ
243344Mbps
bf-sde.pm>
connection closed
wedge32#


On 3/5/21 10:19 AM, mc36 wrote:
hi,
some intermediate result, if you're interested, still some more dacs to apply
and nmaas to set up
and the iperf need to be tuned for 100g instead of 50, but the idea works as
expected, and at the
current 50gbps on each port, i see no issues at all..... we use this config
generator:
http://src.nop.hu/misc/tests/traffic-chain-ports.tcl
and it's the yesterday baked image, and just for fun, we do over the reserved
ip range...
regrads,
cs



rare#show interfaces hwtraffic
interface state tx         rx       Â
 drop
sdn1      admin 0          0   Â
      0
sdn10     up    4617446945 79       Â
 0
sdn11     up    4617446690 79       Â
 0
sdn12     up    0         Â
4583133380Â 0
sdn13     up    0         Â
4583133379Â 0
sdn14     up    4617446181 91       Â
 0
sdn15     up    4617446690 79       Â
 0
sdn16     up    1313       4583132889 0
sdn17     up    0         Â
4583132621Â 0
sdn18     up    4617447199 91       Â
 0
sdn19     up    4617446945 56       Â
 0
sdn2      up    0          0 Â
        0
sdn20     up    0         Â
4583133371Â 0
sdn21     up    0          56 Â
       0
sdn22     up    0          69 Â
       0
sdn23     up    0          0  Â
       0
sdn24     up    0          0  Â
       0
sdn25     up    0          0  Â
       0
sdn26     up    0          0  Â
       0
sdn32     admin 0          0    Â
     0
sdn5      up    0         Â
4583133563Â 0
sdn6      up    4617447199 0       Â
  0
sdn7      up    4617447708 79       Â
 0
sdn8      up    0         Â
4583133632Â 0
sdn9      up    0         Â
4583133632Â 0

rare#
rare#show lldp neighbor
interface hostname iface          ipv4   Â
    ipv6
sdn10     rare     sdn9          Â
253.252.2.1Â null
sdn11     rare     sdn12         Â
253.252.3.2Â null
sdn12     rare     sdn11         Â
253.252.3.1Â null
sdn13     rare     sdn14         Â
253.252.4.2Â null
sdn14     rare     sdn13         Â
253.252.4.1Â null
sdn15     rare     sdn16         Â
253.252.5.2Â null
sdn16     rare     sdn15         Â
253.252.5.1Â null
sdn17     rare     sdn18         Â
253.252.6.2Â null
sdn18     rare     sdn17         Â
253.252.6.1Â null
sdn19     rare     sdn20         Â
253.252.7.2Â null
sdn20     rare     sdn19         Â
253.252.7.1Â null
sdn21     rare     sdn22         Â
253.252.8.2Â null
sdn22     rare     sdn21         Â
253.252.8.1Â null
sdn5                0c42.a11e.786e nullÂ
       null
sdn7      rare     sdn8         Â
 253.252.1.2 null
sdn8      rare     sdn7         Â
 253.252.1.1 null
sdn9      rare     sdn10         Â
253.252.2.2Â null

rare#show ipv4 route big101
typ  prefix         metric iface hop   Â
     time
C    10.10.1.0/24   0/0    sdn5  null  Â
     00:07:33
LOC  10.10.1.1/32   0/1    sdn5  null    Â
   00:07:33
P EXÂ 10.10.2.0/24Â Â Â 80/70Â Â sdn7Â Â 253.252.1.2Â 00:07:12
C    253.252.1.0/30 0/0    sdn7  null    Â
   00:09:27
LOC  253.252.1.1/32 0/1    sdn7  null     Â
  00:09:27
PÂ Â Â Â 253.252.2.0/30Â 80/10Â Â sdn7Â Â 253.252.1.2Â 00:09:17
PÂ Â Â Â 253.252.3.0/30Â 80/20Â Â sdn7Â Â 253.252.1.2Â 00:09:16
PÂ Â Â Â 253.252.4.0/30Â 80/30Â Â sdn7Â Â 253.252.1.2Â 00:09:16
PÂ Â Â Â 253.252.5.0/30Â 80/40Â Â sdn7Â Â 253.252.1.2Â 00:09:16
PÂ Â Â Â 253.252.6.0/30Â 80/50Â Â sdn7Â Â 253.252.1.2Â 00:09:16
PÂ Â Â Â 253.252.7.0/30Â 80/60Â Â sdn7Â Â 253.252.1.2Â 00:09:16
PÂ Â Â Â 253.252.8.0/30Â 80/70Â Â sdn7Â Â 253.252.1.2Â 00:09:16
PÂ Â Â Â 253.252.9.0/30Â 80/80Â Â sdn7Â Â 253.252.1.2Â 00:09:16

rare#
rare#show vrf routing
                 ifc    uni Â
  mlt    flw    lab    con
name   rd      v4 v6 v4 v6 v4 v6 v4Â
v6Â v4Â v6Â v4Â v6
big101Â 101:101Â 2Â Â 0Â Â 13Â 0Â Â 13Â 0Â Â 0Â Â 0Â
 0  0  2  0
big102Â 103:102Â 2Â Â 0Â Â 13Â 0Â Â 13Â 0Â Â 0Â Â 0Â
 0  0  2  0
big103Â 105:103Â 2Â Â 0Â Â 13Â 0Â Â 13Â 0Â Â 0Â Â 0Â
 0  0  2  0
big104Â 107:104Â 2Â Â 0Â Â 13Â 0Â Â 13Â 0Â Â 0Â Â 0Â
 0  0  2  0
big105Â 109:105Â 2Â Â 0Â Â 13Â 0Â Â 13Â 0Â Â 0Â Â 0Â
 0  0  2  0
big106Â 110:106Â 1Â Â 0Â Â 2Â Â 0Â Â 2Â Â 0Â Â 0Â Â
0Â Â 0Â Â 0Â Â 1Â Â 0
big501Â 102:501Â 2Â Â 0Â Â 13Â 0Â Â 13Â 0Â Â 0Â Â 0Â
 0  0  2  0
big502Â 104:502Â 2Â Â 0Â Â 13Â 0Â Â 13Â 0Â Â 0Â Â 0Â
 0  0  2  0
big503Â 106:503Â 2Â Â 0Â Â 13Â 0Â Â 13Â 0Â Â 0Â Â 0Â
 0  0  2  0
big504Â 108:504Â 3Â Â 0Â Â 14Â 0Â Â 14Â 0Â Â 0Â Â 0Â
 0  0  3  0
big505Â 110:505Â 2Â Â 0Â Â 4Â Â 0Â Â 4Â Â 0Â Â 0Â Â
0Â Â 0Â Â 0Â Â 2Â Â 0
lin    0:0     1  0  2  0  2 Â
0Â Â 0Â Â 0Â Â 0Â Â 0Â Â 1Â Â 0
oob    0:0     1  1  4  2  4 Â
2Â Â 0Â Â 0Â Â 0Â Â 0Â Â 1Â Â 1
p1p1Â Â Â 0:0Â Â Â Â Â 0Â Â 0Â Â 0Â Â 0Â Â 0Â Â 0Â
 0  0  0  0  0  0
p1p2Â Â Â 0:0Â Â Â Â Â 0Â Â 0Â Â 0Â Â 0Â Â 0Â Â 0Â
 0  0  0  0  0  0
p4Â Â Â Â Â 0:0Â Â Â Â Â 0Â Â 0Â Â 0Â Â 0Â Â 0Â Â
0Â Â 0Â Â 0Â Â 0Â Â 0Â Â 0Â Â 0

rare#



On 3/5/21 7:24 AM, Xavier JEANNIN wrote:
Hello Csaba,

This is the kind of test that will reassure the R&E community

Thank you
Xavier Jeannin

Le 05/03/2021 à 07:06, mc36 a écrit :
hi,
jordi "accidentally" "forgot" an iperf on his server yesterday evening
so we did the 162tb in the past 8 hours... today plan is to melt the
tofino a bit by passing traffic back-and-forth between it's ports and
vrfs to reach and maintain the 1tbps forwarding rate for a while...
stay tuned, hopefully we'll some (good) news on the topic soon! :)
regards,
cs



the idea:
namespace1----[dac]---port1(vrf1)----<routing>-----port2(vrf1)-----[dac]------port3(vrf2)----<routing>-----port4(vrf2)----[dac]-----port5(vrf3)----<routing>-----port6(vrf3)--------namespace2



the setting in the attachment.... thx jordi again!




rare#show interfaces hwtraffic | exclude 0\s+0\s+0
interface state tx         rx       Â
 drop
sdn5      up    0         Â
5450777393Â 0
sdn6      up    5489159734 0       Â
  0

rare#
rare#show interfaces hwsummary | exclude 0\s+0\s+0
interface state tx              rx  Â
           drop
sdn5      up    0            Â
  162672737494416 0
sdn6      up    163852570013292 235344    Â
     0

rare#



Archive powered by MHonArc 2.6.19.

Top of Page